Evaluation and outcome prediction of vulnerable plaques in coronary artery disease

Abstract

Systems and methods for vulnerable plaque assessment and outcome prediction in coronary artery disease. Medical imaging data is used to generate a coronary tree model of a coronary centerline of the patient. The coronary artery tree model includes a plurality of nodes representing locations in the coronary artery tree model. A feature embedding associated with each node is determined from a plurality of features derived from medical imaging data. The feature embedding is input into a trained graph neural network configured to output an assessment of vulnerable plaques at a node level, a segment level, and / or a coronary tree level.

Description

Technical Field

[0001] The present disclosure relates to medical imaging and assessment of coronary artery disease. Background Art

[0002] Coronary artery disease is a common heart condition in which the main blood vessels supplying the heart (coronary arteries) struggle to deliver enough blood, oxygen, and nutrients to the heart muscle. Coronary artery disease begins when fat, cholesterol, and other substances build up on the inner walls of the heart's arteries, forming plaque. This condition is called atherosclerosis. This plaque causes the arteries to narrow, restricting blood flow. Plaque can also rupture, causing blood clots.

[0003] Coronary artery stenosis is a narrowing of the coronary lumen caused by atherosclerotic lesions. Treatment for coronary artery disease typically involves lifestyle changes, such as not smoking, eating a healthy diet, and exercising more. Sometimes, medication and surgery are required. The main goals of treating coronary artery stenosis are: (i) improving the patient's quality of life and (ii) minimizing the risk of future cardiac events. Treatment decisions are primarily based on the functional or anatomical significance of the stenosis. Generally, a type of plaque in the coronary arteries that is more likely to rupture or cause blood clots is called a "vulnerable plaque." Vulnerable plaques can be present in both significant and non-significant coronary artery lesions. A small percentage of stenoses are functionally and / or anatomically insignificant but may cause MACE (major adverse cardiovascular events). As such, functional or anatomical significance is not the sole determinant of future adverse events. A recent prospective study even demonstrated that non-flow-limiting lesions with vulnerable characteristics were the cause of the majority of adverse events during follow-up. Accurately identifying the vulnerability of coronary plaques can lead to better medical diagnosis and can benefit patient health outcomes. Summary of the Invention

[0004] By way of introduction, the preferred embodiments described below include methods, systems, instructions, and computer-readable media for assessing coronary plaque vulnerability using a graph neural network based on coronary tree-level data and patient-level data.

[0005] In a first aspect, a method for vulnerable plaque assessment and outcome prediction in coronary artery disease is provided. The method comprises: acquiring medical imaging data of a patient; generating a coronary artery tree model of the patient's coronary artery centerlines from the medical imaging data, the coronary artery tree model comprising a plurality of nodes representing locations in the coronary artery tree model; determining a feature embedding associated with each node from a plurality of features derived from the medical imaging data; inputting the feature embedding into a graph neural network; and outputting an assessment of vulnerable plaque at a node level, a segment level, and / or a coronary artery tree level based on an output of the graph neural network.

[0006] The medical imaging data may be coronary computed tomography angiography (CCTA) data. In one embodiment, the method further comprises acquiring patient-level data, wherein the patient-level data is not specific to any location in the coronary artery tree model, wherein the patient-level data is input into the graph neural network. The patient-level data comprises at least one of the presence of other pathologies associated with coronary artery disease, patient demographics, patient medical history, family medical history, calcium score, total plaque burden, laboratory results, or results of a stress test.

[0007] In one embodiment, generating the coronary artery tree model comprises segmenting the medical imaging data using a thresholding method, wherein the coronary artery tree model includes all locations with a diameter greater than a given threshold of 1.0 mm. Generating the coronary artery tree model may comprise deriving a 2.5D geometry by matching 2D geometries extracted from each acquisition.

[0008] In one embodiment, determining the feature embedding includes: defining a set of features fi(x, t) for each corresponding node in the plurality of nodes, where fi refers to the feature, x refers to the 3D position in the coronary artery tree model, and t refers to time; and inputting the set of features into a machine-trained network, the machine-trained network configured to output a feature embedding for each corresponding node. The values ​​of the set of features can vary depending on the patient's condition. Multiple instances of certain features are averaged to determine the set of features.

[0009] In one embodiment, the graph neural network aggregates, processes, and transmits information between nodes of the graph neural network using a message passing mechanism. Virtual edges connecting nodes belonging to the same coronary artery segment are added to the graph neural network for use in the message passing mechanism.

[0010] In one embodiment, the assessment includes a risk score defined at the lesion or patient level, thereby assessing the risk of rupture, clot formation, or erosion within a specified timeframe. The method may further include assessing the patient's vulnerable plaque based on the assessment; and providing procedural instructions that minimize the patient's risk of developing vulnerable plaque. In one embodiment, the patient's vulnerable plaque is assessed after acquiring the medical imaging data, wherein the assessment is updated during a catheterization laboratory examination, and wherein the assessment is further updated after the catheterization laboratory examination.

[0011] In a second aspect, a system for vulnerable plaque assessment is provided. The system includes a medical imaging system, an image processing system, and an output interface. The medical imaging system is configured to acquire medical imaging data of a patient. The image processing system is configured to generate a coronary tree model of the patient's coronary artery centerlines from the medical imaging data, the coronary tree model including a plurality of nodes representing locations in the coronary tree model, the image processing system further configured to calculate or derive one or more features associated with each node from a plurality of features derived from the medical imaging data, the image processing system further configured to determine a feature embedding for each node based on the one or more features, and input the feature embedding into a graph neural network configured to generate a vulnerable plaque assessment. The output interface is configured to provide the vulnerable plaque assessment.

[0012] In a third aspect, a non-transitory computer-implemented storage medium storing machine-readable instructions executable by at least one processor is provided. The machine-readable instructions include generating a coronary tree model of a patient's coronary artery centerlines from medical imaging data, the coronary tree model comprising a plurality of nodes representing locations in the coronary tree model; determining a feature embedding associated with each node from a plurality of features derived from the medical imaging data; inputting the feature embedding into a graph neural network; and outputting an assessment of vulnerable plaque at a node level, a segment level, and / or a coronary tree level based on an output of the graph neural network.

[0013] Any one or more of the aspects described above may be used individually or in combination. These and other aspects, features, and advantages will become apparent from the following detailed description of the preferred embodiments, which is to be read in conjunction with the accompanying drawings. The present invention is defined by the following claims, and nothing in this section should be construed as limiting those claims. Additional aspects and advantages of the present invention will be discussed below in conjunction with the preferred embodiments and may be claimed later, independently or in combination. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 An example system for assessing coronary plaque vulnerability is depicted, according to one embodiment.

[0015] Figure 2 Depicting an example CT scanning system.

[0016] Figure 3 An example depicting a 3D centerline tree structure.

[0017] Figure 4 An example method for assessing coronary plaque vulnerability is depicted, according to one embodiment. DETAILED DESCRIPTION

[0018] Embodiments described herein provide systems and methods for assessing coronary plaque vulnerability at the lesion level and the risk of major adverse clinical events (MACE) at the patient level. Two types of input data are used, including coronary tree-level data and patient-level data. Feature data (coronary tree-level data) are used to generate and populate a model of the patient's coronary artery centerlines. Patient characteristics and demographic characteristics (patient-level data) are also obtained. A graphnet architecture is used to predict vulnerable plaque indicators, such as hemodynamic measurements, QCA scores, and plaque vulnerability scores. The predictions / assessments can be used to provide or support clinical decisions for patients.

[0019] As used herein, "vulnerable plaque" refers to a type of plaque in a coronary artery that is more likely to rupture or cause a blood clot, for example, it is prone to causing acute thrombosis by rupture or erosion, or causing a coronary event by rapid plaque progression leading to significant stenosis and subsequent blood flow restriction. Vulnerable plaque characteristics have been defined in previous studies and include features such as punctate calcifications, thin-capped fibroatherosclerotic plaques, napkin ring signs, eccentricity, and low-attenuation plaques. However, these aspects only define properties that indicate a higher risk of rupture and do not allow for accurate identification of plaques that are likely to rupture. Evaluation of ordinary angiographic images can better inform the operator of lesion details, such as the presence of calcification, thrombosis, or the degree of lumen obstruction as expressed in percentage of diameter stenosis. However, without the use of intracoronary imaging (such as IVUS or OCT and the most advanced CT systems (PCCT)), it is generally impossible to distinguish the morphological properties of plaques. IVUS is an invasive imaging technique that uses ultrasound to make the interior of the coronary artery wall visible. The technology uses ultrasonography, in which high-energy sound waves are radiated into tissue, and the reflections are returned to a transducer and converted into an image. OCT uses coherent near-infrared light to generate images by measuring the intensity of light returning from the vessel wall. However, the application of invasive functional and morphological assessments is limited by equipment availability, financial considerations, and extended surgical procedure durations. Accurately identifying "true" vulnerable plaques from more generally available information would potentially allow for better risk stratification of patients with coronary artery disease.

[0020] Embodiments provide systems and methods for implementing efficient deep learning-based methods, such as graph neural networks, by organizing all relevant data for vulnerable plaque assessment and outcome prediction in coronary artery disease into a tree-like structure. The disclosed embodiments can be implemented to computationally facilitate the processing of medical imaging data and thereby improve and optimize medical diagnosis. Embodiments leverage the power of artificial intelligence (AI), such as using graph neural networks, to improve the assessment of vulnerable plaque. The systems and methods provide automated interpretation of imaging data, thereby streamlining the diagnostic process. Use of the described methods and systems further limits errors by removing user errors and decisions from the process.

[0021] Figure 1A system 100 for assessing coronary plaque vulnerability is described. The system 100 includes a medical imaging device 130 and an image processing system 105. A server 140 may be provided, for example using a cloud-based system, to store or process data. The medical imaging device 130 acquires medical imaging data. The image processing system 105 uses a graph neural network to assess coronary plaque vulnerability. The graph neural network is configured to input a coronary tree model comprising a plurality of nodes and edges, wherein a node represents a location in the coronary tree and comprises a set of features fi(x, t), wherein fi refers to a feature, x refers to a 3D / 2D location, and t refers to time. The image processing system 105 determines a feature embedding associated with each node from the set of corresponding features and patient-level data for the corresponding node. The feature embedding is input into the graph neural network, which outputs diagnostic information at the node level, segment level, and / or coronary tree level.

[0022] In one example, medical imaging device 130 performs computed tomography (CT) to generate image data used by the system. Other types of scanners (e.g., MR, PET, SPECT, or other medical imaging devices) can be used. The CT scanning device is merely exemplary, and a variety of CT scanning systems can be used to collect CT data. Figure 2An example CT imaging system 130 is depicted. An object 210 (e.g., a patient 210) is placed on a table 220, which is configured to be moved to a plurality of positions through a circular opening 230 in the CT imaging system 130 via a motorized system. An X-ray source 240 (or other radiation source) and (one or more) detector elements 250 are part of the CT imaging system 130 and are configured to rotate around the subject 210 on the gantry while the subject is within the opening / aperture 230. The rotation can be combined with movement of the table to scan along the longitudinal extent of the patient 210. Alternatively, the gantry moves the source 240 and detector 250 in a spiral path around the patient 210. In the CT imaging system 130, a single rotation may take approximately one second or less. During rotation of the X-ray source 240 and / or detector, the X-ray source 240 generates a narrow, fan-shaped (or cone-shaped) X-ray beam that passes through the target portion of the body of the subject 210 being imaged. (One or more) detector elements 250 (e.g., multi-ring detector elements) are opposite the X-ray source 240 and register the X-rays passing through the body of the subject being imaged, and in the process, record snapshots used to create the image. Many different snapshots passing through the subject at many angles are collected by one or more rotations of the X-ray source 240 and / or the (one or more) detector elements 250. The image data generated by the collected snapshots is transmitted to a control unit, which stores or processes the image data based on the snapshots into one or more cross-sectional images or volumes of the interior of the subject's body (e.g., internal organs or tissues) scanned by the CT imaging system 130. Any now known or later developed CT system can be used. Other x-ray scanners, such as a CT-like C-arm scanner, can be used.

[0023] Medical imaging device 130 is configured to generate imaging data or medical images of patient 110. Imaging data or medical images are data representing two-dimensional slices or three-dimensional volumes of a subject. This data can be in any format. A three-dimensional representation can be formatted as a stack or multiple two-dimensional planes or slices. Values ​​are provided for each of multiple locations distributed in two or three dimensions. Medical imaging data is acquired as one or more data frames. A data frame represents a scanned region at a given time or time period. A data set can represent an area or volume that changes over time, such as providing a 4D representation of a subject. Although the terms image and imaging are used, the image or imaging data can be in a format prior to actual display of the image. For example, medical imaging data can be multiple scalar values ​​representing different locations in a Cartesian or polar coordinate format that is different from a display format. As another example, a medical image can be multiple red, green, and blue (e.g., RGB) values ​​that are output to a display for generating an image in the display format. A medical image can be an image that is currently or previously displayed on a display or in another format. Imaging data is a data set that can be used for imaging, such as scan data or a generated image representing a portion of a patient.

[0024] Medical imaging data or medical images are processed by an image processing system 105. The image processing system 105 includes a processor 110, a display 115, and a memory 120. The image processing system 105 can receive data from or transmit data to a server 140, which can also be configured to process images or store data for future image processing or training / storage of machine-trained models. The image processing system 105 is configured to input imaging data and output a 3D centerline coronary artery tree structure and a set of features at each location in the tree. The image processing system 105 is further configured to determine a feature embedding at each node. The image processing system 105 is further configured to input the feature embedding and the coronary artery tree structure into a trained graph neural network, which is configured to output a prediction of vulnerable plaque.

[0025] The processor 110 is a general purpose processor, a digital signal processor, a graphics processing unit, an application specific integrated circuit, a field programmable gate array, an artificial intelligence processor, a digital circuit, an analog circuit, a combination thereof, or other now known or later developed device for processing images, normalizing image data, registering image data, augmenting image data, and other steps described below. The processor is a single device, multiple devices, or a network. For more than one device, parallel or sequential division of processing can be used. The different devices that make up the processor 110 can perform different functions. In one embodiment, the processor 110 is a control processor or other processor of the medical imaging device 130. In other embodiments, the processor 110 is part of a stand-alone workstation or computer. The processor 110 operates according to stored instructions to perform the various actions described herein. The processor 110 is configured by software, design, firmware, and / or hardware to perform Figure 4 Any or all actions.

[0026] The image processing system 105 is configured to input imaging data and output a 3D centerline coronary artery tree structure. Figure 3 An example of a 3D centerline tree structure 300 consisting of centerlines 310 and nodes 320 is depicted. At each node 320, a set of features Fi(x, t) is provided. In one embodiment, the 3D model of the coronary artery centerlines includes all locations where the diameter is greater than a given threshold of 1.0 mm or 1.5 mm. The coronary artery tree model 300 can be reconstructed from coronary computed tomography angiography (CCTA) (e.g., photon counting computed tomography (PCCT) or conventional CTA) or from coronary angiography data. For example, vessel centerlines can be extracted by segmenting and thinning the vessel or by direct tracing. In one embodiment, a convolutional neural network (CNN) is provided for automatic centerline extraction. Any method can be used to generate the 3D coronary artery tree model 300 from imaging data or other acquired data. In one embodiment, the image processing system 105 determines the anatomical structure, for example, by identifying the locations of the vessels in the acquired CT data. Any currently known or later developed method can be used to determine the anatomical structure, for example, adaptive region growing and skeletonization methods. One or more seeds, such as seeds in or for a vessel, are positioned by a processor or manual input. The one or more seeds are used in region growing to find the location of the vessel tree. Skeletonization can be used to model the vessel structure, such as using lines to represent the centers of branches of the vessel tree. Using thresholds, random walks, or other methods, the coronary artery tree model 300 of the coronary artery centerline and the vessels can be segmented for calculation of anatomical features. The general model of the coronary artery tree model 300 of the coronary artery centerline can be adapted to the segmented anatomical structure or the location identified as a vessel. Any fitting method can be used.

[0027] The coronary artery tree model 300 is based on Figure 3 , t). At each location / node 320, a set of features fi(x, t) is defined by the image processing system 105, where fi refers to features, x refers to 3D position, and t refers to time. In one embodiment, rather than using 3D geometry reconstructed from multiple coronary angiography acquisitions, a 2.5D geometry can be derived by matching 2D geometry extracted from each acquisition. In this case, each feature fi can have multiple instances, i.e., one instance for each acquisition, or a unique feature value fi(x, t) can be derived by aggregating the matched corresponding features from each view. Alternatively, the features fi can be represented as a probability distribution derived from information extracted from multiple acquisitions. This can also allow quantification of the uncertainty of the predicted results.

[0028] Certain features may remain constant over time. Other features may change over time. Certain features may be obtained by using AI models or algorithms on available input data. The values ​​of these features may change depending on the patient's state (rest, exercise-congestion, intermediate state). Therefore, multiple instances of these features may be available. Derived features such as percent diameter stenosis, wall strain (min / max), time-averaged WSS may also be included. These have been shown in the past to be associated with vulnerable plaque. When angiography and / or CCTA are used, multiple image acquisitions depicting the same coronary tree or portion of a coronary tree may be available. Therefore, multiple instances of certain features may be available. These may be provided independently or mixed (e.g., by averaging).

[0029] The list of potentially relevant features includes the following features described below. Additional or fewer features may be used. Not all of the features described below may be used. Whether to calculate values ​​for certain features may be determined based on input data and computing resources. After configuring and training the model (described below), certain features may be deleted due to processing time / resources or based on their impact on the output of the model. Potential features include:

[0030] r(x,t): time-varying cavity radius at each centerline location.

[0031] rref(x,t): The time-varying reference (ie, healthy) lumen radius at each centerline location.

[0032] router_wall(x,t): The time-varying outer wall radius at each centerline location. This allows for the assessment of positive remodeling. Vulnerable plaque causes an outward bulge in the arterial wall, which is known as positive remodeling. This can be seen as an increase in the diameter of the arterial segment containing the plaque compared to the normal adjacent segment.

[0033] curvature(x,t): The time-varying curvature value at each centerline position.

[0034] stent(x): A binary variable specifying whether a stent is present at each centerline position. Alternatively, an integer type can be used, since stents can overlap and multiple stents can be present at a given centerline position.

[0035] label(x): An integer variable specifying the coronary artery segment to which the current position belongs.

[0036] stenosis(x): Can be either an integer variable specifying the unique id of the stenosis, or a floating point variable specifying the probability of stenosis at this location.

[0037] contrast(x,t): An integer variable specifying the contrast intensity at this position, which typically varies with time.

[0038] CABG(x): A binary variable specifying whether each centerline location is part of a bypass graft.

[0039] collateral(x): A binary variable specifying whether each centerline location is part of a collateral artery.

[0040] CTO(x): A binary variable specifying whether a CTO is encountered at a centerline location (usually a leaf node).

[0041] myocardialBridging(x): A binary variable specifying whether a myocardial bridge was observed at the centerline location. Additional similar variables can be defined for other coronary artery anomalies (e.g., aneurysms).

[0042] artefact(x): A binary variable specifying whether an image artifact is observed at the centerline location (e.g., motion artifact, vessel overlap, etc.).

[0043] angulation(x): A floating-point variable specifying the angle of the bifurcation at the centerline location (or 0 if there is no bifurcation).

[0044] plaqueTpye(x): integer variable specifying the plaque type: fibrous, fatty, calcified, mixed, etc. Such information can be extracted from angiography (limited) or CCTA (rich).

[0045] materialProperties(x): Where PCCT is available, the multiple energy capabilities of PCCT are exploited, which allow for better characterization of tissue properties and material composition.

[0046] radiomicFeatures(x)

[0047] plaquePattern(x): An integer variable describing the presence of various plaque patterns indicative of vulnerable plaque. These patterns can include: spot calcifications, where vulnerable plaque may have small areas of calcification within the plaque that appear as small discrete spots on CT images; thin-cap fibroatherosclerotic plaque; and the napkin ring sign, where the napkin ring sign is a feature seen on CT images that may indicate vulnerable plaque. The napkin ring sign appears as a high-attenuation ring surrounding a low-attenuation core, similar to a napkin ring. This sign indicates that the plaque has a large, lipid-rich core and a thin, fibrous cap. Eccentricity includes where vulnerable plaque often has an irregular or eccentric shape, where the plaque material is primarily located on one side of the vessel wall. Low-attenuation plaque includes where vulnerable plaque often has a large, lipid-rich core that has lower attenuation values ​​on CT images compared to the surrounding tissue. This can appear as a soft, low-density area within the wall of the artery.

[0048] regionalWallMotion(x): can be derived from angiography, echocardiography, or multiphase CCTA.

[0049] downstreamMyocardialVolume(x): The myocardial volume subtended by the coronary artery tree downstream of a given position x.

[0050] Calculated features may also include:

[0051] FFR(x), IFR(x), restPdPa(x), etc.: hemodynamic characteristics calculated or predicted for the coronary artery tree.

[0052] WSS(x,t), OSI(x,t): location- and time-dependent wall shear stress and oscillatory shear index. Previous studies have shown that high / low stress is associated with plaque evolution and / or rupture.

[0053] Q(x,t): time-varying flow rate.

[0054] V(x,t): time-varying velocity.

[0055] Features are extracted / calculated automatically using the image processing system 105 or a processor, or semi-automatically using, for example, a processor and user input. In a fully automated approach, the underlying image processing algorithm detects features or values ​​through filtering, thresholding, pattern matching, and / or other image processing. The processor extracts features from the medical image data representing the detected area. This automated approach occurs without user input regarding the location of the anatomical structure. In a semi-automated approach, some of the features may be automatically extracted by the algorithm, while others may be manually annotated or edited / corrected. In another example, a user places a caliper or other measuring tool, and the processor determines the value of the feature from the caliper. In an alternative approach, one or more features are manually extracted. The user views the image and manually determines the value of the feature. The user enters annotations or fills in the manually determined value of the feature in a field.

[0056] In one embodiment, one or more machine learning (ML) trained models are used to extract features of plaque and / or vessels from medical image data. For example, a processor or other device uses a specially trained model to extract the shape, geometry, inlet and outlet angles, and location of the plaque relative to the vessel, vessel stenosis, vessel size, vessel shape, or a combination thereof. Other features may be extracted or derived from the data. For example, anatomical features of the plaque, such as shape, geometry, inlet and outlet angles, and location (e.g., bifurcation or other) may be extracted from segmented or processed CT angiography images.

[0057] In one embodiment, the machine-learned network(s) or model(s) used to extract feature data comprises a neural network defined as a plurality of sequential feature units or layers. Sequence is used to indicate the general flow of output feature values ​​from one layer to the input to the next layer. Sequence is used to indicate the general flow of output feature values ​​from one layer to the input to the next layer. Information from the next layer is fed to the next layer, and so on until the final output. The layers can be feed-forward only or bidirectional, including some feedback to the previous layer. The nodes of each layer or unit can be connected to all or only a subset of the nodes of the previous layer or unit and / or the next layer or unit. Skip connections can be used, such as one layer outputting to the next layer in sequence and other layers. Rather than pre-programming features and attempting to associate them with attributes, a deep architecture is defined to learn features at different levels of abstraction based on the input data. The features are learned to reconstruct lower-level features (i.e., features at a more abstract or compressed level). Each node of a unit represents a feature. Different units are provided for learning different features. Various units or layers can be used, such as convolutional layers, pooling layers (e.g., max pooling layers), deconvolutional layers, fully connected layers, or other types of layers. Within a unit or layer, any number of nodes can be provided. For example, 100 nodes are provided. Subsequent or subsequent units can have more, fewer, or the same number of nodes.

[0058] The processor 110 is configured to train (one or more) models using a machine training method and training data. The training data can be acquired at any time before the training data is input into the model. Different models can be configured for different tasks, for example, a different model can be configured for determining each relevant feature. Some models can be used to calculate multiple relevant features. The output of some models can be used by other models to determine certain relevant features. For example, one machine-trained model can perform segmentation, while another machine-trained model can use the output of segmentation to derive values ​​or provide classifications for specific relevant features. There are two phases for training and applying machine-trained models: a training phase for generating or training the model using a set of training data, and an application phase for applying the generated / trained entity matching network to new unseen (unlabeled) data. The training phase includes acquiring training data during a patient scan, processing the training data, and inputting the training data into the model to generate a trained model. The output is the trained model that is applied in the application phase. The application phase includes receiving real-time data from, for example, a CT scan, and applying the trained model trained during the training phase to calculate the values ​​of the corresponding relevant features. The training phase can be executed at any time before the application phase. The training phase can be repeated after acquiring new training data. The application phase can be executed at any time after the training phase generates a trained network and receives real-time data.

[0059] (One or more) machine-learned networks and other data may be stored separately in memory 120, and instructions for implementing the processes, methods, and / or techniques discussed herein may be provided in a non-transitory computer-readable storage medium or memory, such as a cache, buffer, RAM, removable media, a hard drive, or other computer-readable storage medium. The instructions may be executed by the processor or another processor. Computer-readable storage media include various types of volatile and non-volatile storage media. In response to one or more sets of instructions stored in or on a computer-readable storage medium, the functions, actions, or tasks illustrated in the figures or described herein are performed. The functions, actions, or tasks are independent of the instruction set, storage medium, processor, or processing strategy, and may be performed by software, hardware, integrated circuits, firmware, microcode, etc., operating alone or in combination. In one embodiment, the instructions are stored on a removable media device for reading by a local or remote system. In other embodiments, the instructions are stored in a remote location for transmission over a computer network. In still other embodiments, the instructions are stored in a given computer, CPU, GPU, or system. Because some of the constituent system components and method steps depicted in the figures may be implemented in software, the actual connections between the system components (or process steps) may vary depending on how the embodiment is programmed.

[0060] In addition to the relevant characteristic data, the system 100 is configured to acquire or otherwise use patient-level data. Patient-level data is not associated with a specific location in the coronary artery tree; instead, patient-level data refers to characteristics of the patient, such as the presence of other conditions associated with coronary artery disease, such as AV stenosis, the patient's demographic characteristics, and the patient's medical and family history. Patient-level data may further include total plaque burden, such as the number of plaques in the coronary arteries. Higher levels of plaque burden may indicate the presence of vulnerable plaque. The patient's laboratory results, particularly markers related to inflammation, may be used, as inflammation plays a key role in the development and progression of vulnerable plaque. Elevated levels of inflammatory markers, such as C-reactive protein (CRP), may indicate the presence of vulnerable plaque. Stress testing of the patient may also be used. For example, a positive stress test may indicate the presence of vulnerable plaque in the coronary arteries. Patient-level data may be acquired automatically or semi-automatically. For example, the system may parse or search the patient's medical history or data for various information as described above. Operators can annotate a patient's data with additional information such as family medical history or demographic characteristics.

[0061] System 100 is configured to determine the feature embedding at each node of the coronary artery graph network representing the topology. The coronary artery tree model described above can be converted into a graph, which can be represented as a sparse adjacency matrix or via an adjacency list. The sparse matrix only stores non-zero matrix entries. The adjacency list includes an array of nodes, and each node points to its adjacency list containing only its neighboring nodes. In addition to representing the topology, each node can include an associated feature vector consisting of the relevant features described above. Each of these vectors can be processed by a machine-trained model (e.g., a fully connected neural network) to obtain a feature embedding associated with each node. In addition to using the features associated with each node, patient-level data can also be included. The graph formulated in this way can then be processed via a graph neural network.

[0062] A graph is a type of data structure containing nodes and edges. A graph neural network (GNN) is a type of neural network that can work with graph data structures. GNNs are used to predict tasks based on nodes, edges, and graphs. In one embodiment, the graph neural network architecture uses a message passing mechanism to aggregate, process, and transmit information between nodes of the graph. Due to the sparse topological structure of the graph, directly applying the message passing method is inefficient because the information needs to be passed through many nodes to characterize the global characteristics of the coronary artery tree. This problem can be solved in one of two ways: adding virtual edges connecting nodes belonging to the same coronary artery segment and / or generating a higher-level nested graph by calculating its embedding based on the embedding of the underlying nodes belonging to the coronary artery segment, in which each node represents a coronary artery segment from the original graph. In a similar manner, a global node can be generated that aggregates information from the entire coronary artery tree.

[0063] Graph neural networks are used to predict information at the node level, segment level, and / or coronary artery tree level. The graph is input to the graph neural network, which returns an output graph with the same number of nodes, edges, and edge connectivity, but with updated edges, nodes, and global features. Based on the graph structure, all output features are conditioned on the input features and are fully differentiable. Classifiers can be used to interpret the output, for example, by providing a risk score or other metric. For example, the output of the graph neural network can be a risk score defined at the lesion or patient level, thereby assessing the risk of rupture, clot formation, or erosion within a specific timeframe.

[0064] The graph neural network is trained using ground truth labels, including, for example, follow-up data collected from patients with plaque rupture. Because this type of data is very sparse and expensive to collect, the model can first be pre-trained on a large dataset using vulnerable plaque indicators, such as maximum wall strain, plaque types prone to rupture (punctate calcifications, thin-cap fibroatherosclerotic plaques, napkin ring sign, low-attenuation plaques), abnormal wall shear stress or outliers for other hemodynamic characteristics, and / or other features. Once the graph neural network is pre-trained, it can be fine-tuned on a smaller dataset where follow-up data is available. Multi-task learning methods can be used to learn multiple measurements such as hemodynamic measures, QCA scores, plaque vulnerability scores, etc., allowing the predicted measurements to be more consistent and improving training behavior due to the provision of more supervisory signals. To generate ground truth values ​​for hemodynamic measures, the system can, for example, generate a large comprehensive database of coronary artery trees using population averages, and then run a physics-based model, such as a reduced-order model, on the coronary artery tree data.

[0065] System 100 is configured to output predictions of the graph neural network, for example, using a user interface. The user interface may include a display 115 as an output device. Other outputs may be used, such as an interface, network connector, or printer. The output is configured to output the risk of rupture or other risks associated with the plaque to the patient. The display is a monitor, LCD, projector, plasma display, CRT, printer, or other device now known or later developed for outputting visual information. The display receives images, graphics, text, quantities, or other information from a processor, memory, or medical imaging device 130. The output may be one or more values ​​describing an assessment of vulnerable plaque in the patient, or a visual depiction thereof. For example, one or more medical images depicting vulnerable plaque may be displayed. The image may be an image of a region of the patient. In one embodiment, the image is an image of a blood vessel or vascular tree. The location of the plaque located by the image processor is highlighted or displayed alongside other tissues or fluids that are not highlighted. The image includes an indication of the risk for each plaque in the patient, or for plaque in general, such as a graphic or colorization. Alternatively or additionally, the image includes a quantification based on the classification, such as a risk score. The quantification may be displayed as an image without a medical image representation of the patient.

[0066] In one embodiment, the system 100 can be further configured to generate or support clinical decisions that not only estimate the risk of plaque rupture at different time points during the clinical workflow, but also provide procedural instructions that allow for minimizing risk given all patient-specific data. The system can be configured to provide multiple assessments at different times. For example, the first assessment can be performed after CCTA. The assessment can be updated during a catheterization lab exam as more data becomes available. The assessment can be further updated after the catheterization lab exam, particularly if PCI is performed on some lesions, i.e., performed in a post-PCI setting.

[0067] The system can recommend one or more methods for treating vulnerable coronary plaques. The method can include recommending certain medications. For example, statins, antiplatelet drugs, and anticoagulants can be prescribed to reduce the risk of plaque rupture and blood clot formation. The system can recommend lifestyle changes, such as encouraging a healthy diet, regular exercise, and smoking cessation. The system can also recommend CABG and PCI. In addition, or alternatively, the proposed method can also be used to predict patient outcomes given a pre-specified time interval.

[0068] Figure 4 Depicts an example method for assessing vulnerable plaque. The actions are performed by Figure 1 、 Figure 2 The actions are performed in the order shown (e.g., from top to bottom) or other orders.

[0069] In action A110, the medical imaging device 130 scans the patient. A medical image or data set is acquired. A medical image is a frame of data representing the patient. The data can be in any format. Although the terms "image" and "imaging" are used, the image or imaging data can be in a format prior to the actual display of the image. For example, a medical image can be a plurality of scalar values ​​representing different positions in a Cartesian or polar coordinate format that is different from the display format. As another example, a medical image can be a plurality of red, green, and blue (e.g., RGB) values ​​output to a display for generating an image in a display format. The medical image can be an image that is currently or previously displayed on a display or in another format. An image or imaging is a data set that can be used for imaging, such as scan data representing a patient.

[0070] Any type of medical image can be used. In one embodiment, the medical image is a computed tomography (CT) image acquired using the CT system 130. For example, a CT data set can be used to detect blood vessels. For CT, the raw data is reconstructed into a three-dimensional representation. Coronary computed tomography angiography (also known as coronary CT angiography or CCTA) uses an injection of an iodine-containing contrast material and a CT scan to examine the arteries that supply blood to the heart and determine whether they have narrowed. Plaque is composed of various substances such as fat, cholesterol, and calcium that are deposited along the inner lining of the arteries. Plaque that gradually builds up over time can reduce or, in some cases, completely block blood flow. Patients undergoing CCTA scans receive the iodine-containing contrast material as an intravenous (IV) injection to ensure the best possible images of the heart's blood vessels.

[0071] Images generated during a CT scan can be reformatted to create three-dimensional (3D) images that can be viewed on a monitor, printed on film or via a 3D printer, or transferred to electronic media. Other medical imaging modalities used to acquire scan data include X-ray angiography (XA), optical coherence tomography (OCT), ultrasound (US), intravascular ultrasound (IVUS), and near-infrared spectroscopy (NIRS). Intracoronary imaging data (IVUS, OCT) can also be used to extract information for assessing plaque vulnerability. Medical image data or scan data represents tissue of a patient. Alternatively, the medical image represents flow, velocity, or fluid within the patient's body. In other embodiments, the data represents both flow and structure. The medical data represents a one-dimensional, two-dimensional, or three-dimensional portion of a patient. For example, the medical data represents a region or slice of a patient. Values ​​are provided for each of a plurality of locations distributed in two or three dimensions. Medical data is acquired as data frames. A data frame represents a scanned portion at a given time or period. The data set can represent an area or volume that changes over time, such as providing a 4D representation of a patient. Medical images or datasets are acquired through a scan of a patient. Acquisition occurs as part of the scan. Alternatively, acquisition is from a storage device or memory, such as a previously created dataset from a PACS. In one embodiment, intracoronary imaging data (IVUS, OCT) can also be used to extract information for assessing plaque vulnerability.

[0072] In one embodiment, rather than relying solely on data from a single time point or single coronary examination, longitudinal data can be incorporated to achieve a more comprehensive approach: advanced features that capture the dynamic evolution of the coronary arteries or plaque within the coronary arteries (based on all available imaging examinations) can be included, such as tracking changes in plaque size, composition, and other characteristics over time. In addition, the overall progression of a patient's health status throughout their medical history can provide other key data to improve the performance of the predictive model based on a deeper understanding of the patient, thereby increasing its accuracy.

[0073] In one embodiment, the system further acquires patient-level data that is not associated with a specific location but rather refers to characteristics of the patient. This type of data may include information such as the presence of other conditions associated with coronary artery disease in the patient (such as AV stenosis, etc.). Patient-level data may also include patient demographics, family medical history, calcium scores, plaque burden levels, laboratory results, stress tests, and other patient-level data.

[0074] At A120, the system 100 generates a coronary artery tree model 300 of the patient's coronary artery centerline 310, the coronary artery tree model including a plurality of nodes 320 representing locations in the coronary artery tree model and edges representing relationships between the nodes. The 3D centerline model is based on the above-described Figure 3 A modeling algorithm for the coronary arteries can be used to generate a coronary artery tree model. In one embodiment, the image data is segmented and one or more machine learning models or algorithms are used to identify vessels and centerlines.

[0075] System 100 can be configured to segment CT data using a segmentation model. CT data is input into a segmentation model, which is configured to output a segmented mask upon input of the CT data. Any segmentation method can be used. For example, segmentation can be threshold-based, region-based, shape-based, model-based, proximity-based, and / or machine learning-based, among other segmentation techniques. Threshold-based methods segment image data by creating a binary partition based on image attenuation values, as determined by the relative attenuation of structures on the image. Region-based segmentation compares a pixel in the image to neighboring pixels, and if predefined region criteria (e.g., homogeneity) are met, the pixel / voxel is assigned the same class as one or more of its neighbors. Shape-based techniques use either atlas-based or model-based methods to find lumen boundaries. Model-based methods use a priori shape information, similar to atlas-based methods; however, to better accommodate shape variability, model-based methods use an optimization procedure to fit a statistical shape or appearance model of the heart to the image. Vessels and vessel centerlines can be derived from the segmentation mask, for example, using a classifier.

[0076] In one embodiment, instead of using 3D geometry reconstructed from multiple coronary angiography acquisitions, a 2.5D geometry is derived by matching 2D geometry extracted from each acquisition. Instead of using a 3D centerline-based representation of the coronary arteries, the system can use a 3D volume reconstruction of the centerlines from CTA or X-ray angiography. The features calculated at action A130 described below can then be calculated in 2D (+ time) and aggregated in a 3D (+ time) volume grid for X-ray angiography. Similarly, features from CTA images can be extracted in 3D (+ time) and effectively fused with features obtained from X-ray angiography. This approach can more easily fuse information from multiple image modalities (CTA, X-ray, IVUS / OCT) with the help of a 3D volume grid.

[0077] At A130, at each location (node) of the coronary artery tree model, a set of features fi(x, t) is determined / calculated / derived from the image data and the patient data, where fi refers to features, x refers to 3D location, and t refers to time. Some of the features may be constant over time. Some of the features may be obtained by running other AI models or algorithms on the available input data. At A110, anatomical features of the vessel or plaque and / or morphological features of the plaque may be extracted from the patient's scan by the medical imaging device 130. Other extracted features include parameters for one or more abnormalities of the vascular structure. Abnormal morphology can be characterized by the nature of the calcifications, the nature of the plaque (e.g., fibrous, lipidic, necrotic, calcified), the nature of the thrombus, the nature of the diffuse disease, the presence of complete or subtotal occlusions, the presence of myocardial bridges (superficial and / or deep), congenital anomalies of the coronary arteries (e.g., anomalous origin of a coronary artery arising from an anomalous sinus of Valsalva in an interarterial course between the great arteries, anomalous origin of a coronary artery arising from the pulmonary trunk, or other), aneurysmal dilatation and superimposed atherosclerosis, "high-take off) coronary arteries (e.g., opening a few millimeters above the sinotubular junction (the artery may have a sharp downward angle and partially penetrate the aortic wall)), myocardial bridges: superficial and deep, coronary artery fistulas, coronary artery dissections, coronary vasculitis (e.g., rheumatoid arthritis, systemic lupus erythematosus (SLE), or Behçet's disease, Kawasaki disease, polyarteritis nodosa, and / or persistent (post)inflammatory aneurysms), fibromuscular dysplasia, coronary microembolism, and / or left or right dominance. Additional, different, or fewer abnormal features may be used.

[0078] Functional features representing the operation of the vascular structure can be extracted. Functional information includes functional imaging (such as measurements of uptake), or other operational information (such as contrast agent measurements). For training data, functional features can be determined from simulations, synthetically created images, modeling, and / or other representations of the operation of the vessel.

[0079] In addition to anatomical and morphological features from medical images or synthetic representations of vascular trees, functional features can also be extracted. For example, data from perfusion scans or other medical imaging scans (e.g., single photon emission computed tomography (SPECT), positron emission tomography (PET), or perfusion imaging) can also be used to extract features, such as metrics characterizing relative and / or absolute tissue perfusion in each coronary territory at rest and / or during stress. As another example, angiographic data can characterize contrast agent propagation. Some features characterize the flow of contrast agent at a given location (such as the time to peak tracer concentration) and the splitting of the contrast agent across different daughter vessels at a bifurcation.

[0080] A comprehensive list of relevant features is provided below. r(x,t): Time-varying cavity radius at each centerline location. ref (x,t): Time-varying baseline (ie, healthy) lumen radius at each centerline location. outer_wall(x,t): The time-varying outer wall radius at each centerline location. This allows for the assessment of positive remodeling. Vulnerable plaque causes the arterial wall to bulge outward, which is called positive remodeling. This can be seen as an increase in the diameter of the arterial segment containing the plaque compared to the normal adjacent segment. curvature(x,t): The time-varying curvature value at each centerline location. stent(x): A binary variable specifying whether a stent is present at each centerline location. Alternatively, an integer type can be used since stents can overlap and multiple stents can be present at a given centerline location. label(x): An integer variable specifying the coronary artery segment to which the current location belongs. stenosis(x): This can be either an integer variable specifying the unique id of the stenosis or a floating-point variable specifying the probability of stenosis at this location. contrast(x,t): An integer variable specifying the contrast intensity at this location, which typically varies over time. CABG(x): A binary variable specifying whether each centerline location is part of a bypass graft. collateral(x): A binary variable specifying whether each centerline location is part of a collateral artery. CTO(x): A binary variable specifying whether a CTO was encountered at the centerline location (typically a leaf node). myocardialBridging(x): A binary variable specifying whether a myocardial bridge was observed at the centerline location. Additional similar variables can be defined for other coronary artery anomalies (e.g., aneurysms). artefact(x): A binary variable specifying whether an image artifact (e.g., motion artifact, vessel overlap, etc.) was observed at the centerline location. angulation(x): A floating-point variable specifying the bifurcation angle at the centerline location (0 if no bifurcation exists). plaqueTpye(x): An integer variable specifying the plaque type: fibrous, fatty, calcified, mixed, etc. Such information can be extracted from angiography (limited) or CCTA (rich). materialProperties(x): Where available, exploits the multi-energy capabilities of PCCT, which allows for better characterization of tissue properties and material composition. radiomicFeatures(x). plaquePattern(x): Integer variable describing the presence of various plaque patterns that are indicative of vulnerable plaque: Punctate calcifications: Vulnerable plaque may have small areas of calcification within the plaque that appear as small discrete spots on CT images; Thin-cap fibroatheroma; Napkin ring sign, which is a feature seen on CT images that may indicate vulnerable plaque. It appears as a high-attenuation ring surrounding a low-attenuation core, similar to a napkin ring. This sign indicates that the plaque has a large lipid-rich core and a thin fibrous cap. Eccentricity: Vulnerable plaque often has an irregular or eccentric shape, with the plaque material located primarily on one side of the vessel wall.Low-attenuation plaque: Vulnerable plaque typically has a large, lipid-rich core that has lower attenuation values ​​on CT images compared to surrounding tissue. This can be visualized as a soft, low-density area within the artery wall. Regional Wall Motion (x): Can be derived from angiography, echocardiography, or multiphase CCTA. Downstream Myocardial Volume (x): The volume of myocardium subtended by the coronary artery tree downstream of a given location x.

[0081] Calculated features may also include: FFR(x), IFR(x), restPdPa(x), etc.: Calculated or predicted hemodynamic characteristics for the coronary artery tree. WSS(x,t), OSI(x,t): Location- and time-dependent wall shear stress and oscillatory shear index. Previous studies have shown that high and low stress are associated with plaque evolution and / or rupture. Q(x,t): Time-varying flow rate. V(x,t): Time-varying velocity.

[0082] Rather than relying solely on data from a single time point or single coronary examination, a more comprehensive approach can be achieved by incorporating longitudinal data: advanced features that capture the dynamic evolution of the coronary arteries or plaque within them, such as tracking changes in plaque size, composition, and other properties over time, can be included (based on all available imaging tests). In addition, the overall progression of a patient's health status throughout their medical history can provide other key data to improve the performance of the predictive model based on a deeper understanding of the patient, thereby increasing its accuracy.

[0083] A machine-learned network or model can be used to provide derived and calculated features. Different networks / models can be specifically configured for each corresponding task of deriving or calculating features. Different networks and configurations can be used. For example, DenseNet or other network arrangements can also be used for the trained networks or other trained networks described above for segmentation, classification, or analysis. DenseNet connects each layer in the network to every other layer in a feedforward manner. For each layer in DenseNet, the feature maps of all previous layers are used as input, and the output feature maps of that layer are used as input to all subsequent layers. In DenseNet, for each layer, the feature maps of all previous layers are used as input, and its own feature maps are used as input to all subsequent layers. To reduce the size of the network, DenseNet can include transition layers. The layers include convolutions followed by average pooling. The transition layers reduce the height and width dimensions, but keep the feature dimensions unchanged. The neural network can be further configured as a U-net (U-Net). A U-net is an autoencoder in which the output from the encoder (one half of the network) is concatenated with its mirrored counterpart in the decoder (one half of the network). Skip connections prevent the middle of the network from becoming a bottleneck.

[0084] Other deep architectures that can be used include convolutional neural networks (CNNs) or deep belief networks (DBNs), but other deep networks can also be used. CNNs learn a feedforward mapping function, while DBNs learn a generative model of the data. In addition, CNNs use shared weights for all local regions, while DBNs are fully connected networks (e.g., different weights for all regions of an image). CNN training is completely discriminative through backpropagation. On the other hand, DBNs use layer-by-layer unsupervised training (e.g., pre-training), followed by discriminative refinement with backpropagation when necessary. In one embodiment, the arrangement of the trained network is a fully convolutional network (FCN). Alternative network arrangements can be used, for example, a 3D very deep convolutional network (3D-VGGNet). VGGNet stacks many layer blocks, which include narrow convolutional layers followed by a maximum pooling layer. A 3D deep residual network (3D-ResNet) architecture can be used. Resnet uses residual blocks and skip connections to learn residual mappings.

[0085] At action A140, the system determines a feature embedding associated with each node from the set of corresponding features and patient-level data for the corresponding node. In addition to representing the topological structure, each node can have an associated feature vector consisting of the features previously described. Each of these vectors can be processed by a fully connected neural network to obtain a feature embedding associated with each node. When computing these embeddings, in addition to using the features associated with each node, the patient-level data from A140 can also be included. The coronary artery tree model of action A120 can be presented as a graph, which can be represented as a sparse adjacency matrix or via an adjacency list. The adjacency list describes the connectivity of the edges between nodes as tuples in the entries of the adjacency list.

[0086] At action A150, the system inputs the feature embeddings into a graph neural network (GNN) and predicts diagnostic information at the node, segment, and / or arterial tree level based on the output of the GNN. A GNN is an optimizable transformation of all graph attributes (nodes, edges, global context) that preserves graph symmetry (permutation invariance). The GNN uses a graph-in, graph-out architecture, meaning the model accepts an arterial tree as input, where feature embeddings are loaded into its nodes, edges, and global context, and gradually transforms the embeddings without changing the connectivity of the input graph. The primary unit of computation is the GN block, a graph-to-graph module that takes a graph as input, performs computation on the structure, and returns a graph as output. Entities are represented by graph nodes, relationships by edges, and system-level features by global attributes. Prediction is achieved by the GNN using pooling within the GNN's layers to make the learned embeddings aware of the graph connectivity. This is accomplished using message passing, where neighboring nodes or edges exchange information and influence each other's updated embeddings. The proposed architecture uses a message passing mechanism to aggregate, process, and pass information between nodes in a graph. Message passing works in three steps. For each node in the graph, all neighboring node embeddings (or messages) are collected. Messages are aggregated using an aggregation function (e.g., sum). The pooled messages are passed through an update function (e.g., a neural network). Just as pooling can be applied to nodes or edges, message passing can occur between nodes or edges.

[0087] Due to the sparse topology of the graph, directly applying message passing methods can be inefficient, as information needs to be passed through many nodes to globally characterize the coronary artery tree. This problem is addressed in two ways: by adding virtual edges that connect nodes belonging to the same coronary artery segment or by generating a higher-level nested graph in which each node represents a coronary artery segment from the original graph. Embeddings can be computed based on the embeddings of the underlying nodes belonging to the coronary artery segments. In a manner similar to the second approach, a global node can be generated that aggregates information from the entire coronary artery tree.

[0088] By applying the proposed graphnet architecture on the obtained nested graph, information can be predicted at the node level, segment level and coronary tree level. In one embodiment, the output of the GNN is a prediction, such as a risk score. In one embodiment, the risk of rupture is predicted. In another embodiment, risk stratification is provided. For example, both risk stratification and plaque vulnerability assessment are calculated from or as a risk score for a patient in coronary artery disease analysis. Vulnerable coronary plaque refers to a type of plaque in the coronary artery that is more likely to rupture or cause blood clots, which can lead to a heart attack. Therefore, the output of the model can be a risk score defined at the lesion or patient level, thereby assessing the risk of rupture, clot formation, and erosion within a specific time frame.

[0089] In one embodiment, clinical decision support can be provided that not only estimates the risk of plaque rupture at different time points during the clinical workflow, but also provides procedural instructions that allow minimizing the risk given all patient-specific data. The first assessment can be performed after CCTA. The assessment may be updated during a catheterization lab exam as more data becomes available. The assessment can be further updated after the catheterization lab exam, especially if PCI is performed on some lesions, i.e., performed in a post-PCI setting.

[0090] There are several approaches to treating vulnerable coronary plaques, including: Medical therapy: Statins, antiplatelet drugs, and anticoagulants can be prescribed to reduce the risk of plaque rupture and blood clot formation. Lifestyle changes: Encouraging a healthy diet, regular exercise, and quitting smoking can help prevent the development of vulnerable plaques. Alternatively, the proposed method can also be used to predict patient outcomes given pre-specified time intervals.

[0091] In one embodiment, the ground truth labels used to train the model include follow-up data collected from patients who experienced plaque rupture. Since this type of data is very sparse and very expensive to collect, the model can first be pre-trained on a large dataset using vulnerable plaque indicators such as maximum wall strain, plaque types prone to rupture (punctate calcifications, thin-cap fibroatherosclerotic plaques, napkin ring sign, low-attenuation plaques), abnormal wall shear stress, or outliers for other hemodynamic characteristics. Once the model is pre-trained, it can be fine-tuned on a smaller dataset where follow-up data is available. In addition, a multi-task learning approach can be used to learn multiple measurements such as hemodynamic measures, QCA scores, plaque vulnerability scores, etc., allowing the predicted measurements to be more consistent and improving the training behavior due to the provision of more supervisory signals. To generate ground truth values ​​for hemodynamic measures, the system can, for example, generate a large comprehensive database of coronary artery trees using population averages, and then run a physics-based model, such as a reduced-order model, on the coronary artery tree data.

[0092] It is to be understood that the elements and features recited in the appended claims may be combined in various ways to create new claims that also fall within the scope of the invention. Thus, although the dependent claims appended below depend solely upon a single independent or dependent claim, it is to be understood that these dependent claims may alternatively be made dependent in alternative forms to any preceding or following claim, whether independent or dependent, and such new combinations are to be understood to form part of this specification.

[0093] Although the present invention has been described above with reference to various embodiments, it will be appreciated that many variations and modifications may be made to the embodiments described. The foregoing description is therefore intended to be illustrative rather than restrictive, and it is to be understood that all equivalents and / or combinations of the embodiments are intended to be included in this description. Regardless of the use of grammatical terms, individuals with the masculine and feminine genders are included in this term.

[0094] The following is a list of non-limiting illustrative embodiments disclosed herein:

[0095] Illustrative embodiment 1. A method for vulnerable plaque assessment and outcome prediction in coronary artery disease, the method comprising: acquiring medical imaging data of a patient; generating a coronary tree model of the patient's coronary artery centerlines from the medical imaging data, the coronary tree model comprising a plurality of nodes representing positions in the coronary tree model; determining a feature embedding associated with each node from a plurality of features derived from the medical imaging data; inputting the feature embedding into a graph neural network; and outputting an assessment of vulnerable plaque at a node level, a segment level, and / or a coronary tree level based on an output of the graph neural network.

[0096] Illustrative embodiment 2. The method of illustrative embodiment 1, wherein the medical imaging data is coronary computed tomography angiography (CCTA) data.

[0097] Illustrative embodiment 3. The method of illustrative embodiment 1, further comprising: obtaining patient-level data, wherein the patient-level data is not specific to any location in the coronary artery tree model, wherein the patient-level data is input into the graph neural network.

[0098] Illustrative Example 4. The method of Illustrative Example 3, wherein the patient-level data includes at least one of the presence of other conditions associated with coronary artery disease, patient demographics, patient medical history, family medical history, calcium score, total plaque burden, laboratory results, or results of stress testing.

[0099] Illustrative Embodiment 5. The method of Illustrative Embodiment 1, wherein generating the coronary artery tree model comprises segmenting the medical imaging data using a thresholding method, wherein the coronary artery tree model includes all locations having a diameter greater than a given threshold of 1.0 mm.

[0100] Illustrative Embodiment 6. The method of Illustrative Embodiment 1, wherein generating the coronary artery tree model comprises deriving 2.5D geometry by matching 2D geometry extracted from each acquisition.

[0101] Illustrative embodiment 7. The method of illustrative embodiment 1, wherein determining the feature embedding comprises: defining a set of features fi(x, t) for each respective node in the plurality of nodes, where fi refers to the feature, x refers to the 3D position in the coronary artery tree model, and t refers to time; and inputting the set of features into a machine-trained network, wherein the machine-trained network is configured to output the feature embedding for each respective node.

[0102] Illustrative Embodiment 8. The method of Illustrative Embodiment 7, wherein the values ​​of the set of features vary depending on the state of the patient.

[0103] Illustrative Embodiment 9. The method of Illustrative Embodiment 7, wherein multiple instances of certain features are averaged to determine the set of features.

[0104] Illustrative embodiment 10. The method of illustrative embodiment 1, wherein the graph neural network uses a message passing mechanism to aggregate, process and transmit information between nodes of the graph neural network.

[0105] Illustrative Embodiment 11. The method of Illustrative Embodiment 10, wherein virtual edges connecting nodes belonging to the same coronary artery segment are added to the graph neural network for use in the message passing mechanism.

[0106] Illustrative Example 12. The method of Illustrative Example 1, wherein the assessment includes a risk score defined at a lesion or patient level to assess the risk of rupture, clot formation, or erosion within a specific time frame.

[0107] Illustrative Example 13. The method of Illustrative Example 1, further comprising: assessing the patient for vulnerable plaque based on the assessment; and providing procedural instructions that allow minimizing the risk of the patient developing vulnerable plaque.

[0108] Illustrative Embodiment 14. The method of Illustrative Embodiment 13, wherein the patient is assessed for vulnerable plaque after acquiring the medical imaging data, wherein the assessment is updated during a catheterization laboratory examination, and wherein the assessment is further updated after the catheterization laboratory examination.

[0109] Illustrative embodiment 15. A system for vulnerable plaque assessment, the system comprising: a medical imaging system configured to acquire medical imaging data of a patient; an image processing system configured to generate a coronary tree model of the patient's coronary artery centerlines from the medical imaging data, the coronary tree model comprising a plurality of nodes representing positions in the coronary tree model, the image processing system further configured to calculate or derive one or more features associated with each node from a plurality of features derived from the medical imaging data, the image processing system further configured to determine a feature embedding for each node based on the one or more features, and input the feature embedding into a graph neural network configured to generate a vulnerable plaque assessment; and an output interface configured to provide the vulnerable plaque assessment.

[0110] Illustrative embodiment 16. The system of illustrative embodiment 15, wherein each node of the graph neural network is associated with a feature vector consisting of the one or more features, wherein the feature vector is processed by a fully connected neural network to obtain a feature embedding associated with each node.

[0111] Illustrative embodiment 17. The system of illustrative embodiment 15, wherein the one or more features include a set of features fi(x, t) for each respective node in the plurality of nodes, where fi refers to the feature, x refers to a 3D position in the coronary artery tree model, and t refers to time.

[0112] Illustrative Embodiment 18. The system of Illustrative Embodiment 15, wherein generating the coronary artery tree model comprises segmenting the medical imaging data using a thresholding method, wherein the coronary artery tree model includes all locations having a diameter greater than a given threshold of 1.0 mm.

[0113] Illustrative embodiment 19. The system of illustrative embodiment 15, wherein the medical imaging data is coronary computed tomography angiography (CCTA) data.

[0114] Illustrative embodiment 20. A non-transitory computer-implemented storage medium storing machine-readable instructions executable by at least one processor, the machine-readable instructions comprising: generating a coronary tree model of a patient's coronary artery centerlines from medical imaging data, the coronary tree model comprising a plurality of nodes representing locations in the coronary tree model; determining a feature embedding associated with each node from a plurality of features derived from the medical imaging data; inputting the feature embedding into a graph neural network; and outputting an assessment of vulnerable plaque at a node level, a segment level, and / or a coronary tree level based on an output of the graph neural network.

Claims

1. A method for vulnerable plaque assessment and outcome prediction in coronary artery disease, the method comprising: Obtaining medical imaging data of patients; generating a coronary artery tree model of the patient's coronary artery centerlines from the medical imaging data, the coronary artery tree model comprising a plurality of nodes representing locations in the coronary artery tree model; determining a feature embedding associated with each node from a plurality of features derived from the medical imaging data; Embedding the features into a graph neural network; as well as An assessment of vulnerable plaque at a node level, a segment level, and / or a coronary artery tree level is output based on the output of the graph neural network.

2. The method according to claim 1, wherein The medical imaging data is coronary computed tomography angiography (CCTA) data.

3. The method according to claim 1, further comprising: Patient-level data is acquired, wherein the patient-level data is not specific to any location in the coronary artery tree model, wherein the patient-level data is input into the graph neural network.

4. The method according to claim 3, wherein: The patient-level data includes at least one of the presence of other conditions associated with coronary artery disease, patient demographics, patient medical history, family medical history, calcium score, total plaque burden, laboratory results, or results of a stress test.

5. The method according to claim 1, wherein Generating the coronary artery tree model includes segmenting the medical imaging data using a thresholding method, wherein the coronary artery tree model includes all locations having a diameter greater than a given threshold of 1.0 mm.

6. The method according to claim 1, wherein Generating the coronary artery tree model includes deriving a 2.5D geometry by matching the 2D geometry extracted from each acquisition.

7. The method according to claim 1, wherein Determining the feature embedding includes: defining a set of features fi(x, t) for each respective node of the plurality of nodes, where fi refers to the features, x refers to a 3D position in the coronary artery tree model, and t refers to time; and The set of features is input into a machine-trained network, which is configured to output a feature embedding for each corresponding node.

8. The method according to claim 7, wherein: The values ​​of the set of features vary depending on the patient's condition.

9. The method according to claim 7, wherein: Multiple instances of certain features are averaged to determine the set of features.

10. The method according to claim 1, wherein The graph neural network uses a message passing mechanism to aggregate, process and transmit information between multiple nodes of the graph neural network.

11. The method according to claim 10, wherein: Virtual edges connecting nodes belonging to the same coronary artery segment are added to the graph neural network for use in the message passing mechanism.

12. The method according to claim 1, wherein The assessment includes risk scores defined at the lesion or patient level, estimating the risk of rupture, clot formation, or erosion within a specific timeframe.

13. The method according to claim 1, further comprising: assessing vulnerable plaque in the patient based on the assessment; as well as Procedural instructions are provided that allow minimizing the patient's risk of developing vulnerable plaque.

14. The method according to claim 13, wherein The patient is assessed for vulnerable plaque after acquiring the medical imaging data, wherein the assessment is updated during a catheterization laboratory examination, and wherein the assessment is further updated after the catheterization laboratory examination.

15. A system for vulnerable plaque assessment, the system comprising: a medical imaging system configured to acquire medical imaging data of a patient; an image processing system configured to generate a coronary artery tree model of the patient's coronary artery centerlines from the medical imaging data, the coronary artery tree model comprising a plurality of nodes representing locations in the coronary artery tree model, the image processing system further configured to calculate or derive one or more features associated with each node from a plurality of features derived from the medical imaging data, the image processing system further configured to determine a feature embedding for each node based on the one or more features, and input the feature embedding into a graph neural network configured to generate a vulnerable plaque assessment; as well as An output interface is configured to provide the vulnerable plaque assessment.

16. The system according to claim 15, wherein: Each node of the graph neural network is associated with a feature vector consisting of the one or more features, wherein the feature vector is processed by a fully connected neural network to obtain a feature embedding associated with each node.

17. The system according to claim 15, wherein: The one or more features include a set of features fi(x, t) for each respective node of the plurality of nodes, where fi refers to the feature, x refers to a 3D position in the coronary artery tree model, and t refers to time.

18. The system according to claim 15, wherein: Generating the coronary artery tree model includes segmenting the medical imaging data using a thresholding method, wherein the coronary artery tree model includes all locations having a diameter greater than a given threshold of 1.0 mm.

19. The system of claim 15, wherein: The medical imaging data is coronary computed tomography angiography (CCTA) data.

20. A non-transitory computer-implemented storage medium storing machine-readable instructions executable by at least one processor, the machine-readable instructions comprising: generating a coronary artery tree model of the patient's coronary artery centerlines from the medical imaging data, the coronary artery tree model comprising a plurality of nodes representing locations in the coronary artery tree model; determining a feature embedding associated with each node from a plurality of features derived from the medical imaging data; Embedding the features into a graph neural network; as well as An assessment of vulnerable plaque at a node level, a segment level, and / or a coronary artery tree level is output based on the output of the graph neural network.

Images