Coronary heart disease artificial intelligence risk assessment method based on cross attention mechanism

Abstract

The invention provides a coronary heart disease multi-omics artificial intelligence risk assessment method based on a cross attention mechanism. The method comprises the following steps: in order to reduce interference of noise generated in a data acquisition process on an experiment, firstly preprocessing data, then analyzing characteristics of clinical data, CT image data and electrocardiosignal data, and designing different feature extractors to respectively extract measurement features, spatial features and time sequence features of the clinical data, the CT image data and the electrocardiosignal data; meanwhile, in order to better learn space-time interaction features between CT and ECG data, fusion of space and time sequence cross-modal data features is realized based on a cross attention mechanism; and finally, further integrating clinical measurement characteristics, and performing coronary heart disease risk assessment.

Description

Technical Field

[0001] This invention relates to the field of medical risk assessment, and more specifically to an artificial intelligence risk assessment method for coronary heart disease based on a cross-attention mechanism. Background Technology

[0003] Current coronary artery disease (CAD) risk assessments primarily rely on patients' clinical data, CT images, or electrocardiogram (ECG) data. Mathematical models and algorithms are used to extract features from these different data sets to predict the risk of CAD. However, most methods integrating features from different modalities tend to be simple fusions, such as averaging, weighting, or splicing. They focus only on the specific features of different modalities, neglecting the interactive information provided by the features from different modalities. For example, clinical data provides metric information, CT images provide spatial information, and ECG provides temporal information. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the present invention aims to provide a multi-omics artificial intelligence risk assessment method for coronary heart disease based on a cross-attention mechanism. This method further improves the accuracy of coronary heart disease risk assessment results through cross-modal feature fusion.

[0005] To achieve the above objectives, the present invention provides the following technical solution: an artificial intelligence risk assessment method for coronary heart disease based on cross-attention mechanism, comprising the following steps: Step 1: Collect three modalities of data for coronary heart disease: clinical data, CT images, and ECG signals. Preprocess the collected multimodal data by removing features with too many missing values ​​and filling features with fewer missing values ​​using the mean or median. Then, use ANOVA variance screening to identify features that show significant differences between high-risk and low-risk populations. Finally, perform z-score standardization on the data. For ECG data, remove information below 0.5 Hz and process the ECG using a Butterworth bandpass filter with a frequency range of 0.5–45 Hz. Step two: Based on the characteristics of different modal data, different feature extractors are used to extract features of different modalities from the data processed in step one. Step 3: Based on the data features extracted in Step 2, cross-modal spatiotemporal feature fusion of CT image data and ECG data is achieved using a cross-attention mechanism; Step four involves further integrating clinical metrics and learning the probability of a patient having a high risk of coronary heart disease through a fully connected layer.

[0006] As a further improvement of the present invention, the specific steps of step two, which involves providing different feature extractors to extract features of different modalities based on the characteristics of different modal data, are as follows: For the clinical data obtained after step one preprocessing Further extraction of metric features through linear mapping The details are as follows: in, It is a learnable weight matrix. For bias, For activation functions; For the CT image data obtained after step one preprocessing Image features F are extracted using a spatial feature extractor. CT； For the ECG signal data after step one preprocessing Feature F is extracted using a temporal feature extractor. ECG .

[0007] As a further improvement of the present invention, the image feature F is extracted by a spatial feature extractor. CT The specific method is as follows: First, use Layered CNNs are used to extract features from CT images, specifically: When N=1, i.e., its initial characteristics Then, global average pooling is used to map image features to the channel dimension, and then a fully connected layer is used to learn the importance of different channels. Specifically, in and These represent the height and width of the image, respectively. These are the feature values ​​at the corresponding locations in the image. The weight matrix is ​​learnable, and then low-level features and high-level features are associated through skip connections; the specific process can be represented as follows: Finally, the features F of the CT image are learned by superimposing Ns of this module. CT .

[0008] As a further improvement of the present invention, the extraction of feature F by the temporal feature extractor ECG The specific method is as follows: [The text abruptly ends here, likely due to an incomplete sentence or a formatting error. Divided into A length of For vectors ; The ECG representation is constructed by concatenating representations learned from a forward and backward LSTM; specifically: in, Indicates the forward process, For backward process, This represents a splicing operation; By introducing an input gate Forgotten Gate Output gate Three gating mechanisms are used to learn long-term dependencies in ECG signals. The specific process is as follows: Forgetting Gate In other words: For input gate part: For output gate part: in, For the current input, for The hidden state at that time These are all learnable weight parameters. It's a bias. Used for message passing. The hyperbolic tangent activation function is used. This is a dot product operation.

[0009] As a further improvement of the present invention, the specific steps of achieving cross-modal spatiotemporal feature fusion of CT image data and ECG data based on the cross-attention mechanism in step three are as follows: First, CT image features are generated through three linear mapping layers. and ECG characteristics The Q (Query), K (Key), V (Value) matrix; For CT image data: Regarding ECG data: Then, a cross-attention mechanism is used to capture the dependencies between different input features. By combining the features from both parts, the cross-modal fusion features of CT images and ECG data can be obtained. .

[0010] As a further improvement of the present invention, the specific method for capturing the dependency relationship between different input features through the cross-attention mechanism is as follows: by obtaining key features from one modality and query features from another modality, the similarity between the two features is calculated by dot product and weights are assigned, thereby focusing more on the feature-related parts in CT and ECG during the learning process, as specifically expressed below: By applying the learned matrix A to the matrix V, we can obtain: .

[0011] As a further improvement of the present invention, the cross-modal fusion features of CT images and ECG data can be obtained by combining the features of the two parts. The specific representation is as follows: Simultaneously, integrating modality-specific features and Furthermore, the fused features and single-modal features are further fused by weighting different features; Among them, weight These are learnable parameters.

[0012] As a further improvement to the present invention, the specific way to further integrate clinical measurement features in step four is as follows: by... and learned metric features The final feature is obtained by splicing. Then, the probability of a patient having a high risk of coronary heart disease is learned through a fully connected layer. The specific process is as follows: in, This represents the probability of a patient having a high risk of coronary heart disease; the higher the value, the higher the risk.

[0013] The beneficial effects of this invention are as follows: This invention analyzes the characteristics of three modalities—clinical data, CT images, and ECG signals—and extracts the clinical measurement features, spatial features, and temporal features contained therein. Then, based on the cross-attention mechanism, it learns the interaction features between different modal features. While capturing the specific features of different modalities, it also learns the cross-modal interaction features. By analyzing the acquired clinical data, CT images, and ECG signals, the invention can be used for coronary heart disease risk assessment. Attached Figure Description

[0014] Figure 1 This is a flowchart of the present invention; Figure 2 This is a structural diagram of the image spatial feature extractor used in this invention; Figure 3 This is a structural diagram of the ECG temporal feature extractor used in this invention; Figure 4 This is a structural diagram of the cross-modal feature fusion model used in this invention. Detailed Implementation

[0015] The present invention will now be described in further detail with reference to the embodiments shown in the accompanying drawings.

[0016] Reference Figures 1 to 4 As shown in this embodiment, a multi-omics artificial intelligence risk assessment method for coronary heart disease based on cross-attention mechanism is described, and the method steps are as follows: Step 1: Data preprocessing to reduce noise generated during data acquisition; For clinical data, features with too many missing values ​​are first removed, while features with fewer missing values ​​are filled with the mean or median. Then, ANOVA variance is used to screen for features that show significant differences between high-risk and low-risk populations. Finally, z-score standardization is performed on the data to prevent different data volumes from affecting the experiment. To mitigate noise interference during data acquisition, such as baseline drift, for ECG data, information below 0.5 Hz is removed. Furthermore, research indicates that ECG signals above 45 Hz often lack important information; therefore, a Butterworth bandpass filter with a frequency range of 0.5–45 Hz is used to process the ECG data.

[0017] Step 2: Design different feature extractors based on the characteristics of different modal data to extract features of different modalities; Firstly, regarding the clinical data obtained from preprocessing... Further extraction of metric features through linear mapping Specifically, in It is a learnable weight matrix. For bias, This is the activation function.

[0018] Secondly, regarding CT image data Image features F are extracted using a spatial feature extractor. CT ; Specifically, firstly utilize Layered CNNs are used to extract features from CT images, specifically: When N=1, i.e., its initial characteristics .

[0019] To better extract key image features, a channel attention mechanism is used to learn the importance of different image channels. First, global average pooling is used to map image features to the channel dimension. Then, a fully connected layer learns the importance of different channels. Specifically... in and These represent the height and width of the image, respectively. These are the feature values ​​at the corresponding locations in the image. This is a learnable weight matrix. Then, low-level features and high-level features are associated through skip connections; the specific process can be represented as follows: Finally, the features F of the CT image are learned by superimposing Ns of this module. CT Coronary CT angiography images contain rich details and texture information. Multi-layer convolutions in the spatial feature extractor can progressively extract features from low to high levels. Then, skip connections are used to associate low-level and high-level features, improving generalization ability. Simultaneously, by introducing a channel attention mechanism, key features can be selectively focused on, further enhancing the feature extraction effect.

[0020] Finally, regarding ECG signal data Feature F is extracted using a temporal feature extractor. ECG ; Continuous ECG signals Divided into A length of For vectors ; The ECG representation is constructed by concatenating representations learned from a forward and backward LSTM; specifically: in Indicates the forward process, For backward process, This represents a splicing operation.

[0021] and By introducing an input gate Forgotten Gate Output gate Three gating mechanisms are used to learn long-term dependencies in ECG signals. The specific process is as follows: Forgetting Gate In other words: For input gate part: For output gate part: in For the current input, for The hidden state at that time These are all learnable weight parameters. It's a bias. Used for message passing. The hyperbolic tangent activation function is used. This is a dot product operation.

[0022] In summary, LSTM, through its memory units and gating mechanisms, can capture long-term dependencies in ECG signals, especially certain features that span multiple cardiac cycles. Furthermore, by employing forward LSTM and backward LSTM, it can simultaneously process both forward and backward information from sequence data. This bidirectional information capture capability contributes to a more comprehensive understanding of the dynamic characteristics of cardiac activity.

[0023] Step 3: Achieve cross-modal spatiotemporal feature fusion of CT and ECG based on cross-attention mechanism; To better model the cross-modal interaction between CT images and ECGs, the learning process focuses more on relevant components of the CT and ECG signals, employing a cross-attention mechanism to effectively fuse and extract spatiotemporal information from both signals. Similar to traditional attention mechanisms, CT image features are first generated through three linear mapping layers. and ECG characteristics The Q (Query), K (Key), V (Value) matrix; For CT image data: Regarding ECG data: Then, a cross-attention mechanism is used to capture the dependencies between different input features. Specifically, key features are obtained from one modality, while query features are obtained from another. The similarity between the two features is then calculated using a dot product and weights are assigned, thus focusing more on the feature-related parts of CT and ECG during the learning process. This is specifically represented as follows: By applying the learned matrix A to the matrix V, we can obtain: Cross-modal fusion features of CT images and ECG data can be obtained by combining the features of the two parts. Its specific representation is as follows: Simultaneously, integrating modality-specific features and Furthermore, the fused features and single-modal features are further fused by weighting different features; Among them, weight These are learnable parameters.

[0024] Step 4: Further integrate clinical measurement features and learn the probability of high risk of coronary heart disease in patients through fully connected layers; After obtaining features that integrate CT image spatial information and ECG temporal information. Afterwards, by and learned metric features The final feature is obtained by splicing. Then, the probability of a patient having a high risk of coronary heart disease is learned through a fully connected layer. The specific process is as follows: in This indicates the probability of a patient having a high risk of coronary heart disease. The higher the value, the higher the risk, and the more timely the intervention and treatment are needed.

[0025] In summary, the AI-based risk assessment method for coronary heart disease based on cross-attention mechanism in this embodiment integrates different patient data (clinical data, CT images, ECG signals) and learns their interactive features, thereby improving the accuracy of coronary heart disease risk assessment results.

[0026] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A coronary artery disease risk assessment method based on cross-attention mechanism, characterized in that: Includes the following steps: Step 1: Collect three modalities of data for coronary heart disease: clinical data, CT images, and ECG signals. Preprocess the collected multimodal data by removing features with excessive missing values ​​and filling features with fewer missing values ​​using the mean or median. Then, use ANOVA variance screening to identify features that show significant differences between high-risk and low-risk populations. Finally, perform z-score standardization on the data. For ECG data, remove information below 0.5 Hz and process the ECG using a Butterworth bandpass filter with a frequency range of 0.5–45 Hz. Step two: Based on the characteristics of different modal data, different feature extractors are used to extract features of different modalities from the data processed in step one. Step 3: Based on the data features extracted in Step 2, cross-modal spatiotemporal feature fusion of CT image data and ECG data is achieved using a cross-attention mechanism; Step four involves further integrating clinical metrics and learning the probability of a patient having a high risk of coronary heart disease through a fully connected layer.

2. The coronary artery disease artificial intelligence risk assessment method based on cross-attention mechanism according to claim 1, characterized in that: In step two, different feature extractors are provided to extract features for different modalities based on the characteristics of different modal data. The specific steps are as follows: For the clinical data obtained after step one preprocessing Further extraction of metric features through linear mapping The details are as follows: in, It is a learnable weight matrix. For bias, The activation function is used for the CT image data obtained after preprocessing in step one. Image features F are extracted using a spatial feature extractor. CT； For the ECG signal data after step one preprocessing Feature F is extracted using a temporal feature extractor. ECG .

3. The coronary artery disease risk assessment method based on cross-attention mechanism according to claim 2, characterized in that: The image features F are extracted using a spatial feature extractor. CT The specific method is as follows: First, use Layered CNNs are used to extract features from CT images, specifically: When N=1, i.e., its initial characteristics Then, global average pooling is used to map image features to the channel dimension, and then a fully connected layer is used to learn the importance of different channels. Specifically, in and These represent the height and width of the image, respectively. These are the feature values ​​at the corresponding locations in the image. The weight matrix is ​​a learnable matrix, and then low-level features and high-level features are associated through skip connections; The specific process can be represented as follows: Finally, the features F of the CT image are learned by superimposing Ns of this module. CT .

4. The coronary artery disease artificial intelligence risk assessment method based on cross-attention mechanism according to claim 3, characterized in that: The feature F is extracted by the time-series feature extractor. ECG The specific method is as follows: [The text abruptly ends here, likely due to an incomplete sentence or a formatting error. Divided into A length of For vectors ; The ECG representation is constructed by concatenating representations learned from a forward and backward LSTM; specifically: in, Indicates the forward process, For backward process, This represents a splicing operation; By introducing an input gate Forgotten Gate Output gate Three gating mechanisms are used to learn long-term dependencies in ECG signals. The specific process is as follows: Forgetting Gate In other words: For input gate part: For output gate part: in, For the current input, for The hidden state at that time These are all learnable weight parameters. It's a bias. Used for message passing. The hyperbolic tangent activation function is used. This is a dot product operation.

5. The coronary artery disease artificial intelligence risk assessment method based on cross-attention mechanism according to claim 4, characterized in that: The specific steps for cross-modal spatiotemporal feature fusion of CT image data and ECG data based on the cross-attention mechanism in step three are as follows: First, CT image features are generated through three linear mapping layers. and ECG characteristics The Q (Query), K (Key), V (Value) matrix; For CT image data: Regarding ECG data: Then, a cross-attention mechanism is used to capture the dependencies between different input features. By combining the features from both parts, the cross-modal fusion features of CT images and ECG data can be obtained. .

6. The coronary artery disease artificial intelligence risk assessment method based on cross-attention mechanism according to claim 5, characterized in that: The specific method for capturing the dependencies between different input features through the cross-attention mechanism is as follows: Key features are obtained from one modality, and query features are obtained from another modality. Then, the similarity between the two features is calculated using a dot product and weights are assigned. This allows for greater focus on feature-related parts of CT and ECG during the learning process, as detailed below: By applying the learned matrix A to the matrix V, we can obtain: 。 7. The coronary artery disease artificial intelligence risk assessment method based on cross-attention mechanism according to claim 6, characterized in that: The cross-modal fusion features of CT images and ECG data can be obtained by combining the features of the two parts. The specific representation is as follows: Simultaneously, integrating modality-specific features and Furthermore, the fused features and single-modal features are further fused by weighting different features; Among them, weight These are learnable parameters.

8. The coronary artery disease artificial intelligence risk assessment method based on cross-attention mechanism according to claim 7, characterized in that: The specific method for further integrating clinical measurement features in step four is as follows: by... and learned metric features The final feature is obtained by splicing. Then, the probability of a patient having a high risk of coronary heart disease is learned through a fully connected layer. The specific process is as follows: in, This represents the probability of a patient having a high risk of coronary heart disease; the higher the value, the higher the risk.

Images