Embolism and massive bleeding risk prediction method and system based on causal Transformer model

Abstract

The present invention provides a method and system for predicting the risk of embolism and massive bleeding based on a causal Transformer model, including: step S1: obtaining clinical data of patients with atrial fibrillation, and preprocessing the obtained patient clinical data to obtain preprocessed patient clinical data; step S2: standardizing the preprocessed patient clinical data of different time series lengths, decomposing the standardized patient clinical data into dynamic variables and static variables, and generating multidimensional features; step S3: constructing a prediction model based on the Causal Transformer model, and training the prediction model using multidimensional features to obtain a trained prediction model; step S4: using the trained prediction model to predict the probability of embolism and massive bleeding of the target object within a preset time period in the future.

Description

Technical Field

[0001] The present invention relates to the technical field of disease prediction through big data, and specifically to a method and system for predicting the risk of embolism and massive bleeding based on a causal Transformer model. Background Art

[0002] With the rapid development of artificial intelligence (AI), deep learning-based models have been widely used in the medical field. In complex systems, causal inference and counterfactual prediction techniques have gradually become important tools for optimizing decision-making and improving system performance. However, existing technologies in the fields of causal modeling and counterfactual inference still have many shortcomings, as reflected in the following aspects:

[0003] (1) Inability to effectively capture time series characteristics: In many practical problems, data often have time series characteristics, such as records of patients' dynamic physiological parameters in the medical field or historical data of market fluctuations in the financial field. However, when processing time series data, existing models often fail to fully utilize the long-term dependencies and dynamic change characteristics in the sequence, resulting in inaccurate performance of the model in causal inference;

[0004] (2) Insufficient modeling of the interaction between static and dynamic features: Most current technologies treat static and dynamic features independently and fail to fully model the relationship between them. For example, static features (basic information such as age and gender) may significantly affect dynamic features (such as the changing trend of heart rate and blood pressure), but the lack of effective modeling of the interaction between static and dynamic features will lead to deviations in causal inference results and reduce the prediction accuracy of the model;

[0005] (3) Counterfactual results lack logical constraints: Existing causal inference models often rely on data-driven optimization when generating counterfactual predictions, lacking the constraints of logical rules. For example, the difference between counterfactual results and actual results may violate medical common sense or business logic, resulting in unreasonable model predictions and reducing the reliability and credibility of the application.

[0006] (4) Limited multi-task modeling capabilities: In many practical scenarios, a model needs to support multiple tasks simultaneously. For example, in the medical field, it is necessary to evaluate the effectiveness of treatment and predict potential side effects. However, traditional causal inference models can usually only handle a single task, resulting in insufficient model generalization capabilities and an inability to meet the needs of complex scenarios;

[0007] (5) Lack of effective integration of domain knowledge: In the medical field, rich domain knowledge can provide important guidance for model training and optimization. However, traditional technologies rarely utilize this domain knowledge, resulting in the lack of interpretability and practical guidance of the results of causal inference models. Summary of the Invention

[0008] In view of the defects in the prior art, the purpose of the present invention is to provide a method and system for predicting the risk of embolism and massive bleeding based on a causal Transformer model.

[0009] According to the present invention, a method for predicting the risk of embolism and massive bleeding based on a causal Transformer model is provided, comprising:

[0010] Step S1: Acquire clinical data of a patient suffering from atrial fibrillation, and preprocess the acquired clinical data of the patient to obtain preprocessed clinical data of the patient;

[0011] Step S2: performing standardization on the pre-processed patient clinical data of different time series lengths, decomposing the standardized patient clinical data into dynamic variables and static variables, and generating multidimensional features;

[0012] Step S3: Build a prediction model based on the Causal Transformer model, and train the prediction model using multi-dimensional features to obtain a trained prediction model;

[0013] Step S4: using the trained prediction model to predict the probability of embolism and massive bleeding in the target subject within a preset time period in the future.

[0014] Preferably, step S1 includes: obtaining clinical data of patients with atrial fibrillation, screening the patients' clinical data, deleting the patients' clinical data with a missing ratio greater than a preset value, retaining the patients' clinical data with a missing ratio less than or equal to the preset value, and filling in the missing values ​​to obtain preprocessed patients' clinical data.

[0015] Preferably, step S2 includes: aligning sequence lengths and filling missing values ​​on the pre-processed patient clinical data of different time series lengths, and decomposing the processed patient clinical data into dynamic variables and static variables.

[0016] Preferably, the prediction model constructed based on the Causal Transformer model includes: a multi-head attention module, a feedforward layer and a normalization layer;

[0017] The multi-head attention module includes a multi-head self-attention layer and a cross-attention layer;

[0018] The normalization layer is used to normalize the input sequence to obtain a normalized sequence; the normalized sequence is input into the multi-head self-attention layer to capture the relationship between different positions within the input sequence;

[0019] The normalization layer is used to normalize the dynamic features and the static features to obtain normalized dynamic features and static features; the normalized dynamic features and static features are input into the cross attention layer to capture the interactive relationship between the dynamic features and the static features;

[0020] Normalizing the relationship between different positions within the captured sequence and the interaction relationship between the captured dynamic features and static features to obtain the normalized relationship between different positions within the sequence and the interaction relationship between the captured dynamic features and static features;

[0021] Based on the relationship between different positions within the normalized sequence and the interactive relationship between dynamic features and static features, deep features are extracted through the fully connected network in the feedforward layer; the deep features are normalized to obtain normalized deep features.

[0022] Preferably, the step S3 further includes: loss functions and constraints during training;

[0023] The loss function includes:

[0024]

[0025] in, represents the domain classifier; represents the representation extracted by the main network; Represents the parameters of the main network; represents the parameters of the domain classifier; represents the number of treatment assignment categories; represents the logarithm of the treatment assignment probability predicted by the domain classifier;

[0026] The constraints include: counterfactual balance constraints and common sense constraints;

[0027] The counterfactual balance constraints include: constraints on the output of the Causal Transformer model;

[0028]

[0029] Among them, M is the valid sample set; is the treatment assignment of the i-th sample, where T=0 means untreated and T=1 means treated; is the model's prediction of the counterfactual outcome;

[0030]

[0031] The common sense constraints include:

[0032] Constraints on the probability of massive bleeding and embolism based on medical common sense:

[0033] Embolism: The probability in the same patient when taking anticoagulants should not be higher than when not taking anticoagulants;

[0034] Major bleeding: The probability for the same patient not taking anticoagulants should be no higher than the probability for the same patient taking anticoagulants.

[0035] Preferably, the method further comprises: calculating the individual's net clinical benefit NCB;

[0036]

[0037] When NCB>0, anticoagulants are recommended, and when NCB≤0, anticoagulants are not recommended.

[0038] According to the present invention, a system for predicting the risk of embolism and massive bleeding based on a causal Transformer model is provided, comprising:

[0039] Module M1: acquiring clinical data of a patient suffering from atrial fibrillation, and preprocessing the acquired clinical data of the patient to obtain preprocessed clinical data of the patient;

[0040] Module M2: Standardizes the pre-processed patient clinical data of different time series lengths, decomposes the standardized patient clinical data into dynamic variables and static variables, and generates multidimensional features;

[0041] Module M3: Build a prediction model based on the Causal Transformer model and train the prediction model using multi-dimensional features to obtain the trained prediction model;

[0042] Module M4: Use the trained prediction model to predict the probability of embolism and massive bleeding in the target subject within a preset time period in the future.

[0043] Preferably, the module M1 comprises: acquiring clinical data of patients with atrial fibrillation, screening the clinical data of the patients, deleting the clinical data of patients with a missing ratio greater than a preset value, retaining the clinical data of patients with a missing ratio less than or equal to the preset value, and filling in the missing values ​​to obtain pre-processed clinical data of the patients;

[0044] The module M2 includes: aligning sequence lengths and filling missing values ​​for pre-processed patient clinical data of different time series lengths, and decomposing the processed patient clinical data into dynamic variables and static variables.

[0045] Preferably, the prediction model constructed based on the Causal Transformer model includes: a multi-head attention module, a feedforward layer and a normalization layer;

[0046] The multi-head attention module includes a multi-head self-attention layer and a cross-attention layer;

[0047] The normalization layer is used to normalize the input sequence to obtain a normalized sequence; the normalized sequence is input into the multi-head self-attention layer to capture the relationship between different positions within the input sequence;

[0048] The normalization layer is used to normalize the dynamic features and the static features to obtain normalized dynamic features and static features; the normalized dynamic features and static features are input into the cross attention layer to capture the interactive relationship between the dynamic features and the static features;

[0049] Normalizing the relationship between different positions within the captured sequence and the interaction relationship between the captured dynamic features and static features to obtain the normalized relationship between different positions within the sequence and the interaction relationship between the captured dynamic features and static features;

[0050] Based on the relationship between different positions within the normalized sequence and the interactive relationship between dynamic features and static features, deep features are extracted through the fully connected network in the feedforward layer; the deep features are normalized to obtain normalized deep features.

[0051] Preferably, the module M3 further includes: a loss function and constraints during training;

[0052] The loss function includes:

[0053]

[0054] in, represents the domain classifier; represents the representation extracted by the main network; Represents the parameters of the main network; represents the parameters of the domain classifier; represents the number of treatment assignment categories; represents the logarithm of the treatment assignment probability predicted by the domain classifier;

[0055] The constraints include: counterfactual balance constraints and common sense constraints;

[0056] The counterfactual balance constraints include: constraints on the output of the Causal Transformer model;

[0057]

[0058] Among them, M is the valid sample set; is the treatment assignment of the i-th sample, where T=0 means untreated and T=1 means treated; is the model's prediction of the counterfactual outcome;

[0059]

[0060] The common sense constraints include:

[0061] Constraints on the probability of massive bleeding and embolism based on medical common sense:

[0062] Embolism: The probability in the same patient when taking anticoagulants should not be higher than when not taking anticoagulants;

[0063] Major bleeding: The probability for the same patient not taking anticoagulants should be no higher than the probability for the same patient taking anticoagulants.

[0064] Compared with the prior art, the present invention has the following beneficial effects:

[0065] 1. This paper uses a causal Transformer framework based on multi-head self-attention and cross-attention mechanisms to achieve deep interactive modeling of dynamic and static features, significantly improving the accuracy of causal relationship modeling;

[0066] 2. By introducing counterfactual balance constraints, the present invention significantly improves the logical consistency and clinical interpretability of the model for counterfactual results;

[0067] 3. This paper introduces a logical constraint loss function to achieve the rationality of counterfactual prediction results in multi-task scenarios;

[0068] 4. This invention improves the collaborative efficiency between different prediction tasks through a joint optimization method based on multi-task learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Other features, objects and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0070] Figure 1 This is a flowchart of the embolism and massive bleeding risk prediction method based on the causal Transformer model.

[0071] Figure 2 Schematic diagram of building a prediction model based on the Causal Transformer model. DETAILED DESCRIPTION

[0072] The present invention will be described in detail below with reference to specific embodiments. The following examples will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those skilled in the art, several changes and improvements can be made without departing from the scope of the present invention. These all fall within the scope of protection of the present invention.

[0073] Example 1

[0074] According to the present invention, a method for predicting the risk of embolism and massive bleeding based on a causal Transformer model is provided. Figures 1 to 2 Shown, including:

[0075] Step S1: Acquire clinical data of a patient suffering from atrial fibrillation, and preprocess the acquired clinical data of the patient to obtain preprocessed clinical data of the patient;

[0076] In this embodiment, step S1 includes: examining and screening the clinical data of patients who meet the study inclusion criteria, determining the variables that need to be included in the model training, deleting variables with a missing ratio greater than 20% and case data with outliers, and filling in variables with a missing ratio ≤ 20%, thereby obtaining cleaned data;

[0077] In this example, the inclusion criteria were: patients enrolled in CHINA-AF from August 2011 to December 2022, approximately 34,959 people; the exclusion criteria (recording the number of people excluded at each step in order) included:

[0078] 1. Age missing;

[0079] 2. Age < 18 years old;

[0080] 3. Valvular atrial fibrillation (valvuar_AF_000=2);

[0081] 4. Hypertrophic cardiomyopathy (HCM=1);

[0082] 5. Use of warfarin at baseline or during follow-up (warfarin = 1 or warf_his_m = 1 at any time;

[0083] 6. Severe renal insufficiency (eGFR<30);

[0084] 7. Abnormal liver function (Tbil_000>42 or ALT_000>150 or AST_000>120).

[0085] Step S2: performing standardization on the pre-processed patient clinical data of different time series lengths, decomposing the standardized patient clinical data into dynamic variables and static variables, and generating multidimensional features;

[0086] In this embodiment, patients with different time series lengths are standardized, for example, operations such as aligning sequence lengths and filling missing values ​​are performed, and features are refined and decomposed into dynamic variables and static variables to generate multidimensional features, thereby ensuring the integrity and expressiveness of the dataset.

[0087] Step S3: Build a prediction model based on the Causal Transformer model, and train the prediction model using multi-dimensional features to obtain a trained prediction model;

[0088] In this example, the Causal Transformer model combines patients' dynamic clinical characteristics during long-term follow-up and infers both factual and counterfactual outcomes between different anticoagulant treatment regimens and adverse prognostic events (embolism and major bleeding) to provide personalized risk predictions for embolism and major bleeding. This model employs long-term dependency modeling and counterfactual domain confounding loss to mitigate confounding bias. Furthermore, it incorporates common medical knowledge constraints to ensure that predictions are consistent with clinical logic. Furthermore, through the principle of clinical net benefit, which provides personalized predictions of embolism and major bleeding risks based on whether or not anticoagulants are being taken, it provides reliable anticoagulant decision support, helping clinicians make scientific and accurate decisions regarding anticoagulant therapy.

[0089] More specifically, the Causal Transformer model is a novel architecture designed to capture complex long-term dependencies in time series data, especially the effects of temporal confounding factors. It includes a multi-head attention module, a feedforward layer, and a normalization layer.

[0090] The multi-head attention module includes a multi-head self-attention layer and a cross-attention layer. Self-attention captures the relationship between different positions in the input sequence (such as long-range dependencies in time series), while cross-attention models the interaction between two feature sets (such as dynamic features and static features).

[0091] More specifically, the normalization layer is used to normalize the input sequence to obtain a normalized sequence; the normalized sequence is input into the multi-head self-attention layer to capture the relationship between different positions within the input sequence;

[0092] The normalization layer is used to normalize the dynamic features and the static features to obtain normalized dynamic features and static features; the normalized dynamic features and static features are input into the cross attention layer to capture the interactive relationship between the dynamic features and the static features;

[0093] Normalizing the relationship between different positions within the captured sequence and the interaction relationship between the captured dynamic features and static features to obtain the normalized relationship between different positions within the sequence and the interaction relationship between the captured dynamic features and static features;

[0094] Based on the relationship between different positions within the normalized sequence and the interactive relationship between dynamic features and static features, deep features are extracted through the fully connected network in the feedforward layer; the deep features are normalized to obtain normalized deep features.

[0095] Among them, the normalization layer normalizes the input of each layer to stabilize model training, accelerate convergence, and alleviate the problem of gradient disappearance or explosion.

[0096] Assume that the input sequence is , where N represents the sequence length and d represents the feature dimension.

[0097] Multi-head attention mechanism

[0098] For input X, we first generate query (Query, Q), key (Key, K) and value (Value, V) matrices:

[0099]

[0100] in, is a learnable weight matrix; is the number of heads;

[0101] Calculate the output through the attention mechanism:

[0102]

[0103] For Multi-head Attention, the results of multiple heads are concatenated and projected into the final output space:

[0104]

[0105] in, It is the result of MHA;

[0106] Layer Normalization

[0107]

[0108]

[0109] in, are scale and translation parameters, is the element-wise product;

[0110] Feedforward layer

[0111]

[0112] Layer Normalization

[0113]

[0114] The training process also includes loss functions and constraints;

[0115] The loss function includes:

[0116] This loss aims to learn a balanced representation that is predictive of future outcomes but unpredictable of current treatment assignments, thereby attenuating the effects of confounding factors.

[0117] Through adversarial learning methods, we ensure that the model simultaneously optimizes predictive power and robustness of causal inference.

[0118]

[0119] in, : Domain classifier, predicting treatment assignment. : The representation extracted by the main network. : Parameters of the main network (Representation Network). : Parameters of the domain classifier. : The number of treatment assignment categories (the dimension of the categorical space). : The logarithm of the treatment assignment probability predicted by the domain classifier.

[0120] Counterfactual balance and common sense constraints

[0121] Counterfactual balance: For each patient's real-world data, only a single factual outcome (either taking or not taking anticoagulants) is observed. However, in counterfactual predictions, the unobserved treatment assignments need to be properly modeled to prevent the model from making unreasonable predictions about the counterfactual outcome. To this end, we propose a balance constraint between the factual and counterfactual outcomes to ensure that the predictions are clinically interpretable on a logical basis.

[0122] Take TE as an example:

[0123] Batch loss calculation formula

[0124] reversed_loss

[0125]

[0126] Where: M is the valid sample set (determined by valid_mask). is the treatment assignment of the i-th sample (T=0 means untreated, T=1 means treated). is the model's prediction of the counterfactual outcome.

[0127]

[0128] reversed_loss emphasizes the separate modeling of facts and counterfactuals; treat_loss ensures a logical balance between factual and counterfactual predictions.

[0129] Common sense constraints:

[0130] Constraints on the probability of massive bleeding and embolism based on medical common sense:

[0131] Embolism: The probability in the same patient should be no higher when taking anticoagulants than when not taking anticoagulants.

[0132] Major bleeding: The probability for the same patient not taking anticoagulants should be no higher than the probability for the same patient taking anticoagulants.

[0133] By embedding these common-sense rules into model training, the model's prediction range is limited, thereby improving the credibility of the prediction results.

[0134] Factual event preference: For observed factual events (such as severe bleeding when taking anticoagulants), the model prefers to predict that the event did not occur in the counterfactual scenario (such as there would be no severe bleeding if anticoagulants were not taken) to enhance the rationality and stability of the counterfactual results.

[0135] By introducing innovative methods of counterfactual balance and common sense constraints, the model's causal inference ability can be effectively improved, ensuring that it is more in line with the logic and practical needs of the medical field when predicting counterfactual results.

[0136] Step S4: using the trained prediction model to predict the probability of embolism and massive bleeding in the target subject within a preset time period in the future.

[0137] Step S5: Calculate the individual's Net Clinical Benefit (NCB) according to the formula: NCB = (Probability of embolism 不服抗凝 -Probability of embolism 服抗凝 )-1.5×(Major Bleeding Probability 服抗凝 - Chance of major bleeding 不服抗凝 Here, anticoagulants are recommended when NCB>0, and are not recommended when NCB≤0.

[0138] The present invention also provides an embolism and massive bleeding risk prediction system based on a causal Transformer model. The embolism and massive bleeding risk prediction system based on a causal Transformer model can be implemented by executing the process steps of the embolism and massive bleeding risk prediction method based on a causal Transformer model. That is, those skilled in the art can understand the embolism and massive bleeding risk prediction method based on a causal Transformer model as a preferred implementation of the embolism and massive bleeding risk prediction system based on a causal Transformer model.

[0139] Those skilled in the art will appreciate that, in addition to implementing the system, device, and various modules provided by the present invention in purely computer-readable program code, it is entirely possible to implement the same program in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, embedded microcontrollers, and the like by logically programming the method steps. Therefore, the system, device, and various modules provided by the present invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; the modules for implementing various functions can also be considered both software programs for implementing the method and structures within the hardware component.

[0140] The above describes specific embodiments of the present invention. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art may make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. The embodiments of this application and the features in the embodiments may be combined with each other in any manner unless there is a conflict.

Claims

1. A method for predicting the risk of embolism and massive bleeding based on a causal Transformer model, characterized in that: include: Step S1: Acquire clinical data of a patient suffering from atrial fibrillation, and preprocess the acquired clinical data of the patient to obtain preprocessed clinical data of the patient; Step S2: performing standardization on the pre-processed patient clinical data of different time series lengths, decomposing the standardized patient clinical data into dynamic variables and static variables, and generating multidimensional features; Step S3: Build a prediction model based on the Causal Transformer model, and train the prediction model using multi-dimensional features to obtain a trained prediction model; Step S4: using the trained prediction model to predict the probability of embolism and massive bleeding in the target subject within a preset time period in the future; The prediction model constructed based on the Causal Transformer model includes: a multi-head attention module, a feedforward layer, and a normalization layer; The multi-head attention module includes a multi-head self-attention layer and a cross-attention layer; The normalization layer is used to normalize the input sequence to obtain a normalized sequence; the normalized sequence is input into the multi-head self-attention layer to capture the relationship between different positions within the input sequence; The normalization layer is used to normalize the dynamic features and the static features to obtain normalized dynamic features and static features; the normalized dynamic features and static features are input into the cross attention layer to capture the interactive relationship between the dynamic features and the static features; Normalizing the relationship between different positions within the captured sequence and the interaction relationship between the captured dynamic features and static features to obtain the normalized relationship between different positions within the sequence and the interaction relationship between the captured dynamic features and static features; Based on the relationship between different positions in the normalized sequence and the interaction between dynamic features and static features, deep features are extracted through the fully connected network in the feedforward layer; the deep features are normalized to obtain normalized deep features; The step S3 also includes: loss function and constraints during training; The loss function includes: in, represents the domain classifier; represents the representation extracted by the main network; Represents the parameters of the main network; represents the parameters of the domain classifier; represents the number of treatment assignment categories; represents the logarithm of the treatment assignment probability predicted by the domain classifier; The constraints include: counterfactual balance constraints and common sense constraints; The counterfactual balance constraints include: constraints on the output of the Causal Transformer model; Among them, M is the valid sample set; is the treatment assignment of the i-th sample, where T=0 means untreated and T=1 means treated; is the model's prediction of the counterfactual outcome; The common sense constraints include: Constraints on the probability of massive bleeding and embolism based on medical common sense: Embolism: The probability in the same patient when taking anticoagulants should not be higher than when not taking anticoagulants; Major bleeding: The probability of the same patient not taking anticoagulants should be no higher than the probability of the same patient taking anticoagulants; The method further comprises: calculating the individual's net clinical benefit NCB; When NCB>0, anticoagulants are recommended, and when NCB≤0, anticoagulants are not recommended.

2. The method for predicting the risk of embolism and massive bleeding based on the causal Transformer model according to claim 1, characterized in that: The step S1 includes: obtaining clinical data of patients with atrial fibrillation, screening the clinical data of the patients, deleting the clinical data of patients with a missing ratio greater than a preset value, retaining the clinical data of patients with a missing ratio less than or equal to the preset value, and filling in the missing values ​​to obtain preprocessed clinical data of the patients.

3. The method for predicting the risk of embolism and massive bleeding based on the causal Transformer model according to claim 1, characterized in that: The step S2 includes: aligning sequence lengths and filling missing values ​​for pre-processed patient clinical data of different time series lengths, and decomposing the processed patient clinical data into dynamic variables and static variables.

4. A causal Transformer model-based embolism and massive bleeding risk prediction system, characterized by: include: Module M1: acquiring clinical data of a patient suffering from atrial fibrillation, and preprocessing the acquired clinical data of the patient to obtain preprocessed clinical data of the patient; Module M2: Standardizes the pre-processed patient clinical data of different time series lengths, decomposes the standardized patient clinical data into dynamic variables and static variables, and generates multidimensional features; Module M3: Build a prediction model based on the Causal Transformer model and train the prediction model using multi-dimensional features to obtain the trained prediction model; Module M4: Use the trained prediction model to predict the probability of embolism and massive bleeding in the target subject within a preset time period in the future; The prediction model constructed based on the Causal Transformer model includes: a multi-head attention module, a feedforward layer, and a normalization layer; The multi-head attention module includes a multi-head self-attention layer and a cross-attention layer; The normalization layer is used to normalize the input sequence to obtain a normalized sequence; the normalized sequence is input into the multi-head self-attention layer to capture the relationship between different positions within the input sequence; The normalization layer is used to normalize the dynamic features and the static features to obtain normalized dynamic features and static features; the normalized dynamic features and static features are input into the cross attention layer to capture the interactive relationship between the dynamic features and the static features; Normalizing the relationship between different positions within the captured sequence and the interaction relationship between the captured dynamic features and static features to obtain the normalized relationship between different positions within the sequence and the interaction relationship between the captured dynamic features and static features; Based on the relationship between different positions in the normalized sequence and the interaction between dynamic features and static features, deep features are extracted through the fully connected network in the feedforward layer; the deep features are normalized to obtain normalized deep features; The module M3 also includes: loss functions and constraints during training; The loss function includes: in, represents the domain classifier; represents the representation extracted by the main network; Represents the parameters of the main network; represents the parameters of the domain classifier; represents the number of treatment assignment categories; represents the logarithm of the treatment assignment probability predicted by the domain classifier; The constraints include: counterfactual balance constraints and common sense constraints; The counterfactual balance constraints include: constraints on the output of the Causal Transformer model; Among them, M is the valid sample set; is the treatment assignment of the i-th sample, where T=0 means untreated and T=1 means treated; is the model's prediction of the counterfactual outcome; The common sense constraints include: Constraints on the probability of massive bleeding and embolism based on medical common sense: Embolism: The probability in the same patient when taking anticoagulants should not be higher than when not taking anticoagulants; Major bleeding: The probability for the same patient not taking anticoagulants should be no higher than the probability for the same patient taking anticoagulants.

5. The embolism and massive bleeding risk prediction system based on the causal Transformer model according to claim 4, characterized in that: The module M1 includes: obtaining clinical data of patients with atrial fibrillation, screening the clinical data of the patients, deleting the clinical data of patients with a missing ratio greater than a preset value, retaining the clinical data of patients with a missing ratio less than or equal to the preset value, and filling in the missing values ​​to obtain pre-processed clinical data of the patients; The module M2 includes: aligning sequence lengths and filling missing values ​​for pre-processed patient clinical data of different time series lengths, and decomposing the processed patient clinical data into dynamic variables and static variables.

Images