Embolism and hemorrhage risk prediction method and system based on causal Transform model

Abstract

The invention provides an embolism and hemorrhage risk prediction method and system based on a causal Transform model, and the method comprises the steps: S1, obtaining the clinical data of a patient suffering from atrial fibrillation, and carrying out the preprocessing of the obtained clinical data of the patient, and obtaining the preprocessed clinical data of the patient; s2, performing standardization processing on the preprocessed clinical data of the patients with different time sequence lengths, decomposing the standardized clinical data of the patients into dynamic variables and static variables, and generating multi-dimensional features; s3, constructing a prediction model based on a Causic Transform model, and training the prediction model by using the multi-dimensional features to obtain a trained prediction model; and S4, predicting the probability of embolism and massive hemorrhage of the target object in a future preset time period by using the trained prediction model.

Description

Technical Field

[0001] The present invention relates to the technical field of disease prediction through big data, and specifically, to a method and system for predicting embolism and major bleeding risks based on a causal Transformer model. Background Art

[0002] With the rapid development of artificial intelligence technology, deep learning-based models have been widely applied in the medical field. In complex systems, causal inference and counterfactual prediction technologies have gradually become important tools for optimizing decisions and improving system performance. However, there are still many deficiencies in the existing technologies in the fields of causal modeling and counterfactual inference, which are specifically reflected in the following aspects: (1) Unable to effectively capture time series characteristics: In many practical problems, data usually has time series characteristics, such as the recording of patients' dynamic physiological parameters in the medical field or the historical data of market fluctuations in the financial field. However, existing models often cannot fully utilize the long-term dependence relationships and dynamic change features in the sequence when processing time series data, resulting in inaccurate performance of the models in causal inference; (2) Insufficient interaction modeling between static and dynamic features: Most current technologies process static and dynamic features independently and fail to fully model the association between the two. For example, static features (such as basic information like age and gender) may significantly affect dynamic features (such as the change trends of heart rate and blood pressure), but the lack of effective modeling of the interaction between static and dynamic features will lead to biases in causal inference results and reduce the prediction accuracy of the models; (3) Lack of logical constraints on counterfactual results: Existing causal inference models usually rely on data-driven optimization when generating counterfactual predictions and lack the constraints of logical rules. For example, the differences between counterfactual results and factual results may violate medical common sense or business logic, resulting in unreasonable results predicted by the models and reducing the reliability and credibility of the applications; (4) Limited multi-task modeling ability: In many practical scenarios, a model needs to support multiple tasks simultaneously. For example, in the medical field, it is necessary to both evaluate the treatment effect and predict potential side effects. However, traditional causal inference models can usually only handle single tasks, resulting in insufficient generalization ability of the models and unable to meet the requirements of complex scenarios; (5) Lack of effective integration of domain knowledge: In the medical field, rich domain knowledge can provide important guidance for the training and optimization of models. However, traditional technologies rarely utilize this domain knowledge, resulting in the lack of interpretability and practical guiding significance of the results of causal inference models. Summary of the Invention

[0003] Aiming at the defects in the prior art, the purpose of the present invention is to provide a method and system for predicting embolism and major bleeding risks based on a causal Transformer model.

[0004] A method for predicting embolism and major bleeding risks based on a causal Transformer model provided by the present invention includes: Step S1: Obtain the clinical data of patients with atrial fibrillation, and preprocess the obtained patient clinical data to obtain the preprocessed patient clinical data; Step S2: Perform standardization processing on the preprocessed patient clinical data with different time series lengths, decompose the standardized patient clinical data into dynamic variables and static variables, and generate multi-dimensional features; Step S3: Construct a prediction model based on the Causal Transformer model, and use the multi-dimensional features to train the prediction model to obtain the trained prediction model; Step S4: Use the trained prediction model to predict the probabilities of embolism and major bleeding of the target object within a preset time period in the future.

[0005] Preferably, the step S1 includes: obtaining the clinical data of patients with atrial fibrillation, screening the patient clinical data, deleting the patient clinical data with a missing ratio greater than a preset value, retaining the patient clinical data with a missing ratio less than or equal to the preset value, and filling in the missing values to obtain the preprocessed patient clinical data.

[0006] Preferably, the step S2 includes: performing alignment of sequence lengths and filling in missing values on the preprocessed patient clinical data with different time series lengths, and decomposing the processed patient clinical data into dynamic variables and static variables.

[0007] Preferably, constructing the prediction model based on the Causal Transformer model includes: a multi-head attention module, a feed-forward layer, and a normalization layer; The multi-head attention module includes a multi-head self-attention layer and a cross-attention layer; Use the normalization layer to perform normalization processing on the input sequence to obtain the normalized sequence; input the normalized sequence into the multi-head self-attention layer to capture the relationships between different positions within the input sequence; Use the normalization layer to perform normalization processing on the dynamic features and static features to obtain the normalized dynamic features and static features; input the normalized dynamic features and static features into the cross-attention layer to capture the interaction relationships between the dynamic features and static features; Normalize the relationships between different positions within the captured sequence and the interaction relationships between the captured dynamic and static features to obtain the relationships between different positions within the normalized sequence and the interaction relationships between the captured dynamic and static features; Based on the relationships between different positions within the normalized sequence and the interaction relationships between the dynamic and static features, extract deep features through a fully connected network in the feed-forward layer; normalize the deep features to obtain the normalized deep features.

[0008] Preferably, step S3 further includes: the loss function and constraint conditions during the training process; The loss function includes:

[0009] Among them, represents the domain classifier; represents the representation extracted by the main network; represents the parameters of the main network; represents the parameters of the domain classifier; represents the number of treatment assignment categories; represents the logarithm of the treatment assignment probability predicted by the domain classifier; The constraint conditions include: counterfactual balance constraints and common sense constraints; Among them, the counterfactual balance constraint includes: constraints on the output of the Causal Transformer model;

[0010] Among them, M is the set of valid samples; is the treatment assignment of the i-th sample, where T = 0 indicates untreated and T = 1 indicates treated; is the model's prediction of the counterfactual result;

[0011] The common sense constraint includes: Constraints on the probabilities of massive hemorrhage and embolism based on medical common sense: Embolism: The probability when a patient takes anticoagulants should not be higher than the probability when not taking anticoagulants; Massive hemorrhage: The probability when a patient does not take anticoagulants should not be higher than the probability when taking anticoagulants.

[0012] Preferably, the method further includes: calculating the individual's clinical net benefit NCB;

[0013] When NCB > 0, anticoagulants are recommended; when NCB ≤ 0, anticoagulants are not recommended.

[0014] A thromboembolism and major bleeding risk prediction system based on a causal Transformer model provided by the present invention includes: Module M1: Obtain the clinical data of patients with atrial fibrillation, and preprocess the obtained patient clinical data to obtain the preprocessed patient clinical data; Module M2: Perform standardization processing on the preprocessed patient clinical data with different time series lengths, decompose the standardized patient clinical data into dynamic variables and static variables, and generate multi-dimensional features; Module M3: Construct a prediction model based on the Causal Transformer model, and use the multi-dimensional features to train the prediction model to obtain the trained prediction model; Module M4: Use the trained prediction model to predict the probabilities of thromboembolism and major bleeding in the target object within a preset time period in the future.

[0015] Preferably, the module M1 includes: obtaining the clinical data of patients with atrial fibrillation, screening the patient clinical data, deleting the patient clinical data with a missing ratio greater than a preset value, retaining the patient clinical data with a missing ratio less than or equal to the preset value, and filling the missing values to obtain the preprocessed patient clinical data; The module M2 includes: performing alignment of sequence lengths and filling of missing values on the preprocessed patient clinical data with different time series lengths, and decomposing the processed patient clinical data into dynamic variables and static variables.

[0016] Preferably, constructing the prediction model based on the Causal Transformer model includes: a multi-head attention module, a feed-forward layer, and a normalization layer; The multi-head attention module includes a multi-head self-attention layer and a cross-attention layer; Use the normalization layer to perform normalization processing on the input sequence to obtain the normalized sequence; input the normalized sequence into the multi-head self-attention layer to capture the relationships between different positions within the input sequence; Use the normalization layer to perform normalization processing on the dynamic features and static features to obtain the normalized dynamic features and static features; input the normalized dynamic features and static features into the cross-attention layer to capture the interaction relationships between the dynamic features and static features; Normalize the relationships between different positions within the captured sequence and the interaction relationships between the captured dynamic and static features to obtain the relationships between different positions within the normalized sequence and the interaction relationships between the captured dynamic and static features; Based on the relationships between different positions within the normalized sequence and the interaction relationships between the dynamic and static features, extract deep features through a fully connected network in the feedforward layer; normalize the deep features to obtain the normalized deep features.

[0017] Preferably, the module M3 further includes: the loss function and constraint conditions during training; The loss function includes:

[0018] Wherein, represents the domain classifier; represents the representation extracted by the main network; represents the parameters of the main network; represents the parameters of the domain classifier; represents the number of treatment assignment categories; represents the logarithm of the treatment assignment probability predicted by the domain classifier; The constraint conditions include: counterfactual balance constraints and common sense constraints; Among them, the counterfactual balance constraint includes: the constraint on the output of the Causal Transformer model;

[0019] Among them, M is the set of valid samples; is the treatment assignment of the i-th sample, where T = 0 indicates untreated and T = 1 indicates treated; is the prediction of the counterfactual result by the model;

[0020] The common sense constraint includes: Constraints on the probabilities of massive hemorrhage and embolism based on medical common sense: Embolism: The probability when the same patient takes anticoagulants should not be higher than the probability when not taking anticoagulants; Massive hemorrhage: The probability when the same patient does not take anticoagulants should not be higher than the probability when taking anticoagulants.

[0021] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention realizes the in-depth interactive modeling of dynamic features and static features through a causal Transformer framework based on the multi-head self-attention and cross-attention mechanisms, significantly improving the modeling accuracy of causal relationships. 2. The present invention significantly improves the logical consistency and clinical interpretability of the model for counterfactual results by introducing counterfactual balance constraints. 3. The present invention realizes the rationality of counterfactual prediction results in multi-task scenarios by introducing a logical constraint loss function. 4. The present invention improves the collaborative efficiency between different prediction tasks through a joint optimization method based on multi-task learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Other features, objects, and advantages of the present invention will become more apparent by reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings. Figure 1 It is a flowchart of an embolism and major bleeding risk prediction method based on a causal Transformer model.

[0023] Figure 2 It is a schematic diagram of constructing a prediction model based on a Causal Transformer model. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] The present invention will be described in detail below with reference to specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be noted that those of ordinary skill in the art can make several changes and improvements without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0025] Embodiment 1 A method for predicting embolism and major bleeding risks based on a causal Transformer model provided by the present invention, as Figures 1 to 2 shown, includes: Step S1: Obtain the clinical data of patients with atrial fibrillation, and preprocess the obtained patient clinical data to obtain the preprocessed patient clinical data. In this embodiment, the step S1 includes: checking and screening the clinical data of patients who meet the research inclusion criteria, determining the variables to be included in model training, deleting variables with a missing ratio > 20% and case data with outliers, and filling variables with a missing ratio ≤ 20%, so as to obtain the cleaned data. In this embodiment, the inclusion criteria: CHINA-AF enrolled patients from August 2011 to December 2022, about 34,959 people; the exclusion criteria (recording the number of people excluded at each step in order) include: 1. Those with missing age; 2. Age < 18 years old (age < 18); 3. Valvular atrial fibrillation (valvuar_AF_000 = 2); 4. Hypertrophic cardiomyopathy (HCM = 1); 5. Warfarin used at baseline or during follow - up (warfarin = 1 or warf_his_m has any occurrence = 1; 6. Severe renal insufficiency (eGFR < 30); 7. Abnormal liver function (Tbil_000 > 42 or ALT_000 > 150 or AST_000 > 120).

[0026] Step S2: Standardize the pre - processed patient clinical data with different time - series lengths, decompose the standardized patient clinical data into dynamic variables and static variables, and generate multi - dimensional features; In this embodiment, standardize the patients with different time - series lengths, for example, operations such as aligning the sequence lengths and filling in missing values, refine and decompose the features into dynamic variables and static variables, and generate multi - dimensional features to ensure the integrity and expressiveness of the data set.

[0027] Step S3: Construct a prediction model based on the Causal Transformer model, and use the multi - dimensional features to train the prediction model to obtain the trained prediction model; In this embodiment, the Causal Transformer model combines the dynamic clinical features of patients during long - term follow - up, infers the factual and counterfactual results of different anticoagulation treatment regimens and adverse prognostic events (embolism, major bleeding), and provides individualized risk prediction for embolism and major bleeding. This model uses long - term dependence modeling and counterfactual domain confusion loss to reduce confounding bias; at the same time, it incorporates medical common - sense constraints to ensure that the prediction results conform to clinical logic. Further, through the principle of clinical net benefit, that is, based on the individualized prediction of the risks of embolism and major bleeding with or without taking anticoagulants, it provides reliable anticoagulation decision - making support, thereby helping clinicians make scientific and accurate decisions in anticoagulation treatment selection.

[0028] More specifically, the Causal Transformer model is a novel architecture designed to capture complex long - term dependence relationships in time - series data, especially the influence of time - varying confounding factors; it includes: a multi - head attention module, a feed - forward layer, and a normalization layer; The multi-head attention module includes a multi-head self-attention layer and a cross-attention layer; among them, multi-head self-attention (Self-Attention): captures the relationships between different positions within the input sequence (such as long-range dependencies in a time series). Cross-attention: models the interaction relationships between two feature sets (such as dynamic features and static features).

[0029] More specifically, use the normalization layer to normalize the input sequence to obtain a normalized sequence; input the normalized sequence into the multi-head self-attention layer to capture the relationships between different positions within the input sequence; Use the normalization layer to normalize the dynamic features and static features to obtain normalized dynamic features and static features; input the normalized dynamic features and static features into the cross-attention layer to capture the interaction relationships between the dynamic features and static features; Normalize the captured relationships between different positions within the sequence and the captured interaction relationships between the dynamic features and static features to obtain the normalized relationships between different positions within the sequence and the captured interaction relationships between the dynamic features and static features; Based on the normalized relationships between different positions within the sequence and the interaction relationships between the dynamic features and static features, extract deep features through the fully connected network in the feed-forward layer; normalize the deep features to obtain normalized deep features.

[0030] Among them, the normalization layer normalizes the input of each layer to stabilize model training, accelerate convergence, and alleviate the problems of gradient disappearance or explosion.

[0031] Assume the input sequence is , where N represents the sequence length and d represents the feature dimension.

[0032] Multi-head attention mechanism For the input X, we first generate query (Query, Q), key (Key, K), and value (Value, V) matrices:

[0033] Among them, is a learnable weight matrix; is the number of heads; Calculate the output through the attention mechanism:

[0034] For multi - head attention, the results of multiple heads are concatenated and projected into the final output space:

[0035] where, is the result of MHA; Layer normalization

[0036]

[0037] where, are the scale and translation parameters, is the element - wise product; Feed - forward layer

[0038] Layer normalization

[0039] During the training process, it also includes a loss function and constraints; where, the loss function includes: This loss aims to learn a balanced representation that has predictive power for future outcomes but is unpredictable for the current treatment assignment, thereby weakening the influence of confounding factors.

[0040] Through an adversarial learning method, ensure that the model simultaneously optimizes the predictive power and the robustness of causal inference.

[0041]

[0042] where, : Domain classifier, predicting the treatment assignment. : Representation extracted by the main network. : Parameters of the main network (Representation Network). : Parameters of the domain classifier. : Number of treatment assignment categories (dimension of the categorical space). : Log of the treatment assignment probability predicted by the domain classifier.

[0043] Counterfactual balance and commonsense constraints Counterfactual balance: For the real data of each patient, only a single factual result of taking or not taking anticoagulants is observed. In counterfactual prediction, it is necessary to reasonably model the unobserved treatment assignment to prevent the model from making unreasonable predictions about counterfactual results. To this end, a balance constraint on factual and counterfactual results is proposed to ensure that the prediction results are clinically interpretable on a logical basis.

[0044] Taking TE as an example: Batch loss calculation formula reversed_loss

[0045] Where: M is the set of valid samples (determined by valid_mask). is the treatment assignment of the i-th sample (T = 0 means untreated, T = 1 means treated). is the model's prediction of the counterfactual result.

[0046]

[0047] reversed_loss emphasizes separate modeling of facts and counterfacts; treat_loss ensures logical balance between factual and counterfactual predictions.

[0048] Common sense constraint: Constraints on the probabilities of major bleeding and embolism based on medical common sense: Embolism: The probability when a patient takes anticoagulants should not be higher than the probability when not taking anticoagulants.

[0049] Major bleeding: The probability when a patient does not take anticoagulants should not be higher than the probability when taking anticoagulants.

[0050] By embedding these common sense rules into model training, the prediction range of the model is restricted, thereby improving the credibility of the prediction results.

[0051] Factual event preference: For the observed factual events (such as major bleeding occurred when taking anticoagulants), the model preferably tends to predict that the event does not occur in the counterfactual scenario (such as no major bleeding when not taking anticoagulants) to enhance the rationality and stability of the counterfactual results.

[0052] By introducing innovative methods of counterfactual balance and common sense constraints, the causal inference ability of the model can be effectively improved, ensuring that it is more in line with the logic and actual needs of the medical field when predicting counterfactual results.

[0053] Step S4: Use the trained prediction model to predict the probabilities of embolism and major bleeding for the target object within a preset future time period.

[0054] Step S5: Calculate the Net Clinical Benefit (NCB) of an individual according to the formula: NCB = (embolism probability 不服抗凝 - embolism probability 服抗凝 ) - 1.5×(major bleeding probability 服抗凝 - major bleeding probability 不服抗凝 ). Here, anticoagulants are recommended when NCB > 0, and anticoagulants are not recommended when NCB ≤ 0.

[0055] The present invention also provides an embolism and major bleeding risk prediction system based on a causal Transformer model. The embolism and major bleeding risk prediction system based on the causal Transformer model can be implemented by executing the process steps of the embolism and major bleeding risk prediction method based on the causal Transformer model. That is, those skilled in the art can understand the embolism and major bleeding risk prediction method based on the causal Transformer model as a preferred implementation manner of the embolism and major bleeding risk prediction system based on the causal Transformer model.

[0056] Those skilled in the art know that in addition to implementing the systems, devices, and their respective modules provided by the present invention in the form of pure computer-readable program codes, the method steps can be logically programmed to enable the systems, devices, and their respective modules provided by the present invention to be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers, etc., to achieve the same program. Therefore, the systems, devices, and their respective modules provided by the present invention can be regarded as a kind of hardware component, and the modules included therein for implementing various programs can also be regarded as the structures within the hardware component; the modules for implementing various functions can also be regarded as either software programs for implementing the method or the structures within the hardware component.

[0057] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. Without conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A method for predicting the risk of embolism and massive bleeding based on a causal Transformer model, characterized in that: include: Step S1: Acquire clinical data of a patient suffering from atrial fibrillation, and preprocess the acquired clinical data of the patient to obtain preprocessed clinical data of the patient; Step S2: standardize the preprocessed patient clinical data of different time series lengths, decompose the standardized patient clinical data into dynamic variables and static variables, and generate multidimensional features; Step S3: construct a prediction model based on the Causal Transformer model, and train the prediction model using multi-dimensional features to obtain a trained prediction model; Step S4: using the trained prediction model to predict the probability of embolism and massive bleeding in the target subject within a preset time period in the future.

2. The method for predicting the risk of embolism and massive bleeding based on the causal Transformer model according to claim 1, characterized in that: The step S1 includes: obtaining clinical data of patients with atrial fibrillation, screening the clinical data of the patients, deleting the clinical data of patients with a missing ratio greater than a preset value, retaining the clinical data of patients with a missing ratio less than or equal to the preset value, and filling the missing values ​​to obtain the pre-processed clinical data of the patients.

3. The method for predicting the risk of embolism and massive bleeding based on a causal Transformer model according to claim 1, characterized in that: The step S2 includes: aligning the sequence length and filling missing values ​​of the pre-processed patient clinical data of different time series lengths, and decomposing the processed patient clinical data into dynamic variables and static variables.

4. The method for predicting the risk of embolism and massive bleeding based on a causal Transformer model according to claim 1, characterized in that: The prediction model constructed based on the Causal Transformer model includes: a multi-head attention module, a feedforward layer and a normalization layer; The multi-head attention module includes a multi-head self-attention layer and a cross-attention layer; The input sequence is normalized by using the normalization layer to obtain a normalized sequence; the normalized sequence is input into the multi-head self-attention layer to capture the relationship between different positions in the input sequence; The normalization layer is used to normalize the dynamic features and the static features to obtain the normalized dynamic features and the static features; the normalized dynamic features and the static features are input into the cross attention layer to capture the interactive relationship between the dynamic features and the static features; Normalizing the relationship between different positions in the captured sequence and the interactive relationship between the captured dynamic features and static features to obtain the normalized relationship between different positions in the sequence and the interactive relationship between the captured dynamic features and static features; Based on the relationship between different positions within the normalized sequence and the interactive relationship between dynamic features and static features, deep features are extracted through the fully connected network in the feedforward layer; the deep features are normalized to obtain normalized deep features.

5. The method for predicting the risk of embolism and massive bleeding based on a causal Transformer model according to claim 1, characterized in that: The step S3 also includes: loss function and constraint conditions during training; The loss function includes: in, represents the domain classifier; represents the representation extracted by the main network; Represents the parameters of the main network; represents the parameters of the domain classifier; represents the number of treatment assignment categories; represents the logarithm of the treatment assignment probability predicted by the domain classifier; The constraints include: counterfactual balance constraints and common sense constraints; The counterfactual balance constraints include: constraints on the output of the Causal Transformer model; Among them, M is the valid sample set; is the treatment assignment of the ith sample, where T=0 means untreated and T=1 means treated; is the model's prediction of the counterfactual outcome; The common sense constraints include: Constraints on the probability of massive bleeding and embolism based on medical common sense: Embolism: The probability in the same patient when taking anticoagulants should be no higher than when not taking anticoagulants; Major bleeding: The probability for the same patient not taking anticoagulants should be no higher than the probability for the same patient taking anticoagulants.

6. The method for predicting the risk of embolism and massive bleeding based on a causal Transformer model according to claim 1, characterized in that: The method further comprises: calculating the individual's net clinical benefit NCB; It is recommended to take anticoagulants when NCB>0, and it is not recommended to take anticoagulants when NCB≤0.

7. An embolism and massive bleeding risk prediction system based on a causal Transformer model, characterized in that: include: Module M1: acquiring clinical data of a patient with atrial fibrillation, and preprocessing the acquired clinical data of the patient to obtain the preprocessed clinical data of the patient; Module M2: Standardize the preprocessed clinical data of patients with different time series lengths, decompose the standardized clinical data of patients into dynamic variables and static variables, and generate multidimensional features; Module M3: Build a prediction model based on the Causal Transformer model, and use multi-dimensional features to train the prediction model to obtain the trained prediction model; Module M4: Use the trained prediction model to predict the probability of embolism and massive bleeding in the target subject within a preset time period in the future.

8. The embolism and massive bleeding risk prediction system based on the causal Transformer model according to claim 7, characterized in that: The module M1 includes: obtaining clinical data of patients with atrial fibrillation, screening the clinical data of the patients, deleting the clinical data of patients with a missing ratio greater than a preset value, retaining the clinical data of patients with a missing ratio less than or equal to a preset value, and filling the missing values ​​to obtain the pre-processed clinical data of the patients; The module M2 includes: aligning the sequence length and filling the missing value of the pre-processed patient clinical data of different time series lengths, and decomposing the processed patient clinical data into dynamic variables and static variables.

9. The embolism and massive bleeding risk prediction system based on the causal Transformer model according to claim 7, characterized in that: The prediction model constructed based on the Causal Transformer model includes: a multi-head attention module, a feedforward layer and a normalization layer; The multi-head attention module includes a multi-head self-attention layer and a cross-attention layer; The input sequence is normalized by using the normalization layer to obtain a normalized sequence; the normalized sequence is input into the multi-head self-attention layer to capture the relationship between different positions in the input sequence; The normalization layer is used to normalize the dynamic features and the static features to obtain the normalized dynamic features and the static features; the normalized dynamic features and the static features are input into the cross attention layer to capture the interactive relationship between the dynamic features and the static features; Normalizing the relationship between different positions in the captured sequence and the interactive relationship between the captured dynamic features and static features to obtain the normalized relationship between different positions in the sequence and the interactive relationship between the captured dynamic features and static features; Based on the relationship between different positions within the normalized sequence and the interactive relationship between dynamic features and static features, deep features are extracted through the fully connected network in the feedforward layer; the deep features are normalized to obtain normalized deep features.

10. The embolism and massive bleeding risk prediction system based on the causal Transformer model according to claim 7, characterized in that: The module M3 also includes: loss functions and constraints during training; The loss function includes: in, represents the domain classifier; represents the representation extracted by the main network; Represents the parameters of the main network; represents the parameters of the domain classifier; represents the number of treatment assignment categories; represents the logarithm of the treatment assignment probability predicted by the domain classifier; The constraints include: counterfactual balance constraints and common sense constraints; The counterfactual balance constraints include: constraints on the output of the Causal Transformer model; Among them, M is the valid sample set; is the treatment assignment of the ith sample, where T=0 means untreated and T=1 means treated; is the model's prediction of the counterfactual outcome; The common sense constraints include: Constraints on the probability of massive bleeding and embolism based on medical common sense: Embolism: The probability in the same patient when taking anticoagulants should be no higher than when not taking anticoagulants; Major bleeding: The probability for the same patient not taking anticoagulants should be no higher than the probability for the same patient taking anticoagulants.

Images