Causal and anti-fact sepsis auxiliary decision-making method of Transform

Abstract

The invention relates to a causal anti-fact sepsis auxiliary decision-making method of Transform, and belongs to the field of medical big data. The method is realized by a causal anti-fact sepsis auxiliary decision-making model based on Transform. The causal anti-fact sepsis auxiliary decision-making model based on the Transform is formed by connecting an action generation network with a causal Transform network in series. The method comprises the following steps: S1, preprocessing input data; s2, establishing a causal anti-fact sepsis auxiliary decision-making model based on a Transform; s3, predicting the state of the patient at the next moment by using a multi-layer decoder; s4, using the action generation network to predict an intervention action at the next moment; s5, updating the state of a multi-layer decoder of the causal Transform network; and S6, predicting the risk of sepsis of the patient, and carrying out interpretable analysis. According to the method, accurate selection and recommendation of intervention actions can be realized, accurate feature expression is predicted and generated, judgment of an intervention effect is assisted, and interpretable analysis is realized.

Description

Technical Field

[0001] The present invention relates to a causal counterfactual sepsis decision-making assistance method based on a Transformer, belongs to the field of medical big data, and is particularly suitable for causal counterfactual sepsis decision-making assistance based on a Transformer. Background Art

[0002] Sepsis is a life-threatening organ dysfunction caused by a dysregulated host response to infection: due to the uncertainty of pathogenesis, the diversity of symptoms, and the complexity of treatment, accurate prediction and decision support have always been difficult, one of the most challenging issues in the field of disease prediction, and one of the global health priorities. There are multiple interventions for sepsis: intravenous (IV) fluids (adjusted according to fluid tension), vasopressors (VP), and anticoagulants (heparin), which are used to correct hypovolemia and counteract vasodilation caused by sepsis, respectively. Although hypovolemia and vasodilation are common in patients with sepsis, there is little clinical consensus on when and how to treat these issues. With the delay of antimicrobial intervention, the mortality rate caused by sepsis will gradually increase, further emphasizing the importance of timely identification and treatment of sepsis. Therefore, it is of great significance to predict the risk of postoperative sepsis and provide doctors with timely auxiliary information.

[0003] Causal reasoning can demonstrate the causal relationship between critical illness variables, and the introduction of causal reasoning can achieve strategy optimization. Under the premise that the causal structure is known, different intervention strategies can be adopted to intuitively observe changes in patient status, helping the intelligent agent to explore strategies more efficiently. Through proper planning of medical intervention measures, the changes in joint distribution caused by the intervention can be observed, and the intelligent agent can infer the effects of different actions, thereby effectively guiding strategy optimization. Therefore, under the partially observable patient diagnosis and treatment conditions, the design of causal decision neural networks and the construction of an intelligent intervention decision model for sepsis-assisted treatment can guide doctors to carry out personalized sepsis treatment interventions and help doctors carry out explainable patient treatment interventions.

[0004] Similarly, there is a very close relationship between counterfactual reasoning and causal reasoning; counterfactual reasoning is the premise that there is no historical record of the intervention strategy. If this intervention strategy is adopted, then the observed change in patient status will be inferred. Through hypothetical strategies, the intelligent agent can perform counterfactual interventions in the imaginary space and optimize medical decision-making strategies; it can generate new patient diagnosis and treatment data in the imaginary space, and can more effectively use information (observational data, counterfactual data) for reasoning, thereby improving the data utilization of the algorithm. Traditional causal effect evaluation estimates the average intervention effect from the overall level of the group, which cannot well express the different responses of different individuals to the intervention.

[0005] With the development of deep learning, it has become possible to use Transformer to represent historical patient monitoring data in partially observable Markov decision processes (POMDP), but how to better utilize it in deep reinforcement learning to complete causal reasoning and counterfactual inference of sepsis remains an open problem. Summary of the invention

[0006] In view of this, the present invention provides a causal counterfactual sepsis decision-making support method based on Transformer, which aims to construct a sepsis prediction decision-making support model integrating a causal variable selection network, an action generation network, and a Transformer network. The causal model is used to realize feature screening, and the Transformer is used to accurately express the counterfactual features and generate auxiliary decisions. The action generation network is used to generate predictions of treatment measures and patient status, so as to accurately warn patients of sepsis risks and provide explainable results.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] A transformer-based causal counterfactual sepsis decision-making assistance method, characterized in that it is implemented by a transformer-based causal counterfactual sepsis decision-making assistance model; the transformer-based causal counterfactual sepsis decision-making assistance model is composed of an action generation network connected in series with a causal transformer network; the causal transformer network is used to simulate an interactive environment, and is based on a transformer network architecture, and is composed of a multi-layer encoder and a multi-layer decoder; the front end of the multi-layer encoder input is connected in series with a causal variable selection network, and each encoder layer is composed of a two-sublayer connection structure: the first sublayer is a multi-head self-attention sublayer, the second sublayer is a feedforward fully connected sublayer, and each sublayer is connected to a normalization layer and a residual connection; each decoder layer in the multi-layer decoder is composed of a three-sublayer connection structure: the first sublayer is a self-attention sublayer, the second sublayer is a multi-head cross-attention sublayer, and the third sublayer is a feedforward fully connected layer, and each sublayer is connected to a normalization layer and a residual connection.

[0009] Furthermore, the action generation network is a DQN (Deep Q-learning Networks) network, which is composed of a main network (Q-network) and a target network (Target network) connected in series; the main network is a neural network for selecting actions and calculating Q values; the target network is a neural network for estimating target Q values.

[0010] Furthermore, the causal variable selection network is a neural network, which is used for 0-1 Bernoulli trial sampling of feature variables, and its sampling probability is determined by the network weight parameters.

[0011] Furthermore, considering the mutual influence on the time scale, the input of the multi-layer encoder is N historical measurement data from time t-N+1 to time t. After passing through the causal variable selection network, the corresponding ti-th input of each layer is the weighted average of the historical measurement data from time (t-N+1) to time (ti), which is achieved by connecting a linear layer in series at the input end, where t is the current time, 0≤N≤i≤t-1.

[0012] Similarly, further, the input of the multi-layer decoder is N prediction data from time t+1 to time t+N, and the corresponding t+i-th input of each layer is the weighted average of the prediction data from time (t+1) to time (t+i), which is achieved by connecting a linear layer in series at the input end, where t is the current time, 1≤i≤N≤t.

[0013] The specific steps of this method are as follows:

[0014] S1: static characteristics of the patient, time series data of the patient's vital signs monitored by the monitoring device, intervention actions, and patient status labels are used as historical measurement data;

[0015] S2: Establish a causal counterfactual sepsis decision-making support model based on Transformer;

[0016] S3: Use the historical measurement data as the input of the multi-layer encoder of the causal Transformer network, use its multi-layer decoder to predict the patient's state at the next moment, and calculate the corresponding reward;

[0017] S4: Based on the patient state and reward at the next moment obtained in step S3, the action generation network is used to predict the intervention action at the next moment;

[0018] S5: The predicted intervention action at the current moment and the patient status at the next moment are input as prediction data into the multi-layer decoder of the causal Transformer network;

[0019] S6: Repeat steps S3 to S5 m times, output the predicted intervention actions at m moments and the patient status at m moments in real time, classify the patient status using the status classifier, and use the parameters of the causal variable selection network to perform interpretability analysis on the current status.

[0020] At time t, the static feature is the feature x that does not change with time, and the time series data is the feature y that changes with time. t , the intervention action is the action a corresponding to the reinforcement learning network at the previous moment t-1 , the patient status is s t , the patient status label is z t , the historical data is manually labeled for binary classification; the predicted intervention action at the current moment and the patient status at the next moment are respectively denoted as a t and t+1 ; The causal variable selection network is denoted as h t =π V (x,y t , a t-1 , z t |θ V ), the multi-layer encoder is denoted as s t =π E (h t |θ E ), the multi-layer decoder is denoted as s t+1 =π D (s t , a t |θ D ), the state binary classifier is denoted as z t+1 =π Z (s t+1 |θ Z ), the main network is denoted as Q(s t , a t |θ Q ), the target network is denoted as Q(s t+1 , a t |θ′ Q ), where θ V ,θ E ,θ D ,θ Z ,θ Q and θ′ Q are the network parameters to be learned.

[0021] Furthermore, at the initial moment, the state of the multi-layer encoder output of the causal Transformer network is directly used as the input of the DQN network to predict the corresponding intervention action.

[0022] Furthermore, the reward calculation process described in step S3 is specifically as follows: using the binary classifier at time t corresponding to the patient label z t The predicted probability p(z t ) Evaluate the state and calculate the reward Among them, z t =0 means the patient is in poor condition, z t =1 means the patient is in good condition;

[0023] Furthermore, the training process of establishing the Transformer-based causal counterfactual sepsis decision-making support model is as follows:

[0024] (1) Using historical measurement data and manual annotation, according to the loss function L1 = ∑[z t -π Z (π E (π V (x,y t , a t-1 , z t |θ V )|θ E )|θ Z )] 2 , the network parameters θ to be learned for the multi-layer encoder of the training state binary classifier, causal variable selection network, and causal Transformer network V ,θ E ,θ Z ;

[0025] (2) Using the state s output by the multi-layer encoder of the causal Transformer network t and intervention actions t , according to the loss function L2 = ∑[z t -π Z (π D (s t-1 , a t-1 |θ D )|θ Z )] 2 , update the state binary classifier to learn the network parameters θ Z , the network parameters θ to be learned for training the multi-layer decoder of the causal Transformer network D ;

[0026] (3) Using the state s output by the multi-layer encoder of the causal Transformer network t and intervention actions t and the state s output by the multi-layer decoder of the causal Transformer network t+1 , combined with the reward rt , train the network parameters θ to be learned for the DQN network Q and θ′ Q .

[0027] Preferably, the output of the multi-layer encoder of the causal Transformer network and the output of the multi-layer decoder of the causal Transformer network are not directly used as states, but are respectively concat-connected with the patient's static features x to form states.

[0028] Preferably, the network parameters to be learned and the amount of calculation can be reduced. The action generation network is a Markov decision process (MDP). t According to the sampling probability p(s t |a t ) Select the optimal intervention action a in the action space t .

[0029] Preferably, in the absence of historical records, the doctor's hypothetical intervention action a t , the multi-layer decoder of the causal Transformer network can be directly used to predict the patient's future state and realize counterfactual reasoning.

[0030] The beneficial effects of the present invention are as follows: the present invention provides a Transformer-based causal counterfactual sepsis decision-making support method, which constructs a Transformer-based causal counterfactual sepsis decision-making support model, uses an action generation network to accurately select and recommend intervention actions, and uses the Transformer network to perform environmental simulation to promote the interaction of reinforcement learning, predict and generate accurate feature expressions, and assist in the judgment of intervention effects; further, a causal variable selection network is integrated in the Transformer network to realize the interpretability analysis of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to make the purpose and technical solution of the present invention, the present invention provides the following drawings for explanation:

[0032] Figure 1 This is the causal Transformer network architecture diagram of Example 1 of the present invention:

[0033] Figure 2 This is a flow chart of Example 1 of the present invention;

[0034] Figure 3 This is a diagram of the DQN network architecture in Example 1 of the present invention;

[0035] Figure 4This is a diagram of the causal Transformer network architecture of Example 2 of the present invention;

[0036] Figure 5 Schematic diagram of the Markov decision process of Example 4 of the present invention. DETAILED DESCRIPTION

[0037] Example 1: A sepsis dataset was collected for surgical patients in a hospital in a certain city from 2014 to 2018. The number of patients in the dataset is 1843, including 405 sepsis-positive patients and 1438 sepsis-negative patients. The binary classification manual annotation has been completed in time series, that is, z t Positive is 0, negative is 1; 80% (1474 patients) are used for training set, and 20% (369 patients) are used for testing set. Patient data include preoperative examination and demographic data, i.e., static features x of patients, and specific feature attributes are shown in Table 1; as well as intraoperative monitoring data of patients, i.e., time series data y of vital signs t , the sampling interval is half an hour, and the specific characteristic attributes are shown in Table 2, where each attribute corresponds to 2 data; doctor intervention action a t , which corresponds to the heparin dose at time t.

[0038] Table 1 Static characteristic attributes of patients

[0039]

[0040] Table 2 Time series data attributes of patients' vital signs

[0041]

[0042]

[0043] For the above patients, the risk warning and interpretability of sepsis are carried out, and considering the interpretability and intuitiveness of the extracted features, the present invention provides a "Transformer causal counterfactual sepsis decision-making method", combined with Figure 1 , a causal counterfactual sepsis decision-making support model based on Transformer is implemented; the causal counterfactual sepsis decision-making support model based on Transformer is composed of a DQN network connected in series with a causal Transformer network. Specifically, combined with Figure 3The DQN network is composed of a main network (Q-network) and a target network (Targetnetwork) connected in series; the main network is a convolutional neural network with an Actor-Critic structure, which is used to select actions and calculate Q values, and its gradient strategy adopts a greedy strategy; the target network is a convolutional neural network with an Actor-Critic structure, which is used to estimate the target Q value; the DQN network loss function is L3=E[(r t -γQ(s t , a t |θ Q )-Q(s t+1 , a t |θ′ Q )) 2 ], where γ is a hyperparameter; the causal Transformer network is used to simulate the interactive environment, is based on the Transformer network architecture, and is composed of a multi-layer encoder and a multi-layer decoder; the front end of the input of the multi-layer encoder is connected in series with a causal variable selection network, and each encoder layer is composed of two sub-layer connection structures: the first sub-layer is a multi-head self-attention sub-layer (Muti-Head Self Attention), the second sub-layer is a feed-forward fully connected sub-layer (Feed-Forward), and each sub-layer is followed by a normalization layer and a residual connection (Add & Norm); each decoder layer in the multi-layer decoder is composed of three sub-layer connection structures: the first sub-layer is a self-attention sub-layer (SelfAttention), the second sub-layer is a multi-head cross attention sub-layer (Muti-Head Cross Attention), and the third sub-layer is a feed-forward fully connected layer (Feed-Forward), and each sub-layer is followed by a normalization layer and a residual connection (Add & Norm).

[0044] In this embodiment, both the multi-layer encoder and the multi-layer decoder adopt a 6-layer structure.

[0045] The causal variable selection network is a neural network, which is used for Bernoulli trial sampling of characteristic variables, and its sampling probability is determined by the network weight parameters.

[0046] The predicted intervention action at the current moment and the patient state at the next moment are respectively denoted as a t and t+1 ; The causal variable selection network is denoted as h t =π V (x,y t , a t-1 , z t |θ V), the multi-layer encoder is denoted as s t =π E (h t |θ E ), the multi-layer decoder is denoted as s t+1 =π D (s t , a t |θ D ), the state binary classifier is denoted as z t+1 =π Z (s t+1 |θ Z ), the main network is denoted as Q(s t , a t |θ Q ), the target network is denoted as Q(s t+1 , a t |θ′ Q ), where θ V ,θ E ,θ D ,θ Z ,θ Q and θ′ Q are the network parameters to be learned.

[0047] Combination Figure 2 , the specific steps are as follows:

[0048] Step 1: The static characteristics of each patient The monitoring device monitors the patient's vital signs to obtain time series data Intervention Action and patient status labels As historical measurement data, there are 32 dimensions in total.

[0049] Step 2: Establish a causal counterfactual sepsis decision support model based on Transformer.

[0050] The causal variable selection network only performs 30 independent Bernoulli trial samplings on static features and time series data. The sampling probability of each feature is different, and the adopted results are compared with the intervention action a. t-1 , and patient status label z t Use concat to connect.

[0051] The specific training process is as follows:

[0052] (1) Using historical measurement data and manual annotation, according to the loss function L1 = ∑[z t -π Z (π E (πV (x,y t , a t-1 , z t |θ V )|θ E )|θ Z )] 2 , the network parameters θ to be learned for the multi-layer encoder of the training state binary classifier, causal variable selection network, and causal Transformer network V ,θ E ,θ Z ;

[0053] (2) Using the state s output by the multi-layer encoder of the causal Transformer network t and intervention actions t , according to the loss function L2 = ∑[z t -π Z (π D (s t-1 , a t-1 |θ D )|θ Z )] 2 , update the state binary classifier to learn the network parameters θ Z , the network parameters θ to be learned for training the multi-layer decoder of the causal Transformer network D ;

[0054] (3) Using the state s output by the multi-layer encoder of the causal Transformer network t and intervention actions t and the state s output by the multi-layer decoder of the causal Transformer network t+1 , combined with the reward r t , train the network parameters θ to be learned for the DQN network Q and θ′ Q .

[0055] The output of the multi-layer encoder of the causal Transformer network and the output of the multi-layer decoder of the causal Transformer network are not directly used as states, but are respectively concat-connected with the patient's static features x to form states.

[0056] Step 3: Use the historical measurement data as the input of the multi-layer encoder of the causal Transformer network, use its multi-layer decoder to predict the patient's state at the next moment, and calculate the corresponding reward.

[0057] The reward calculation process is as follows: using the binary classifier at time t to correspond to the patient label z t The predicted probability p(zt ) Evaluate the state and calculate the reward Among them, z t =0 means the patient is in a bad condition, i.e. positive; t =1 means the patient is in good condition, i.e. negative.

[0058] Among them, the multi-layer encoder adopts a multi-head self-attention mechanism. Figure 1 K, V, and Q are the weight matrices of each layer. The weights of each layer are different and correspond to the h of the corresponding layer. t The new h is output. t .

[0059] Step 4: Based on the patient status and reward at the next moment obtained in step 3, use the DQN network to predict the intervention action at the next moment.

[0060] Step 5: The predicted intervention action at the current moment and the patient status at the next moment are input as prediction data into the multi-layer decoder of the causal Transformer network.

[0061] Among them, the multi-layer encoder adopts a combination of multi-head self-attention mechanism and multi-head cross-attention mechanism. The principle of the multi-head cross-attention mechanism is as follows Figure 1 , where K, V, and Q are weight matrices, and K and V are respectively related to the patient state s measured at time t t Multiply to ensure accuracy with actual data, Q and h of the corresponding layer t Multiply them together and then work together to get the new output h t .

[0062] Step 6. Considering the timeliness and accuracy of the prediction, this embodiment only predicts the patient's condition for 1 hour, repeats steps 3 to 5 twice, and outputs the predicted intervention actions at 2 moments and the patient's status at 2 moments in real time. The patient's status is classified using a status classifier, and the parameters of the causal variable selection network are used to perform interpretability analysis on the current status.

[0063] It can be directly foreseen that the method of the present invention can use the Transformer network to simulate the environment to promote the interaction of reinforcement learning, predict and generate feature expressions, and assist in the judgment of intervention effects. Since the Transformer network and the DQN network are relatively independent, they can play their common advantages, that is, they have high-precision calculations for complex environments and decision-making problems. It can be seen that the method of the present invention can also accurately express features. Combining the causal variable selection network can realize the function of interpretability.

[0064] Example 2: For the scenario of Example 1, considering the accuracy of the extracted features and adding the consideration of historical data, the present invention provides a "Transformer causal counterfactual sepsis decision-making method". Figure 4 , a causal counterfactual sepsis decision-making support model based on Transformer is implemented; the causal counterfactual sepsis decision-making support model based on Transformer is composed of a DQN network connected in series with a causal Transformer network. Specifically, the DQN network consists of a main network (Q-network) and a target network (Target network) in series; the main network is a neural network, which is used to select actions and calculate Q values; the target network is a neural network, which is used to estimate the target Q value; the causal Transformer network, which is used to simulate the interactive environment, is based on the Transformer network architecture and consists of a multi-layer encoder and a multi-layer decoder; the front end of the input of the multi-layer encoder is connected in series with a causal variable selection network, and each encoder layer consists of a two-sublayer connection structure: the first sublayer is a multi-head self-attention sublayer, the second sublayer is a feedforward fully connected sublayer, and each sublayer is connected to a normalization layer and a residual connection; each decoder layer in the multi-layer decoder consists of a three-sublayer connection structure: the first sublayer is a self-attention sublayer, the second sublayer is a multi-head cross-attention sublayer, and the third sublayer is a feedforward fully connected layer, and each sublayer is connected to a normalization layer and a residual connection.

[0065] Furthermore, the causal variable selection network is a neural network, which is used for 0-1 Bernoulli trial sampling of feature variables, and its sampling probability is determined by the network weight parameters.

[0066] Furthermore, considering the mutual influence on the time scale, the input of the multi-layer encoder is N historical measurement data from time t-N+1 to time t. After passing through the causal variable selection network, the corresponding ti-th input of each layer is the weighted average of the historical measurement data from time (t-N+1) to time (ti), which is achieved by connecting a linear layer in series at the input end, where t is the current time, 0≤N≤i≤t-1.

[0067] Similarly, further, the input of the multi-layer decoder is N prediction data from time t+1 to time t+N, and the corresponding t+i-th input of each layer is the weighted average of the prediction data from time (t+1) to time (t+i), which is achieved by connecting a linear layer in series at the input end, where t is the current time, 1≤i≤N≤t.

[0068] Among them, the processing between N-dimensional data adopts Concat connection, which can reduce the number of multi-layer encoders and multi-layer decoders and realize the prediction of patient status at N moments at a time.

[0069] The specific steps of this embodiment are almost the same as those of Embodiment 1 and will not be repeated here.

[0070] Example 3: Based on Example 1, the accuracy of the extracted features is taken into account, and the historical data is considered. The present invention provides a "Transformer-based causal counterfactual sepsis decision-making assistance method". The Transformer-based causal counterfactual sepsis decision-making assistance model and specific steps of Example 2 are almost the same as those of Example 1, and will not be repeated here.

[0071] The difference is that, considering the mutual influence on the time scale, the multi-layer encoder adopts the form of N multi-layer encoders connected in series, and the corresponding input is N historical measurement data from time t-N+1 to time t. After passing through the causal variable selection network, the corresponding ti-th input of each layer is the weighted average of the historical measurement data from time (t-N+1) to time (ti), which is achieved by connecting a linear layer in series at the input end, where t is the current time, 0≤N≤i≤t-1.

[0072] The multi-layer decoder is in the form of N multi-layer decoders connected in series, and the corresponding input is N prediction data from time t+1 to time t+N. The corresponding t+i-th input of each layer is the weighted average of the prediction data from time (t+1) to time (t+i), which is achieved by connecting a linear layer in series at the input end, where t is the current time, 1≤i≤N≤t.

[0073] Correspondingly, the input of the multi-head cross-attention sub-layer of the multi-layer decoder is adjusted to the weighted average of the output of the multi-layer encoder.

[0074] What is different in this method is that N multi-layer encoders and N multi-layer decoders lead to a corresponding increase in the corresponding parameters to be learned, and the L1 and L2 of the loss function need to calculate the N multi-layer encoders and N multi-layer decoders in a summation manner.

[0075] Example 4: For the scenario of Example 1, causal prediction and counterfactual reasoning are performed simultaneously, while considering reducing the network parameters to be learned and the amount of calculation, the present invention provides a "Transformer causal counterfactual sepsis decision-making support method".

[0076] The Transformer-based causal counterfactual sepsis decision-making support model is implemented; the Transformer-based causal counterfactual sepsis decision-making support model is composed of a Markov decision process network connected in series with a causal Transformer network. The Markov decision process contains five elements: state, action, strategy, reward and return, among which the return Combination Figure 5 , the Markov decision process for the current state s t According to the sampling probability p(s t |a t ) Select intervention action a in the action space t The rest is the same as the Transformer-based causal counterfactual sepsis decision support model in Example 1, and will not be described in detail here.

[0077] The following steps are involved:

[0078] Step 1: The static characteristics of each patient The monitoring device monitors the patient's vital signs to obtain time series data Intervention Action and patient status labels As historical measurement data, there are 32 dimensions in total.

[0079] Step 2: Establish a causal counterfactual sepsis decision support model based on Transformer.

[0080] Step 3: When the doctor assumes that he will take intervention actions t When performing counterfactual reasoning, the multi-layer decoder of the causal Transformer network is directly used to predict the patient's future state; when performing causal prediction, steps 4 to 7 are executed.

[0081] Step 4: Use the historical measurement data as the input of the multi-layer encoder of the causal Transformer network, use its multi-layer decoder to predict the patient's state at the next moment, and calculate the corresponding reward.

[0082] Step 5: Based on the patient status and reward at the next moment obtained in step 4, the Markov decision process is used to predict the intervention action at the next moment.

[0083] Step 6: Input the predicted intervention action at the current moment and the patient status at the next moment as prediction data into the multi-layer decoder of the causal Transformer network.

[0084] Step 7: Repeat steps 4 to 6 m times, output the predicted intervention actions at m moments and the patient status at m moments in real time, classify the patient status using the status classifier, and use the parameters of the causal variable selection network to perform interpretability analysis on the current status.

[0085] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made in form and details without departing from the scope defined by the claims of the present invention.

Claims

1. A Transformer-based causal counterfactual sepsis decision-making aid method, characterized in that: A transformer-based causal counterfactual sepsis decision-making support model is implemented; the transformer-based causal counterfactual sepsis decision-making support model is composed of an action generation network connected in series with a causal transformer network; the causal transformer network is used to simulate an interactive environment, and is based on a transformer network architecture, and is composed of a multi-layer encoder and a multi-layer decoder; the front end of the multi-layer encoder input is connected in series with a causal variable selection network, and each encoder layer is composed of a two-sublayer connection structure: the first sublayer is a multi-head self-attention sublayer, the second sublayer is a feed-forward fully connected sublayer, and each sublayer is connected to a normalization layer and a residual connection; each decoder layer in the multi-layer decoder is composed of a three-sublayer connection structure: the first sublayer is a self-attention sublayer, the second sublayer is a multi-head cross-attention sublayer, and the third sublayer is a feed-forward fully connected layer, and each sublayer is connected to a normalization layer and a residual connection; The action generation network is a DQN network, which is composed of a main network and a target network connected in series; the main network is a neural network for selecting actions and calculating Q values; the target network is a neural network for estimating target Q values; The causal variable selection network is a neural network, which is used for 0-1 Bernoulli trial sampling of feature variables, and its sampling probability is determined by the network weight parameter; The method comprises the following steps: S1: static characteristics of the patient, time series data of the patient's vital signs monitored by the monitoring device, intervention actions, and patient status labels are used as historical measurement data; S2: Establish a causal counterfactual sepsis decision-making support model based on Transformer; S3: Use the historical measurement data as the input of the multi-layer encoder of the causal Transformer network, use its multi-layer decoder to predict the patient's state at the next moment, and calculate the corresponding reward; S4: Based on the patient state and reward at the next moment obtained in step S3, the action generation network is used to predict the intervention action at the next moment; S5: The predicted intervention action at the current moment and the patient status at the next moment are input as prediction data into the multi-layer decoder of the causal Transformer network; S6: repeat steps S3 to S5 m times, output the predicted intervention actions at m moments and the patient status at m moments in real time, classify the patient status using the status classifier, and perform interpretability analysis on the current status using the parameters of the causal variable selection network; At time t, the static feature is the feature x that does not change with time, and the time series data is the feature y that changes with time. t , the intervention action is the action a corresponding to the reinforcement learning network at the previous moment t-1 , the patient status is s t , the patient status label is z t , the historical data is manually labeled for binary classification; the predicted intervention action at the current moment and the patient status at the next moment are respectively denoted as a t and t+1 ; The causal variable selection network is denoted as h t =π V (x,y t , a t-1 , z t |θ V ), the multi-layer encoder is denoted as s t =π E (h t |θ E ), the multi-layer decoder is denoted as s t+1 =π D (s t , a t |θ D ), the state binary classifier is denoted as z t+1 =π Z (s t+1 |θ Z ), the main network is denoted as Q(s t , a t |θ Q ), the target network is denoted as Q(s t+1 , a t |θ′ Q ), where θ V ,θ E ,θ D ,θ Z ,θ Q and θ′ Q are the network parameters to be learned.

2. A Transformer causal counterfactual sepsis decision-making aid method according to claim 1, characterized in that: Considering the mutual influence on the time scale, the input of the multi-layer encoder is N historical measurement data from time t-N+1 to time t. After the causal variable selection network, the corresponding ti-th input of each layer is the weighted average of the historical measurement data from time (t-N+1) to time (ti), which is achieved by connecting a linear layer in series at the input end, where t is the current time, 0≤N≤i≤t-1; Similarly, the input of the multi-layer decoder is N prediction data from time t+1 to time t+N, and the corresponding t+i-th input of each layer is the weighted average of the prediction data from time (t+1) to time (t+i), which is achieved by connecting a linear layer in series at the input end, where t is the current time, 1≤i≤N≤t.

3. A Transformer causal counterfactual sepsis decision-making aid method according to claim 1, characterized in that: At the initial moment, the state of the multi-layer encoder output of the causal Transformer network is directly used as the input of the DQN network to predict the corresponding intervention action.

4. A Transformer causal counterfactual sepsis decision-making aid method according to claim 1, characterized in that: The specific calculation process of the reward described in step S3 is: using the binary classifier at time t to correspond to the patient label z t The predicted probability p(z t ) Evaluate the state and calculate the reward Among them, z t =0 means the patient is in poor condition, z t =1 means the patient is in good condition.

5. The causal counterfactual sepsis decision-making aid method of Transformer according to claim 1, characterized in that: The training process of establishing the Transformer-based causal counterfactual sepsis decision-making support model is as follows: (1) Using historical measurement data and manual annotation, according to the loss function L1 = ∑[z t -π Z (π E (π V (x,y t , a t-1 , z t |θ V )|θ E )|θ Z )] 2 , the network parameters θ to be learned for the multi-layer encoder of the training state binary classifier, causal variable selection network, and causal Transformer network V ,θ E ,θ Z ; (2) Using the state s output by the multi-layer encoder of the causal Transformer network t and intervention actions t , according to the loss function L2 = ∑[z t -π Z (π D (s t-1 , a t-1 |θ D )|θ Z )] 2 , update the state binary classifier to learn the network parameters θ Z , the network parameters θ to be learned for training the multi-layer decoder of the causal Transformer network D ; (3) Using the state s output by the multi-layer encoder of the causal Transformer network t and intervention actions t and the state s output by the multi-layer decoder of the causal Transformer network t+1 , combined with the reward r t , the network parameters to be learned for training the DQN network θ Q and θ′ Q .

6. A Transformer causal counterfactual sepsis decision-making aid method according to claim 1, characterized in that: The output of the multi-layer encoder of the causal Transformer network and the output of the multi-layer decoder of the causal Transformer network are not directly used as states, but are respectively concat-connected with the patient's static features x to form states.

7. A Transformer causal counterfactual sepsis decision-making aid method according to claim 1, characterized in that: The network parameters and computational complexity to be learned can be reduced. The action generation network is a Markov decision process (MDP). t According to the sampling probability p(s t |a t ) Select the optimal intervention action a in the action space t .

8. The causal counterfactual sepsis decision-making aid method of Transformer according to claim 1, characterized in that: Without historical records, the doctor assumes that the intervention action a t , the multi-layer decoder of the causal Transformer network can be directly used to predict the patient's future state and realize counterfactual reasoning.

Images