Prediction method for sepsis in early stage based on time convolution attention

Abstract

The invention discloses a sepsis early prediction method based on time convolution attention. Comprising the following steps: processing missing values in electronic medical record data of a patient by using a multi-task Gaussian process method; complex electronic medical record data are converted into time series data which are easy to model and analyze, and attention to the physiological index state of a patient in each time period is enhanced; inputting the time sequence data of the patient into a time convolutional neural network for feature state learning; performing weighting processing on the important physiological feature parameters of the patient through a double-channel attention mechanism; the weighted characteristic parameters are further processed through a feedforward neural network, and the sepsis early prediction problem is converted into a dichotomy problem. The method effectively solves the problems that in the process of predicting the sepsis risk based on the electronic medical record data, modeling of the electronic medical record data is difficult, the prediction effect is poor, and the model is not easy to migrate. The method has high adaptability and stability in predicting the problem of sepsis infection of a patient.

Description

Technical Field

[0001] The patent of this invention relates to an early prediction method of sepsis based on deep learning. It has important application and promotion value in predicting patients' early infection with sepsis. Background Art

[0002] Sepsis diagnosis is a major problem in the world's medical field, claiming more than 6 million lives each year, with a mortality rate of 20-40%. The World Health Organization (WHO) has issued a special call to governments to "give priority to strengthening the prevention, diagnosis and treatment of sepsis". Accurately predicting the risk of patients contracting sepsis is the main means to reduce the mortality rate of patients. Existing early prediction technologies for sepsis mainly include prediction methods based on traditional machine learning and simple clinical scoring prediction methods. Traditional machine learning prediction methods require manual extraction and selection of patient features, and important features require repeated experiments to obtain, resulting in a very time-consuming feature selection process and the inability to accurately predict the incidence of sepsis patients. Simple clinical scoring prediction methods require scoring by a team of experts with professional knowledge from both inside and outside the project. The expert team members analyze and judge the information obtained based on their professional knowledge and experience. However, since the symptoms of sepsis are similar to those of many other diseases, predicting the incidence of sepsis in patients through simple scoring methods will lead to large differences in prediction results, which may cause patients to miss the best treatment time.

[0003] Deep learning is a relatively advanced technology in the field of artificial intelligence. It has been applied to natural language processing, computer vision and other fields, and has achieved good prediction results. By establishing an early prediction method for sepsis through deep learning technology, we can better focus on the patient's physiological status indicators and reduce the mortality rate of patients infected with sepsis. Summary of the invention

[0004] In the method of the present invention, the patient set of the intensive care unit ICU is H = {h1, h2, ..., h n}, h t = {Z t |temp k ,heartrate k ,spo2 k ,…}, eigenvector Z t represents the ICU monitoring information such as "body temperature", "heart rate", and "blood oxygen" of ICU patient h at time point t. Suppose the time interval during which the patient is monitored during ICU hospitalization is T, T = {t1, t2, ..., t j}, represents the multivariate time series feature vector Z of patient c at all time points during hospitalization. Then the sepsis prediction problem can be formally described as: given the multivariate time series feature vector Z of ICU patient h during hospitalization T, the decision goal is to find the probability p of patient h being infected with sepsis through the sepsis prediction model h =model(h,T,Z).

[0005] The present invention mainly includes four parts: (1) for the multivariate time series feature vector Z, a multi-task Gaussian process is used to preprocess the data. This method can handle irregular sampling frequencies and obtain the potential time series Z i (2) The potential time series Z i Input into the time convolutional neural network (TCN) to learn the patient’s feature vector Z at time t t (3) The learned patient features are weighted by the attention mechanism, and different weights are assigned to the features, so that important feature information can be selected while also having good interpretability. (4) The training set and test set are constructed, the weighted features are fused, and transmitted to the feedforward neural network and the softmax function is used for prediction output, making the prediction of patient sepsis infection a binary classification problem.

[0006] The following is a detailed description of the specific implementation process of the four parts:

[0007] (1) For the multivariate time series feature vector Z, a multi-task Gaussian process is used to preprocess the data, which can handle irregular sampling frequencies. It can be formally described as: given the observation value {y i ,t i}(value and time), for evenly spaced query times x i , the multi-task Gaussian process is based on the posterior distribution P(z i |y i ,t i ,x i ; θ) draws a potential time series Z i ;

[0008] (2) The potential time series Z i Input into the time convolutional neural network (TCN) to learn the patient’s feature vector Z at time t t , based on the one-dimensional convolution, the temporal convolutional neural network (TCN) adds causal dilated convolution, which ensures that the output of the model at time t only depends on the information at time t and before, and does not depend on the information after time t, thus preventing future information leakage;

[0009] (3) The learned patient features are weighted by the attention mechanism, including feature channel attention and time channel attention, and different weights are assigned to the features, so that important feature data information can be selected while also having good interpretability;

[0010] (4) After preprocessing and feature extraction of the patient's physiological time series data, the training set and test set were constructed in a ratio of 8:2. The weighted features were fused and transmitted to the feedforward neural network and the output was predicted through the softmax function, so that the problem of predicting early sepsis infection in patients was converted into a binary classification problem.

[0011] The detailed steps of the early prediction method of sepsis based on temporal convolutional attention are as follows:

[0012] Step 1: Extract and preprocess the data through the following steps 1.1, 1.2, and 1.3 respectively to obtain the processed multivariate time series data Z i .

[0013] Step 1.1: Data extraction. The patient's physiological indicators monitored and recorded in the hospital are stored in the electronic medical record database at regular intervals. The data is extracted using the structured query language SQL method based on the modeling requirements. For missing values ​​that are not recorded due to equipment failure or other reasons, a multi-task Gaussian process is used to process the missing values, and then proceed to step 1.2.

[0014] Step 1.2: Data filling: For multivariate time series data Z, a multi-task Gaussian process extracts data Z from the given monitoring data through the posterior distribution at uniformly spaced grid times (usually every hour). i More precisely, given the irregular sample i's monitoring value {y i ,t i}(value and time), for evenly spaced query times x i , the multi-task Gaussian process method is based on the posterior distribution P(z i |y i ,t i ,x i ; θ) draws a potential time series Z i , Z i Obey Z i ~N(μ(Z i ),Σ(Z i );θ) distribution, where μ(Z i ) represents the mean, Σ(Z i ) represents the covariance, i Refers to the number of evenly spaced queries x i With the number of observations ti The correlation matrix between Represents x i The correlation between itself and K D is the task similarity kernel matrix, represents the similarity at position (d, d'), represents the tensor product, K Ti represents the specific T between all observation times encountered by patient i i ×T i The correlation matrix, D, satisfies The diagonal matrix of the noise variance, i is the identity matrix. The posterior mean μ(Z i ) depends on the observed value y i .exist The parameters of the multi-task Gaussian process are collected in , where l represents the length scale of the kernel function. After completing the missing data filling, the processed data Z is obtained i , then proceed to step 1.3.

[0015] Step 1.3: Data normalization: After the original data is normalized, each indicator is at the same order of magnitude, which is suitable for comprehensive comparative evaluation. i ,pass Get the normalized data, where x 原始 represents the original data of an indicator, min represents the smallest value of the indicator, and max represents the largest value of the indicator. Finally, the normalized data sequence x = x0, x1, ..., x t Input into the model for feature learning, and then proceed to step 2.

[0016] Step 2: The patient's physiological time series data x obtained in step 1 is x = x0, x1, ..., x t Input into the temporal convolutional neural network (TCN) model for feature learning.

[0017] Step 2.1: For a one-dimensional input sequence x and a convolution kernel f:{0,1,…,k-1}, the dilated convolution operation can be expressed as Where d is the dilation factor; k is the convolution kernel size; x s-d·i Represents the input sequence data, the sequence data to be learned is x=x0,x1,…,x t Input to the causal dilated convolution to learn the feature F tr , then proceed to step 2.2.

[0018] Step 2.2: In order to ensure that the TCN has a deeper network structure, the residual block is introduced into the TCN. A standard residual block contains a branch that generates a series of transformations. The output of the transformation F is added to the input x of the module. The output of the residual block is F. tr =Activation(x+F(x)), the learned feature F is transformed into tr Make corrections and go to step 3.

[0019] Step 3: The patient features F learned in step 2 tr The important life features and time features in the image are weighted by the dual-channel attention mechanism. The specific steps are as described in 3.1 and 3.2.

[0020] Step 3.1: Use feature channel attention to weight the important prediction features. First, use 1×1 convolution on the original convolution result U to get the feature channel attention m, m = V 1×1 *U, where is the learned feature attention; V 1×1 Is a 1×1 convolution kernel; It is a transformation in the temporal convolutional neural network (TCN); "*" indicates the convolution operation. Then, the feature channel attention and the convolution result are multiplied as the result output The size of the feature channel attention matrix is ​​the same as the original convolution result. The feature channel attention matrix is ​​multiplied by the results of different channels and output as the result. u c is the result of the convolution result U under the cth channel. The output result after calculating the weighted attention of the feature channel is Then proceed to step 3.2.

[0021] Step 3.2: Use time channel attention to perform weighted allocation of important disease onset time intervals. First, construct the Squeeze module of the temporal convolutional neural network to extract global information on the time "channel"; then, construct the Excitation module of the temporal convolutional neural network, perform calculations based on the global information provided by the Squeeze module, and reallocate the attention mechanism to the corresponding time interval. The temporal descriptor is implemented by taking a global average pooling operation to generate time-related statistics. The statistic z is implemented by compressing U on the dimension N×C, and the tth element of z is obtained by the formula Calculated, where z t is the output of the Squeeze module, v t is the convolution result at time t.

[0022] The function of the Excitation module of the temporal convolutional neural network is to use the Squeeze module to obtain global information, calculate the time channel attention, and give different "attention" to different time intervals. In order to apply the gating mechanism to the Excitation module, the formal calculation of this operation is s = F ex (z,W)=σ(g(z,W))=σ(w2σ(W1z)). Here, σ refers to the ReLU activation function and s is the time channel attention. The final output is It is to multiply the time channel attention and the convolution result at time t. s t is the temporal channel attention allocated to time t, v t It is the result of convolution of data at time t. Then proceed to step 4.

[0023] Step 4: After assigning different weights to features through the attention mechanism in step 3, the feature fusion is transmitted to the feedforward neural network and output through the softmax activation function to convert the sepsis prediction problem into a binary classification problem.

[0024] Step 4.1: Calculate the feature channel attention m and time channel attention s in step 3 t Multiply them with the convolution results respectively to get the output results under different attention and Fusion by weighted features And sent to the feedforward neural network, and then go to step 4.2.

[0025] Step 4.2: Through step 4.1, the fusion feature F t After being fed to the feedforward neural network, the learning rate of the model training is set to 0.0005, the batch_size of the number of samples passed to the model for training is 128, the model training iteration cycle epoch is 100, the activation function uses the sotfmax function, and the loss function between the predicted probability and the true distribution of the classification task uses the cross entropy loss function. The convolution kernel size in the temporal convolutional neural network (TCN) is set to 6, with a total of 12 residual blocks, and the expansion coefficient of the dilated convolution is d = 2 n , doubled every other residual block, the initial value of n is 0, and the initial value of the Dropout ratio in the residual module is set to 0.05. Then go to step 4.3.

[0026] Step 4.3: Input the patient's physiological monitoring data into the temporal convolutional neural network (TCN) to predict the risk of sepsis infection, so as to obtain the probability of sepsis infection in the patient, provide clinical decision-making assistance to doctors, and take corresponding preventive and treatment measures to reduce the mortality rate of patients.

[0027] The present invention has the following advantages over the prior art:

[0028] The present invention proposes an early prediction method for sepsis based on temporal convolutional attention. The early prediction method for sepsis has the following characteristics: First, the method does not rely on manual data collection. Hospital physiological indicator monitoring instruments are widely used in clinical practice and can automatically collect data for model input; second, a multi-task Gaussian process method that is more suitable for the medical field is used to supplement the missing values ​​in medical data; third, the method uses deep learning technology to model the data, which is more efficient and accurate than traditional sepsis prediction technology; fourth, the method combines a dual-channel attention mechanism to better focus on the patient's important physiological characteristics, and plays a greater auxiliary diagnostic role in predicting patients infected with sepsis. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is an example of a data segment used in the present invention.

[0030] Figure 2 This is a process diagram of using the multi-task Gaussian process method to handle data missing in the present invention

[0031] Figure 3 This is a flow chart of the early prediction model of sepsis based on temporal convolutional attention of the present invention

[0032] Figure 4 This is a comparative experimental diagram of the AUROC (unit: %) of the present invention and the existing sepsis prediction technology

[0033] Figure 5 This is a comparative experimental diagram of the AUPRC (unit: %) of the present invention and the existing sepsis prediction technology

[0034] Figure 6 This is a comparison chart of clinical decision curves between the present invention and existing sepsis prediction technology.

[0035] Figure 7 This is a clinical impact curve analysis diagram of the sepsis prediction technology of the present invention DETAILED DESCRIPTION

[0036] The present invention is further described below in conjunction with the accompanying drawings.

[0037] The data used in the experiment of this invention is an internationally open source electronic medical record dataset. Figure 1 As shown in Figure 1, currently all hospitals at or above the county level in my country can obtain electronic medical record data through testing instruments. Monitoring instruments can collect the patient's physiological index data at each set time point. For data that is not collected due to mechanical failure (missing values), the multi-task Gaussian process method is used to calculate and fill in the missing values.

[0038] The calculation process of the multi-task Gaussian process method in the present invention is as follows: Figure 2 As shown, the multi-task Gaussian process method fits the irregular data with a function, and then draws a potential time series based on the posterior probability distribution, and obtains the filled data through step 1.2.

[0039] The flowchart of the early prediction method of sepsis based on temporal convolutional attention of the present invention is as follows: Figure 3 As shown, we first obtain the patient's multivariate physiological characteristics Z, and then fill in the missing data through the multi-task Gaussian process in step 1.2 to obtain Z i , Z i The feature state is input into the temporal convolutional neural network (TCN) to learn the feature state; then different weights are assigned to the extracted features according to the feature channel attention and the temporal channel attention; finally, the probability of the patient being infected with sepsis is obtained through the softmax activation function.

[0040] Figure 4 , 5 6 are comparative experimental diagrams of the present invention and other similar models under different evaluation indicators. The larger the area of ​​AUROC and AUPRC, the better the prediction effect of the model. Figure 4 and Figure 5 It can be seen that the method proposed in the present invention is very close to the onset time of sepsis when predicting the risk of sepsis infection in patients, and the evaluation index shows a gradually increasing state, indicating that the prediction effect of this method is good.

[0041] Figure 7 This is the clinical impact curve analysis of the sepsis prediction method proposed in the present invention. Figure 7 When the threshold is greater than 50%, the high-risk population for sepsis predicted by the method proposed in the present invention is highly matched with the population actually suffering from sepsis, indicating that the method proposed in the present invention has a high clinical application value.

Claims

1. An early prediction method for sepsis based on temporal convolutional attention, characterized by: The deep learning method is used to predict the probability of early onset of sepsis; the multi-task Gaussian process is used to preprocess the data; the problem of predicting early sepsis infection in patients is converted into a binary classification problem; the early prediction of sepsis is achieved mainly through the following steps: Step 1: Based on the patient monitoring time series data obtained by the hospital's physiological indicator monitoring instrument, the structured query language SQL method is used to extract the data. For the problem of missing patient monitoring data due to monitoring equipment failure and other reasons, the multi-task Gaussian process method is used to process the missing values, and the normalization method is used to control the missing value range between (0, 1); Step 2: Input the patient monitoring time series data into the temporal convolutional network of the deep learning method for feature state learning, and correct the learned features through the residual block to obtain the patient's physiological state characteristic factors at each moment; Step 3: Since some of the patient physiological status feature predictors obtained in the temporal convolutional network are not conducive to sepsis prediction, different weights are assigned to the feature predictors by combining feature channel attention and temporal channel attention, thereby obtaining important feature predictors to facilitate the prediction of the probability of early onset of sepsis; Step 4: The patient's physiological status characteristic prediction factors are fused according to the weights and transmitted to the feedforward neural network, which converts the early prediction problem of sepsis into a binary classification problem. The patient's physiological monitoring data is input into the temporal convolutional attention network to predict the risk probability of sepsis infection, provide doctors with clinical decision-making assistance suggestions, and take appropriate early diagnosis and treatment measures to reduce the patient's mortality rate.

Images