Deep learning-based lower respiratory tract hospital infection risk prediction method and system

Abstract

The invention provides a lower respiratory tract hospital infection risk prediction method and system based on deep learning, and relates to the field of medical electronic medical record time sequence data analysis processing and deep learning. According to the method, two diagonal masking self-attention mechanism blocks are utilized to carry out missing value filling on electronic medical record medical time sequence data; processing the demographic data in the electronic medical record by using a full-connection layer neural network, and initializing the hidden state of a decoder; constructing an encoder-decoder model based on an attention mechanism, performing model training by using the filled time sequence data and the hidden state of the initialized decoder, and constructing a lower respiratory tract hospital infection risk prediction model; and inputting the processed data into the trained lower respiratory tract hospital infection risk prediction model, and outputting a lower respiratory tract hospital infection risk prediction result of the patient. According to the method, mass data of the electronic medical record can be effectively utilized, the static characteristics and the dynamic time sequence characteristics are fused, the health state of the patient is captured, and the prediction accuracy and effectiveness are improved.

Description

Technical Field

[0001] The present invention relates to the field of medical electronic medical record time series data analysis and processing and deep learning, and in particular to a method and system for predicting the risk of lower respiratory tract hospital infection based on deep learning. Background Art

[0002] Lower respiratory tract hospital infection is one of the most common types of hospital infection worldwide, with a high mortality rate. The management of such infections involves complex links and multiple factors, posing a threat to medical quality and patient safety, while increasing medical costs. The electronic medical record system records the entire medical process of the patient, providing a rich information basis for data-based disease risk prediction. It is particularly important to accurately predict the probability of hospitalized patients' lower respiratory tract hospital infection risk based on massive electronic medical record data, which not only helps to reduce the incidence of infection, but also improves the quality of medical services and patient safety.

[0003] Traditional methods for predicting the risk of lower respiratory tract hospital infection mostly rely on physical models and machine learning methods. Physical models use a series of mathematical equations and physical methods for analysis. However, due to the lack of data and the limitations of statistical methods, such studies cannot use the massive diagnosis and treatment data generated by the entire process of patient hospitalization to make real-time and accurate predictions of lower respiratory tract hospital infection. On the other hand, medical time series data in electronic medical record data are usually sparse and irregular. Existing machine learning methods usually deal with this problem by deleting missing values ​​or interpolating. However, existing interpolation methods are based on strong assumptions and may not truly reflect the actual health status of patients, resulting in damage to the integrity and authenticity of the data. In addition, existing machine learning models mainly focus on physiological status data and ignore static data such as demographic data such as age, gender, and medical history, which are equally important in predicting diseases and understanding health status. Failure to combine these static data with dynamic medical time series data limits the potential of the model in comprehensive analysis and formulation of personalized treatment strategies.

[0004] Therefore, there is an urgent need to provide a lower respiratory tract hospital risk prediction method based on electronic medical records to solve the technical problems raised above. Summary of the invention

[0005] The purpose of the present invention is to provide a method and system for predicting the risk of hospital infection of the lower respiratory tract based on deep learning, using two diagonal masked self-attention (DMSA) mechanism blocks to fill the missing values ​​of the medical time series data of the electronic medical record, and then using a fully connected layer neural network to process the demographic data in the electronic medical record to initialize the hidden state of the decoder. Then, an encoder-decoder model based on the attention mechanism is used to construct a lower respiratory hospital risk prediction model, which can effectively utilize the massive data of the electronic medical record, and by fusing static features with dynamic time series features, better capture the health status of the patient, and improve the accuracy and effectiveness of the prediction.

[0006] To achieve the above object, the present invention provides a method for predicting the risk of lower respiratory tract hospital infection based on deep learning, comprising the following steps: Step S1, using two diagonal masked self-attention mechanism blocks to fill missing values ​​in the electronic medical record medical time series data to obtain filled time series data; Step S2, using a fully connected layer neural network to process demographic data in the electronic medical record and initialize the hidden state of the decoder; Step S3, constructing an encoder-decoder model based on the attention mechanism, using the padded time series data and the initialized decoder hidden state to train the model, and constructing a lower respiratory tract hospital infection risk prediction model; Step S4: input the processed data into the trained lower respiratory tract hospital infection risk prediction model and output the prediction result.

[0007] Preferably, in step S1, two diagonal masked self-attention mechanism blocks are used to fill missing values ​​in the electronic medical record medical time series data to obtain filled time series data, including: Step S11: Set the original physiological state time series data Among them Step , introduce the missing mask vector To mark missing values: (1); in, Indicates that time steps; express dimensional multivariate time series; express Middle Time step Dimensional variables; Indicates that at time step At that time, A mask indicating whether the values ​​in the dimension are missing; The actual input physiological state time series data is , and its corresponding missing mask vector is : (2); in, Indicates that at time step At that time, A mask indicating whether the values ​​in the dimension are artificially randomly masked; express Middle Time step Variables of dimensions; The output is the predicted time series expressed as , introduce the mask vector Distinguish between artificially missing values ​​and original missing values: (3); in, Indicates that at time step At that time, Whether the value in the dimension is a mask of the original missing value; Step S12: Perform the masking task MIT of filling missing values ​​and the reconstruction task ORT of observation values, and calculate the interpolation loss and reconstruction loss through formulas (4)-(6): (4); (5); (6); in, Indicate how the loss is calculated; represents the predicted output of the model; represents the actual target sequence; Represents element-wise multiplication; represents the mask vector; represents the imputation loss of the model; Represents the reconstruction loss of the model; Step S13: Apply the diagonal masked self-attention mechanism module, and describe the calculation process of the diagonal masked self-attention mechanism through formulas (7)-(11): The diagonal masked self-attention mechanism module applies the diagonal mask inside formula (7). The diagonal mask and diagonal masked self-attention are shown in formula (8)-formula (9): (7); (8); (9); in, represents the output of the self-attention layer; Represents the calculation rule of diagonal mask; Represents the specific implementation process of the diagonal masked self-attention mechanism; represents the attention weight; represents the query vector; K represents the key vector; V represents a value vector; Represents feature dimension; represents normalization, converting similarity scores into attention weights; Indicates the first Line The value of the column; The diagonal masked self-attention mechanism is extended to the diagonal masked multi-head self-attention mechanism, and the outputs of multiple heads are integrated. The diagonal masked multi-head self-attention is shown in formula (10)-formula (11): (10); (11); in, Represents the self-attention mechanism through diagonal masking Multi-head self-attention performed; Indicates The output of an attention head; The function concatenates the output of all headers; Represents the specific implementation process of the diagonal masked self-attention mechanism, Indicates that the input Projected onto , and The parameters of the linear layer; express That is, the parameters of the output layer in the diagonal masked multi-head self-attention mechanism; Represents the dimension of the original input; Represents the dimensions of the query vector and key vector; Represents the dimension of the value vector; Represents the dimension of the final output space; Step S14: Using the position encoding module and the hybrid module, the feature matrix is ​​generated and the final representation is constructed through formulas (12)-(22), where the loss function is the weighted sum of the losses of the two tasks, which is described by formulas (23)-(25): The first diagonal masked self-attention mechanism block, the actual input time series is and the corresponding missing mask vector After concatenation, it is used as input and generated through linear transformation and position encoding , as shown in formula (12): (12); in, Indicates positional encoding; and Represent the weight matrix and bias vector respectively; Using stacked attention and feed-forward neural networks Perform multi-layer processing to generate feature matrix , as shown in formula (13): (13); in, represents a feedforward neural network that further processes the features output by the attention module; Indicates stacking layers, each containing and combination of; The feature matrix From high dimension to the same dimension as the input feature, we get , as shown in formula (14): (14); in, Represents a linear transformation matrix, used to transform the feature matrix Dimensions; is the bias vector; Use the mask matrix to merge the original observations and predicted values ​​to get the final complete feature matrix , as shown in formula (15): (15); The second diagonal masked self-attention block takes the output feature matrix of the first diagonal masking block and the missing mask matrix Add position code after splicing Generate a new feature matrix , as shown in formula (16): (16); in, Represents a linear transformation matrix, which is used to transform That is, the output of the first diagonal masked self-attention block and The result of mask vector concatenation is D Dimensions are mapped to Dimension; represents the bias vector; For the feature matrix Perform multi-layer processing to obtain the feature matrix And perform linear projection on it, adding Activation function, generating the second learning representation , as shown in formulas (17)-(18): (17); (18); in, Represents a linear transformation matrix used to transform from The dimension mapping is D Dimension; Represents the bias vector Represents a linear transformation matrix for Perform feature mapping; is the bias vector; Finally, the generated With the generated Dynamic combination to generate the final representation , as shown in formulas (19)-(21): (19); (20); (twenty one); in, Indicates The attention weight matrix calculated by each head; Represents a dynamic weight matrix, used for weighted composition and ; and Represent the weight matrix and bias vector respectively; The final completed feature matrix is ​​the original observations, with the missing values ​​remaining unchanged. The filled matrix is ​​the time series data after interpolation, as shown in formula (22): (twenty two); in, It represents the completed feature vector generated after the input data is processed by two diagonal masked self-attention blocks and weighted combination modules; The loss function is the weighted sum of the losses of the mask filling task and the observation reconstruction task in step S12, and its calculation is shown in formula (23)-formula (25): (twenty three); (twenty four); (25); in, represents reconstruction losses; It means to make up for the loss; represents the weight coefficient, which is used to balance the loss contribution of the two tasks; Represents the total loss function.

[0008] Preferably, in step S2, the demographic data in the electronic medical record is processed using a fully connected layer neural network to initialize the hidden state of the decoder, including: The demographic static features related to the disease are input into the fully connected layer neural network to obtain the corresponding hidden state and synchronously input it into the decoder to initialize the hidden state of the decoder. The weighted input of the hidden layer and the output layer is shown in formula (26), and the calculation formula of the activation function is shown in formula (27): (26); (27); in, Indicates The activation or output value of the layer; Indicates The weighted inputs of the layer; and Respectively represent The weight matrix and bias vector of the layer; Represents the activation function, which is used to introduce nonlinearity; It is The output value of the layer.

[0009] Preferably, in step S3, each GRU unit in the encoder takes a feature vector and a previous hidden state to perform calculations such as formula (28)-formula (31): (28); (29); (30); (31); in, represents the hidden state of the previous time step; Represents the input of the current time step; represents the candidate hidden state; and Represent the weight matrix and bias vector of the update gate respectively; Represents the sigmoid activation function; and Represent the weight matrix and bias vector of the reset gate respectively; represents the final hidden state; represents the update gate; Reset gate. The hidden state of the encoder output and the current hidden state of the decoder Input the attention module and the attention score is calculated as shown in formula (32): (32); Attention Weight and context vector Described by formula (33) and formula (34): (33); (34); in, Represents the current time step of the decoder Each time step of the encoder The attention score between represents the transpose of the attention weight vector; represents the parameter matrix; Represents the index of the current time step; Represents the length of the hidden state sequence output by the encoder.

[0010] Preferably, a fully connected layer is added after the GRU layer to reduce the dimension of the hidden state and adopt ReLu As an activation function, the predicted probability is converted to a non-negative number by the mean absolute error and coefficient: (35); (36); in, Indicates Moment b The predicted value of samples; represents the true value; Represents the true value of all samples The mean of represents the number of time steps; Indicates the number of samples.

[0011] The present invention also provides a lower respiratory tract hospital infection risk prediction system based on deep learning, comprising: Data collection module, used to obtain patients' electronic medical record data from the hospital information system; The data processing module is used to pre-process the acquired electronic medical record data, including data cleaning, formatting and preliminary classification; Irregular time series data processing model construction and training module, which is used to handle missing values ​​in time series data using diagonal masked self-attention mechanism blocks, and to process demographic data for initializing the input of subsequent models; The training module of the lower respiratory tract hospital infection risk prediction model is constructed, and an encoder-decoder model based on the attention mechanism is constructed, in which both the encoder and the decoder use GRU units, and the model is trained using the padded time series data and the initialized hidden state of the decoder to construct a lower respiratory tract hospital infection risk prediction model; The prediction module is used to input the processed data into the trained lower respiratory tract hospital infection risk prediction model and output the patient's lower respiratory tract hospital infection risk prediction result.

[0012] Therefore, the present invention adopts the above-mentioned lower respiratory tract hospital infection risk prediction method and system based on deep learning, and the beneficial technical effects are as follows: (1) In the electronic medical record data processing stage, the present invention uses two diagonal masked self-attentions to effectively interpolate medical sparse and irregular time series data. At the same time, a joint optimization training method of interpolation and reconstruction is designed for the self-attention model, which efficiently captures the dynamic dependencies and feature associations of the time series, effectively restores the structure and regularity of the entire time series, and thus improves the prediction ability of the model.

[0013] (2) During the data input stage of lower respiratory tract hospital infection risk prediction, the present invention combines the patient's static data characteristics with the medical time series data characteristics to capture the key information of multivariate data for predicting the patient's disease risk, effectively improving the prediction ability of the model.

[0014] (3) In the stage of lower respiratory tract hospital infection risk prediction, the present invention initializes the decoder hidden state through the output of the fully connected neural network, uses the attention mechanism to dynamically focus on the information at different positions in the input sequence, dynamically calculates and adjusts the attention weight, and outputs the prediction probability through the encoder-decoder architecture, thereby improving the accuracy of the model prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 Flow chart of the method for predicting the risk of lower respiratory tract hospital infection based on deep learning of the present invention DETAILED DESCRIPTION

[0016] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0017] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.

[0018] Embodiment 1 like Figure 1 As shown, it is a flow chart of the method for predicting the risk of lower respiratory tract hospital infection based on deep learning of the present invention, which includes the following steps: Step S1: Use two diagonal masked self-attention mechanism blocks to fill missing values ​​in the electronic medical record medical time series data to obtain filled time series data, including: Step S11: Set the original physiological state time series data Among them Step , introduce the missing mask vector To mark missing values: (1); in, Indicates that time steps; express dimensional multivariate time series; express Middle Time step Dimensional variables; Indicates that at time step At that time, A mask indicating whether the values ​​in the dimension are missing; The actual input physiological state time series data is , and its corresponding missing mask vector is : (2); in, Indicates that at time step At that time, A mask indicating whether the values ​​in the dimension are artificially randomly masked; express Middle Time step Variables of dimensions; The output is the predicted time series expressed as , introduce the mask vector Distinguish between artificially missing values ​​and original missing values: (3); in, Indicates that at time step At that time, Whether the value in the dimension is a mask of the original missing value; Step S12: Perform the masking task MIT of filling missing values ​​and the reconstruction task ORT of observation values, and calculate the interpolation loss and reconstruction loss through formulas (4)-(6): (4); (5); (6); in, Indicate how the loss is calculated; represents the predicted output of the model; represents the actual target sequence; Represents element-wise multiplication; represents the mask vector; represents the imputation loss of the model; Represents the reconstruction loss of the model; Step S13: Apply the diagonal masked self-attention mechanism module, and describe the calculation process of the diagonal masked self-attention mechanism through formulas (7)-(11): The diagonal masked self-attention mechanism module applies the diagonal mask inside formula (7). The diagonal mask and diagonal masked self-attention are shown in formula (8)-formula (9): (7); (8); (9); in, represents the output of the self-attention layer; Represents the calculation rule of diagonal mask; Represents the specific implementation process of the diagonal masked self-attention mechanism; represents the attention weight; represents the query vector; K represents the key vector; V represents a value vector; Represents feature dimension; represents normalization, converting similarity scores into attention weights; Indicates the first Line The value of the column; The diagonal masked self-attention mechanism is extended to the diagonal masked multi-head self-attention mechanism, and the outputs of multiple heads are integrated. The diagonal masked multi-head self-attention is shown in formula (10)-formula (11): (10); (11); in, Represents the self-attention mechanism through diagonal masking Multi-head self-attention performed; Indicates The output of an attention head; The function concatenates the output of all headers; Represents the specific implementation process of the diagonal masked self-attention mechanism, Indicates that the input Projected onto , and The parameters of the linear layer; express That is, the parameters of the output layer in the diagonal masked multi-head self-attention mechanism; Represents the dimension of the original input; Represents the dimensions of the query vector and key vector; Represents the dimension of the value vector; Indicates the dimension of the final output space.

[0019] Step S14: Using the position encoding module and the hybrid module, the feature matrix is ​​generated and the final representation is constructed through formulas (12)-(22), where the loss function is the weighted sum of the losses of the two tasks, which is described by formulas (23)-(25): The first diagonal masked self-attention mechanism block, the actual input time series is and the corresponding missing mask vector After concatenation, it is used as input and generated through linear transformation and position encoding , as shown in formula (12): (12); in, Indicates positional encoding; and Represent the weight matrix and bias vector respectively; Using stacked attention and feed-forward neural networks Perform multi-layer processing to generate feature matrix , as shown in formula (13): (13); in, represents a feedforward neural network that further processes the features output by the attention module; Indicates stacking layers, each containing and combination of; The feature matrix From high dimension to the same dimension as the input feature, we get , as shown in formula (14): (14); in, Represents a linear transformation matrix, used to transform the feature matrix Dimensions; is the bias vector; Use the mask matrix to merge the original observations and predicted values ​​to get the final complete feature matrix , as shown in formula (15): (15); The second diagonal masked self-attention block takes the output feature matrix of the first diagonal masking block and the missing mask matrix Add position code after splicing Generate a new feature matrix , as shown in formula (16): (16); in, Represents a linear transformation matrix, which is used to transform That is, the output of the first diagonal masked self-attention block and The result of mask vector concatenation is D Dimensions are mapped to Dimension; represents the bias vector; For the feature matrix Perform multi-layer processing to obtain the feature matrix And perform linear projection on it, adding Activation function, generating the second learning representation , as shown in formulas (17)-(18): (17); (18); in, Represents a linear transformation matrix used to transform from The dimension mapping is D Dimension; Represents the bias vector Represents a linear transformation matrix for Perform feature mapping; is the bias vector; Finally, the generated With the generated Dynamic combination to generate the final representation , as shown in formulas (19)-(21): (19); (20); (twenty one); in, Indicates The attention weight matrix calculated by each head; Represents a dynamic weight matrix, used for weighted composition and ; and Represent the weight matrix and bias vector respectively; The final completed feature matrix is ​​the original observations, with the missing values ​​remaining unchanged. The filled matrix is ​​the time series data after interpolation, as shown in formula (22): (twenty two); in, It represents the completed feature vector generated after the input data is processed by two diagonal masked self-attention blocks and weighted combination modules; The loss function is the weighted sum of the losses of the mask filling task and the observation reconstruction task in step S12, and its calculation is shown in formula (23)-formula (25): (twenty three); (twenty four); (25); in, represents reconstruction losses; It means to make up for the loss; represents the weight coefficient, which is used to balance the loss contribution of the two tasks; Represents the total loss function.

[0020] Step S2: Processing demographic data in the electronic medical record using a fully connected neural network to initialize the hidden state of the decoder, including: The demographic static features related to the disease are input into the fully connected layer neural network to obtain the corresponding hidden state and synchronously input it into the decoder to initialize the hidden state of the decoder. The weighted input of the hidden layer and the output layer is shown in formula (26), and the calculation formula of the activation function is shown in formula (27): (26); (27); in, Indicates The activation or output value of the layer; Indicates The weighted inputs of the layer; and Respectively represent The weight matrix and bias vector of the layer; Represents the activation function, which is used to introduce nonlinearity; It is The output value of the layer.

[0021] Step S3, constructing an encoder-decoder model based on the attention mechanism, using the padded time series data and the initialized encoder hidden state to train the model, and constructing a lower respiratory tract hospital infection risk prediction model; Each GRU unit in the encoder takes a feature vector and the previous hidden state to perform calculations such as formula (28)-formula (31): (28); (29); (30); (31); in, represents the hidden state of the previous time step; Represents the input of the current time step; represents the candidate hidden state; and Represent the weight matrix and bias vector of the update gate respectively; Represents the sigmoid activation function; and Represent the weight matrix and bias vector of the reset gate respectively; represents the final hidden state; represents the update gate; Reset gate. The hidden state of the encoder output and the current hidden state of the decoder Input the attention module and the attention score is calculated as shown in formula (32): (32); Attention Weight and context vector Described by formula (33) and formula (34): (33); (34); in, Represents the current time step of the decoder Each time step of the encoder The attention score between represents the transpose of the attention weight vector; represents the parameter matrix; Represents the index of the current time step; Represents the length of the hidden state sequence output by the encoder.

[0022] A fully connected layer is added after the GRU layer to reduce the dimension of the hidden state and adopt ReLu As an activation function, the predicted probability is converted to a non-negative number by the mean absolute error and coefficient: (35); (36); in, Indicates Moment b The predicted value of samples; represents the true value; Represents the true value of all samples The mean of represents the number of time steps; Indicates the number of samples.

[0023] Step S4: input the processed data into the trained lower respiratory tract hospital infection risk prediction model and output the prediction result.

[0024] The present invention will be further described below through specific experiments.

[0025] In order to verify the effectiveness of the proposed interpolation and prediction model, experiments were conducted using real electronic medical record data from a hospital infection management department, and the proposed method was compared with three other baseline models (including LSTM, TCN, and GRU) in the problem of disease prediction. The results are shown in Table 1. The MAE and RMSE of the proposed method are better than those of other baseline models, indicating that it has higher accuracy in predicting lower respiratory tract hospital infections. The lower prediction error indicates that the proposed method can more accurately capture patient risks in early warning of lower respiratory tract hospital infections, which helps to intervene in time, reduce patient mortality and hospital stay, and improve the efficiency of hospital infection management.

[0026] Table 1 Experimental results of this method and other baseline models on the disease prediction problem ;

[0027] Embodiment 2 The lower respiratory tract hospital infection risk prediction system based on deep learning includes: Data collection module, used to obtain patients' electronic medical record data from the hospital information system; The data processing module is used to pre-process the acquired electronic medical record data, including data cleaning, formatting and preliminary classification; Irregular time series data processing model construction and training module, which is used to handle missing values ​​in time series data using diagonal masked self-attention mechanism blocks, and to process demographic data for initializing the input of subsequent models; The training module of the lower respiratory tract hospital infection risk prediction model is constructed, and an encoder-decoder model based on the attention mechanism is constructed, in which both the encoder and the decoder use GRU units, and the model is trained using the padded time series data and the initialized hidden state of the decoder to construct a lower respiratory tract hospital infection risk prediction model; The prediction module is used to input the processed data into the trained lower respiratory tract hospital infection risk prediction model and output the patient's lower respiratory tract hospital infection risk prediction result.

[0028] It is worth noting that the contents not elaborated in detail in the present invention are all prior art and are well known to those skilled in the art.

[0029] Therefore, the present invention adopts the above-mentioned lower respiratory tract hospital infection risk prediction method and system based on deep learning, which can effectively utilize the massive data of electronic medical records, and by integrating static features and dynamic time series features, better capture the patient's health status and improve the accuracy and effectiveness of the prediction.

[0030] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for predicting the risk of lower respiratory tract hospital infection based on deep learning, characterized in that: The following steps are involved: Step S1, using two diagonal masked self-attention mechanism blocks to fill missing values ​​in the electronic medical record medical time series data to obtain filled time series data; Step S2, using a fully connected layer neural network to process demographic data in the electronic medical record and initialize the hidden state of the decoder; Step S3, constructing an encoder-decoder model based on the attention mechanism, using the padded time series data and the initialized decoder hidden state to train the model, and constructing a lower respiratory tract hospital infection risk prediction model; Step S4: input the processed data into the trained lower respiratory tract hospital infection risk prediction model and output the prediction result.

2. The method for predicting the risk of lower respiratory tract hospital infection based on deep learning according to claim 1, characterized in that: In step S1, two diagonal masked self-attention mechanism blocks are used to fill missing values ​​in the electronic medical record medical time series data to obtain filled time series data, including: Step S11: Set the original physiological state time series data Among them Step , introduce the missing mask vector To mark missing values: （1）； in, Indicates that time steps; express dimensional multivariate time series; express Middle Time step Dimensional variables; Indicates that at time step At that time, A mask indicating whether the values ​​in the dimension are missing; The actual input physiological state time series data is , and its corresponding missing mask vector is : （2）； in, Indicates that at time step At that time, A mask indicating whether the values ​​in the dimension are artificially randomly masked; express Middle Time step Variables of dimensions; The output is the predicted time series expressed as , introduce the mask vector Distinguish between artificially missing values ​​and original missing values: （3）； in, Indicates that at time step At that time, Whether the value in the dimension is a mask of the original missing value; Step S12: Perform the masking task MIT of filling missing values ​​and the reconstruction task ORT of observation values, and calculate the interpolation loss and reconstruction loss through formulas (4)-(6): （4）； （5）； （6）； in, Indicate how the loss is calculated; represents the predicted output of the model; represents the actual target sequence; Represents element-wise multiplication; represents the mask vector; represents the imputation loss of the model; Represents the reconstruction loss of the model; Step S13: Apply the diagonal masked self-attention mechanism module, and describe the calculation process of the diagonal masked self-attention mechanism through formulas (7)-(11): The diagonal masked self-attention mechanism module applies the diagonal mask inside formula (7). The diagonal mask and diagonal masked self-attention are shown in formula (8)-formula (9): （7）； （8）； （9）； in, represents the output of the self-attention layer; Represents the calculation rule of diagonal mask; Represents the specific implementation process of the diagonal masked self-attention mechanism; represents the attention weight; represents the query vector; K represents the key vector; V represents a value vector; Represents feature dimension; represents normalization, converting similarity scores into attention weights; Indicates the first Line The value of the column; The diagonal masked self-attention mechanism is extended to the diagonal masked multi-head self-attention mechanism, and the outputs of multiple heads are integrated. The diagonal masked multi-head self-attention is shown in formula (10)-formula (11): （10）； （11）； in, Represents the self-attention mechanism through diagonal masking Multi-head self-attention performed; Indicates The output of an attention head; The function concatenates the output of all headers; Represents the specific implementation process of the diagonal masked self-attention mechanism, Indicates that the input Projected onto , and The parameters of the linear layer; express That is, the parameters of the output layer in the diagonal masked multi-head self-attention mechanism; Represents the dimension of the original input; Represents the dimensions of the query vector and key vector; Represents the dimension of the value vector; Represents the dimension of the final output space; Step S14: Using the position encoding module and the hybrid module, the feature matrix is ​​generated and the final representation is constructed through formulas (12)-(22), where the loss function is the weighted sum of the losses of the two tasks, which is described by formulas (23)-(25): The first diagonal masked self-attention mechanism block, the actual input time series is and the corresponding missing mask vector After concatenation, it is used as input and generated through linear transformation and position encoding , as shown in formula (12): （12）； in, Indicates positional encoding; and Represent the weight matrix and bias vector respectively; Using stacked attention and feed-forward neural networks Perform multi-layer processing to generate feature matrix , as shown in formula (13): （13）； in, represents a feedforward neural network that further processes the features output by the attention module; Indicates stacking layers, each containing and combination of; The feature matrix From high dimension to the same dimension as the input feature, we get , as shown in formula (14): （14）； in, Represents a linear transformation matrix, used to transform the feature matrix Dimensions; is the bias vector; Use the mask matrix to merge the original observations and predicted values ​​to get the final complete feature matrix , as shown in formula (15): （15）； The second diagonal masked self-attention block takes the output feature matrix of the first diagonal masking block and the missing mask matrix Add position code after splicing Generate a new feature matrix , as shown in formula (16): （16）； in, Represents a linear transformation matrix, which is used to transform That is, the output of the first diagonal masked self-attention block and The result of mask vector concatenation is D Dimensions are mapped to Dimension; represents the bias vector; For the feature matrix Perform multi-layer processing to obtain the feature matrix And perform linear projection on it, adding Activation function, generating the second learning representation , as shown in formulas (17)-(18): （17）； （18）； in, Represents a linear transformation matrix used to transform from The dimension mapping is D Dimension; Represents the bias vector Represents a linear transformation matrix for Perform feature mapping; is the bias vector; Finally, the generated With the generated Dynamic combination to generate the final representation , as shown in formulas (19)-(21): （19）； （20）； （21）； in, Indicates The attention weight matrix calculated by each head; Represents a dynamic weight matrix, used to weight the composition and ; and Represent the weight matrix and bias vector respectively; The final completed feature matrix is ​​the original observations, with the missing values ​​remaining unchanged. The filled matrix is ​​the time series data after interpolation, as shown in formula (22): （22）； in, It represents the completed feature vector generated after the input data is processed by two diagonal masked self-attention blocks and weighted combination modules; The loss function is the weighted sum of the losses of the mask filling task and the observation reconstruction task in step S12, and its calculation is shown in formula (23)-formula (25): （23）； （24）； （25）； in, represents reconstruction losses; It means to make up for the loss; represents the weight coefficient, which is used to balance the loss contribution of the two tasks; Represents the total loss function.

3. The method for predicting the risk of lower respiratory tract hospital infection based on deep learning according to claim 2, characterized in that: In step S2, the demographic data in the electronic medical record is processed using a fully connected neural network to initialize the hidden state of the decoder, including: The demographic static features related to the disease are input into the fully connected layer neural network to obtain the corresponding hidden state and synchronously input it into the decoder to initialize the hidden state of the decoder. The weighted input of the hidden layer and the output layer is shown in formula (26), and the calculation formula of the activation function is shown in formula (27): （26）； （27）； in, Indicates The activation or output value of the layer; Indicates The weighted inputs of the layer; and Respectively represent The weight matrix and bias vector of the layer; Represents the activation function, which is used to introduce nonlinearity; It is The output value of the layer.

4. The method for predicting the risk of lower respiratory tract hospital infection based on deep learning according to claim 3, characterized in that: In step S3, each GRU unit in the encoder takes a feature vector and the previous hidden state to perform calculations such as formula (28)-formula (31): （28）； （29）； （30）； （31）； in, represents the hidden state of the previous time step; Represents the input of the current time step; represents the candidate hidden state; and Represent the weight matrix and bias vector of the update gate respectively; Represents the sigmoid activation function; and Represent the weight matrix and bias vector of the reset gate respectively; represents the final hidden state; represents the update gate; Reset gate. The hidden state of the encoder output and the current hidden state of the decoder Input the attention module and the attention score is calculated as shown in formula (32): （32）； Attention Weight and context vector Described by formula (33) and formula (34): （33）； （34）； in, Represents the current time step of the decoder Each time step of the encoder The attention score between represents the transpose of the attention weight vector; represents the parameter matrix; Represents the index of the current time step; Represents the length of the hidden state sequence output by the encoder.

5. The method for predicting the risk of lower respiratory tract hospital infection based on deep learning according to claim 4, characterized in that: A fully connected layer is added after the GRU layer to reduce the dimension of the hidden state and adopt ReLu As an activation function, the predicted probability is converted to a non-negative number by the mean absolute error and coefficient: （35）； （36）； in, Indicates Moment b The predicted value of samples; represents the true value; Represents the true value of all samples The mean of represents the number of time steps; Indicates the number of samples.

6. A lower respiratory tract hospital infection risk prediction system based on deep learning, characterized in that: include: Data collection module, used to obtain patients' electronic medical record data from the hospital information system; The data processing module is used to pre-process the acquired electronic medical record data, including data cleaning, formatting and preliminary classification; Irregular time series data processing model construction and training module, which is used to handle missing values ​​in time series data using diagonal masked self-attention mechanism blocks, and to process demographic data for initializing the input of subsequent models; The training module of the lower respiratory tract hospital infection risk prediction model is constructed, and an encoder-decoder model based on the attention mechanism is constructed, in which both the encoder and the decoder use GRU units, and the model is trained using the padded time series data and the initialized hidden state of the decoder to construct a lower respiratory tract hospital infection risk prediction model; The prediction module is used to input the processed data into the trained lower respiratory tract hospital infection risk prediction model and output the patient's lower respiratory tract hospital infection risk prediction result.

Images