Patient physical condition prediction method and device with missing perception

Abstract

The invention discloses a method and device for predicting the physical condition of a patient with missing perception, and belongs to the technical field of computers. The method comprises the following steps: acquiring EHR data xt sampled by a patient every time, and generating a corresponding binary mask mt according to the missing condition of the EHR data xt; encoding the EHR data xt and the binary mask mt to generate an embedded representation h'v of the patient; performing attention calculation on the embedded representation h'v by using stacked MART blocks; wherein each MART block is used for executing time attention calculation and variable attention calculation; decoding the attention calculation result to obtain a health state representation s of the patient; and performing classification based on the health state representation s of the patient to obtain a physical condition prediction result of the patient. According to the method, the physical state of the patient can be accurately predicted under the condition that the EHR data is partially missing.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology and relates to a method and device for predicting a patient's physical condition with missing perception. Background Art

[0002] In recent years, with the popularization of medical information systems, EHR data has grown and accumulated rapidly. The EHR in the ICU mainly records the patient's diagnosis, laboratory tests, etc., and can be used to describe the patient and analyze the health status, especially to predict the patient's prognosis or physical condition. Therefore, EHR data analysis has become a popular research topic in the medical field. These time series data have attracted extensive attention from computer and medical researchers, and various deep learning methods have been developed to analyze EHR data. For example, early methods that use recurrent networks and attention mechanisms to capture time information assist doctors in making scientific decisions, improving work efficiency, and improving patient prognosis.

[0003] To learn better data representations and improve performance, some studies have tried to improve the classification ability of time series through techniques such as capturing feature associations in complex network architectures and mining the synergistic relationships between variables. For example, there are methods that optimize the design of recurrent networks by combining sampling intervals and disease progression to learn more comprehensive associations. Other studies have also tried to embed medical knowledge into EHR data or combine the knowledge of human experts. For example, some studies enhance the representational ability and interpretability of the model by finding similar patients in the dataset and using their information; there are also studies that combine knowledge graphs to improve the prediction of medical code data.

[0004] However, since patients do not detect all indicators every time, it results in highly sparse EHR data. These missing values will destroy the integrity of the learned representation, and may even mislead the model to capture incorrect feature associations and make incorrect predictions about the patient's health. Currently, these methods do not consider the large number of missing values in EHR data, or simply fill in the missing values (such as mean filling) during data preprocessing, but this may lead to unrealistic assumptions and then mislead the model.

[0005] Some methods focusing on processing irregularly sampled time series try to interpolate missing values to improve prediction performance, but these methods usually perform missing sampling on the existing highly sparse data as the interpolation target and then perform health status prediction, and cannot fully utilize the information in the dataset. In addition, incorrect interpolation will introduce additional noise and mislead subsequent classification tasks.

[0006] Recently, when dealing with missing values, imputation models have been widely used in EHR analysis. Some studies impute missing values through clinical prediction tasks to learn better representations. For example, some methods directly incorporate missing information into time series processing through recurrent models, but these methods do not use reconstruction loss and do not perform actual imputation. Other studies impute irregular observations through methods such as variational autoencoders, generative adversarial networks, and differential equations. However, these methods need to mask some existing data when performing the imputation task, resulting in sparser data and reducing the performance of the prediction task.

[0007] Recently, some studies have proposed a pre-training method to enhance the model's ability to handle missing values through self-supervised pre-training and analyze using complete data in clinical tasks. However, imputation in the input space often traps the model in optimization details and fails to capture the implicit information of the overall sequence, while also potentially changing the data distribution. In contrast, SMART reconstructs missing data in the latent space, prompting the model to learn a more semantic information representation. Summary of the Invention

[0008] The present invention discloses a method and device for predicting a patient's physical condition with missing awareness, which can accurately predict the patient's physical state in the case of partial missing EHR data.

[0009] To achieve the above object, the technical solution of the present invention includes the following content.

[0010] A method for predicting a patient's physical condition with missing awareness, the method comprising:

[0011] Step S1: Obtain the EHR data x of the patient's previous samplings t , and generate a corresponding binary mask m t according to the missing status of the EHR data x t ; where t represents the number of sampling times;

[0012] Step S2: Encode the EHR data x t and the binary mask m t to generate an embedded representation h' v of the patient;

[0013] Step S3: Perform attention calculation on the embedded representation h' v using stacked MART blocks; where each MART block is used to perform time attention calculation and variable attention calculation;

[0014] Step S4: Decode the attention calculation result to obtain a health state representation s of the patient;

[0015] Step S5: Classify based on the patient's health status representation s to obtain the prediction result of the patient's physical condition.

[0016] Further, for the EHR data x t and the binary mask m t are encoded to generate the embedded representation h' v , including:

[0017] Map the EHR data x t and the binary mask m t to the latent representation h t ;

[0018] Concatenate the CLS vector v with all the latent representations h t in the time dimension to obtain the embedded representation h v ;

[0019] Introduce position information into the embedded representation h v through sine position encoding to obtain the embedded representation h' with position information v .

[0020] Further, map the EHR data x t and the binary mask m t to the latent representation h t , including:

[0021] Connect a learned parameter in front of the EHR data x t to obtain the sequence x' t ;

[0022] Connect a True vector in front of the binary mask m t to obtain the sequence m'' t ;

[0023] Connect the sequence x' t and the sequence m'' t on the hidden layer embedding to obtain the latent representation h t .

[0024] Further, the process of calculating the time attention includes:

[0025] Perform a linear transformation on the embedded representation h' v or the output of the previous MART block to obtain the query Q temp , key K temp and value V temp in the time attention mechanism;

[0026] Construct the time attention bias B, and the elements in the time attention bias B Among them, the mask represents the nth variable in the EHR data x i and the bias represents the relationship between the nth variable between the ith sampling and the jth sampling;

[0027] According to the query Q temp , the key K temp and the time attention bias B, generate time attention weights;

[0028] Perform attention calculation on the embedding representation h′ v or the output of the previous MART block based on the time attention weights to obtain the representation generated by time attention.

[0029] Furthermore, the process of the variable attention calculation includes:

[0030] Obtain the representation h generated by time attention temp ;

[0031] Perform a linear transformation on the vector at the first time step in the representation h temp to obtain the query Q in the variable attention mechanism ; var ;

[0032] Perform a linear transformation on the mask with a value of True in the representation h temp to obtain the key K in the variable attention mechanism ; var ;

[0033] Perform a linear transformation on the representation h temp to obtain the value V in the variable attention mechanism var ;

[0034] Based on the query Q var , the key K var and the value V var , generate variable attention weights;

[0035] Perform attention calculation on the representation h temp based on the time attention weights to obtain the output of the corresponding MART block.

[0036] Furthermore, perform the steps S2 to S4 based on a neural network; among them, the neural network includes: an input encoder, a MART block unit, and an embedding decoder, and the process of training the neural network includes:

[0037] Pre-train the neural network by generating augmented data from the original data; wherein, the original data includes: HER data x and binary mask m, and the augmented data includes: HER augmented data x * and binary mask m * ;

[0038] Replace the embedding decoder in the pre-trained neural network with a task-specific decoder, and fine-tune the parameters of the neural network based on the original data to obtain the trained neural network.

[0039] Furthermore, pre-training the neural network by generating augmented data from the original data includes:

[0040] For the given original data, generate augmented data through randomly generated masks ;

[0041] Input the augmented data into the neural network to obtain the reconstructed health state representation

[0042] Input the original data into the label generation network to obtain the reconstruction target wherein, the label generation network includes: an input encoder and a MART block unit, and the parameters of the label generation network are updated by the exponential moving average of the neural network;

[0043] Based on the mask the reconstructed health state representation and the reconstruction target Calculate the loss in the pre-training stage to update the parameters of the neural network.

[0044] A patient physical condition prediction device with missing awareness, the device includes:

[0045] A data collection module for obtaining the EHR data x of the patient's previous samplings t and generating a corresponding binary mask m according to the missing condition of the EHR data x t ; wherein, t represents the number of samplings; t ;

[0046] An input encoder for encoding the EHR data x t and the binary mask m t to generate the embedding representation h' of the patient v ;

[0047] A MART block unit for using stacked MART blocks on the embedding representation h' vPerform attention calculation; wherein, each MART block is used to perform temporal attention calculation and variable attention calculation;

[0048] An embedding decoder, which is used to decode the attention calculation result to obtain the health status representation s of the patient;

[0049] A classification module, which is used to classify based on the patient health status representation s to obtain the physical condition prediction result of the patient.

[0050] An electronic device, characterized in that the electronic device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the missing-aware patient physical condition prediction method described in any one of the above is implemented.

[0051] A computer-readable storage medium, characterized in that computer program instructions are stored on the computer-readable storage medium, and when the computer program instructions are executed by a processor, the missing-aware patient physical condition prediction method described in any one of the above is implemented.

[0052] Compared with the prior art, the present invention has at least the following beneficial effects.

[0053] The present invention proposes a missing-aware model SMART to deal with a large amount of missing in EHR data. The main motivation of the present invention is to enable the model to fully encode the missing information and enable the model to fill in the missing values through a self-supervised pre-training method to improve the model's representation learning ability for EHR time series. Specifically, the model of the present invention independently encodes the features of the multivariate time series input and models the representation of the time series with MART blocks, and both of these two modules introduce missing information. By stacking multiple layers of MART blocks, the model can fully perceive and fill in the missing. In the self-supervised pre-training, the present invention randomly removes some data and allows the model to reconstruct their representations in the latent space. Different from the previous filling methods that reconstruct in the input space, the present invention reconstructs the missing in the latent space. This allows the model to ignore the details that are difficult to interpolate and only learn high-order representations, achieving better generalization. Then, the present invention replaces the filling head of the self-supervised pre-training with a task head for patient health prediction for fine-tuning. To narrow the gap between these two tasks, the present invention introduces a learnable CLS vector as the query of the attention mechanism in the MART block and the basis for patient health prediction.

[0054] The present invention validates the effectiveness of the proposed SMART on three ICU EHR datasets. Through extensive experiments, the present invention finds that both the proposed missing-aware method and self-supervised pre-training method can bring improvements. The experimental results of the present invention show that SMART significantly improves the AUROC and AUPRC of existing methods and becomes the new best method on these datasets. In addition, the present invention also demonstrates that the model is robust under settings with higher missing rates. Description of the Drawings

[0055] Figure 1 Schematic flow diagram of the present invention.

[0056] Figure 2 Performance of SMART at different observation rates in EHR.

[0057] Figure 3 Training time, parameters, and AUPRC (%) of all models on three tasks. The size of the circle represents the number of parameters. GPU running time represents the training time under the same number of training epochs. Detailed Implementation Modes

[0058] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative efforts fall within the protection scope of the present invention.

[0059] The present invention proposes a self-supervised missing-aware representation learning method for predicting the health status of patients - SMART. Figure 1 Shows an overview of its information flow and core design. Among them, on the left: Given EHR data with missing values, the present invention randomly masks them on the existing observed values and reconstructs them in the latent space. The reconstruction target is generated by updating the parameters with EMA. On the right: Shows the detailed architecture of the input encoder and the MART block. The input encoder embeds each variable (which can also be called a test index) and the missing mask into a separate latent space. The MART block captures the feature interactions in the time and variable dimensions while further encoding the missing information.

[0060] Variable-independent Encoder

[0061] Notation description: Let (x, m, y) represent the EHR data of a certain patient, where is an access sequence containing T visits and N variables; is the nth index value of the patient at the tth visit; is a binary mask indicating whether the value is observed; the number of visits T varies from patient to patient; y represents the diagnosis or outcome label.

[0062] To reduce the impact of different access intervals, the present invention aligns the observation intervals. The goal is to optimize the prediction performance of y by improving the model's representation learning ability.

[0063] The present invention adopts a variable independent encoding strategy to map (x, m) into a latent representation where d is the hidden layer dimension. Compared with the variable joint modeling method, this method avoids the information loss that may be caused by variable compression. By combining the variable values and masks, the model of the present invention can automatically capture the interaction between the two. In this way, the encoding processes of different metrics can be carried out in parallel, and the influence of the noise accumulated in historical visits on the current representation is avoided.

[0064] CLS vector: When encoding EHR data, the present invention introduces a learnable vector and connects it before the time series. The role of this vector is to learn the information of the entire sequence in subsequent interactions and serve as the pooled hidden state for prediction (similar to the [CLS] token in the language model BERT). This design also bridges the gap between pre-training and fine-tuning tasks because the information in this vector is used for both reconstruction and prediction. Specifically, the present invention connects the CLS vector to the latent representation h in the time dimension, that is, h v =[v, h], where This process can be understood as connecting a learnable parameter before the visit sequence x and connecting a True vector before the mask m, so that the sequence length increases by one step (as Figure 1 shown). By connecting on the hidden layer embedding h instead of adding a vector to the input data, the present invention brings greater flexibility to the learned representation space. At the end of this module, the present invention introduces position information to the representation h v through sine position encoding to obtain the representation h' with position information v , for subsequent interactions of the MART block.

[0065] MART block

[0066] The MART block is the core module designed to learn the patient's health representation, mainly composed of two attention mechanisms operating in the time and variable dimensions. To further reduce the impact of missing data on representation learning, the present invention introduces masks in the attention mechanism and strengthens the attention weights of existing observations. Temporal attention

[0067] The variable-agnostic encoder mentioned above embeds the patient information into the representation This section describes how to calculate the temporal weights using the mask information. Following the convention of the self-attention mechanism, the present invention calculates the attention weights by applying a linear transformation to h' vPerform a linear transformation and calculate the query, key, and value respectively, i.e., Q temp , K temp , and V temp . To introduce mask information, the present invention constructs a temporal attention bias The bias corresponding to each sampling of each variable represents the relationship between the nth variable when accessing i and j, and is calculated by the following formula:

[0068]

[0069] where the mask m′ of each variable n = [True, m n , [·, ·] represents the concatenation of matrices, is the mask corresponding to each sampling of each variable. Then the present invention obtains the temporal attention weights through the following formula: where both the matrix multiplication and the softmax operation act on the temporal dimension.

[0070] This way strengthens the information from the observed positions and suppresses the information from the unobserved positions. At the same time, the present invention does not completely mask the missing points, but provides the possibility for the model to fill them and utilize the information at their positions.

[0071] Variable attention

[0072] Through the proposed variable attention mechanism, the present invention captures the associations between variables. Different from the previous studies that calculate the relationships between variables for each access separately, the present invention captures the variable relationships from the global perspective of the patient.

[0073] Given the representation generated by the temporal attention the present invention obtains the query Q var , key K var , and value V var of the variable attention mechanism:

[0074]

[0075]

[0076] V var = Linear(h temp )

[0077] The query Q var is generated only by the vector at the first time step (i.e., the position of the learnable parameter v inserted in the input encoder). This operation encourages these vectors to learn the overall health status of the patient. The key K varIt is obtained by the embedded average of all observed access points to minimize the impact of missing access points. Subsequently, the present invention calculates the time-invariant correlation between variables through the following formula and obtains the attention output:

[0078]

[0079] Among these attention mechanisms, the present invention utilizes layer normalization and skip connections to avoid overfitting and accelerate convergence. In addition, a feed-forward network composed of a linear transformation and an activation function is also used, following the design of the Transformer. The MART block can be stacked in multiple layers to achieve sufficient interaction. After encoding by stacking the MART blocks, the embedding of the final MART block is obtained as the patient's health status representation

[0080] Two-stage training strategy

[0081] Pre-training stage:

[0082] To endow the model with the ability to fill in missing values and thus enhance the learned representation, the present invention proposes a self-supervised pre-training method, which is achieved by occluding part of the observed data and reconstructing its representation in the hidden space. Different from previous studies, the present invention does not seek complex manual data augmentation, but simply removes part of the observed data as the target, and the representation of these targets will be reconstructed. When generating the target, the present invention uses a probability interval to randomly generate the target instead of using a fixed probability. This encourages the model to obtain better generalization ability on sequences with different sampling rates rather than overfitting on the missing probability. To further enhance the reconstruction ability and avoid the model falling into local minima, the present invention randomly samples the target in each pre-training cycle instead of using fixed data.

[0083] To implement an effective pre-training task, the present invention adopts a self-motivating paradigm and applies an asymmetric architecture, such as Figure 1 shown. Specifically, given the EHR data (x, m), the present invention randomly generates a mask to partially remove the existing observed data, obtaining the augmented data (x * , m * ). Define the module being trained (input encoder, MART block, and embedding decoder) as f, and define the module for generating labels (input encoder and MART block) as The reconstructed health status representation is generated by f using the augmented data, while the reconstruction target is obtained by inputting the original data into , The parameters are updated by the exponential moving average (EMA) of f. The embedding decoder consists of an MLP with an activation function, taking the patient health state representation using augmented data as input and outputting the reconstructed health state representation This paradigm provides a smooth label update curve, avoiding model oscillation and preventing underfitting of the embedding decoder. The pre-training loss is calculated by L 1 distance, only considering the features that remove data in the latent space (i.e., the positions of

[0084]

[0085] Since the model is trained to reconstruct missing data, it will learn the latent structure of the data and the temporal relationships between variables. In particular, the CLS vector is not aligned in (i.e., the position of the CLS vector in the mask is False, which enables the model to store the information of the entire sequence in the CLS vector and utilize it in the fine-tuning stage.

[0086] Fine-tuning stage:

[0087] After pre-training, the present invention replaces the embedding decoder with a task-specific decoder for predicting the patient health state, such as a classification task. When making predictions, the present invention only uses the representation of the first time step in the health state representation (s 0 ), i.e., the position of the CLS vector. The label decoder is an MLP with layer normalization and an activation function, which can be simplified to a projection function R N×d → R |y| . In the fine-tuning stage, the parameters are updated by the task-specific loss (e.g., cross-entropy for classification).

[0088] In the first few rounds of training, the present invention freezes the parameters of other modules and only updates the label decoder to preserve the pre-trained parameters. The proposed two-stage training process can be represented by Algorithm 1:

[0089] Algorithm 1: Algorithm of SMART

[0090] Input: EHR data x and its mask m.

[0091] Output: Patient health state prediction

[0092] For each pre-training round

[0093] Randomly generate a mask Generate augmented data (x * , m using the probability interval p* )

[0094] Generate a reconstructed patient health state and reconstruction targets

[0095] By minimizing Update f

[0096] Update

[0097] Freeze the parameters of the input encoder and the MART block

[0098] For each fine-tuning round

[0099] If the current round = thawing period

[0100] Thaw the parameters of the input encoder and the MART block

[0101] Update the parameters and make predictions by minimizing the task-specific loss

[0102] Next, a specific experiment is used to illustrate the missing-aware patient physical condition prediction method provided by the present invention.

[0103] Dataset: This experiment follows previous studies and conducts model comparisons on three EHR datasets: Heart Disease, Sepsis, and MIMIC-III dataset.

[0104] · The Heart Disease dataset contains 37 vital signs and biomarkers, describing patients admitted to cardiac, medical, surgical, and trauma intensive care units. Each record contains sparse measurements within 48 hours after admission. This experiment follows the preprocessing process of previous studies and aligns the observation time to hours. After preprocessing, there are 11,988 patients, and the observation rate is 24.7%. The prediction task is to determine in-hospital mortality. 13.8% of the examples belong to the positive class.

[0105] · The Sepsis dataset contains 34 sepsis-related vital signs and laboratory values. The EHR data is recorded hourly and includes records from 40,335 ICU patients. The prediction task is to identify whether a patient is at risk of developing sepsis, and the proportion of the positive class is 7.3%. For convenience, this experiment only uses the first 60 visits of each patient. The observation rate is 19.8%.

[0106] · The MIMIC-III dataset is a multivariate time series dataset containing preprocessed data of 17 physiological signals. This experiment conducts four clinical tasks on MIMIC-III, including in-hospital mortality, organ failure, phenotype classification, and length of stay.

[0107] Evaluation methods: In this experiment, the area under the precision-recall curve (AUPRC) and F1 score on binary classification tasks (including heart disease, sepsis, in-hospital mortality, and organ failure) were used for performance evaluation. AUPRC is the most informative and primary evaluation metric when dealing with highly imbalanced and skewed datasets (such as medical data). In this experiment, the F1 score was calculated, focusing on positive-class patients, which is more relevant in clinical scenarios. Phenotype classification is a multi-label classification, and each label represents the diagnosis of a phenotype. Therefore, in this experiment, phenotype classification was evaluated by the area under the macro and micro receiver operating characteristic curves (AUROC), abbreviated as ma-ROC and mi-ROC. For the prediction of length of hospital stay, the labels were divided into multiple intervals, and ma-ROC and mi-ROC were evaluated. The dataset was randomly divided into an 80% training set, a 10% validation set, and a 10% test set. The model with the best AUPRC (or ma-ROC) on the validation set was evaluated on the test set. To eliminate randomness, this experiment conducted three random seed experiments and reported the mean and standard deviation of the results.

[0108] Baseline methods: In this experiment, existing EHR data processing methods were divided into three paradigms.

[0109] 1. Enhanced representation learning: This class of methods enhances the representation learning of EHR data through various techniques. For example, AdaCare, a multi-scale model that focuses on extracting information from multiple levels; StageNet, which optimizes the LSTM design by combining personalized disease stage development; and ConCare, which learns feature correlations through independent encoders and self-attention mechanisms. Works that use global information or other patient information to assist in modeling also belong to this paradigm, including SAFARI, which learns and integrates feature correlations from a population perspective through graph neural networks (GNNs), PPN, which helps prediction and interpretation by extracting typical patients, and GRASP, which improves representation learning through similar patients and GNNs.

[0110] 2. Missing value interpolation: The second paradigm is the current state-of-the-art technology, aiming to better handle missing value interpolation and achieve better performance in clinical tasks. For example, RainDrop, which uses GNNs to capture dependencies between variables, and Warpformer, which estimates the sampling frequency and uses attention mechanisms to interpolate missing values.

[0111] 3. Self-supervised pre-training methods: This class of methods includes PrimeNet, a model that can sense missing values and uses contrastive learning for input reconstruction for pre-training.

[0112] Experimental results

[0113] Main results

[0114] The overall comparison results of this experiment on six tasks are shown in Table 1. The present invention outperforms the existing baseline methods on all prediction metrics. Specifically, in the binary classification task, the average improvements of the present invention on AUPRC and F1-score are 3.80% and 5.15% respectively, compared to the best baseline. These results demonstrate the effectiveness of the present invention in learning representations and predicting the health status of patients.

[0115] Table 1 Performance comparison with standard deviation on six clinical tasks. The best results are marked in bold.

[0116]

[0117]

[0118] Among the baseline methods, AdaCare performs the worst, probably because its convolutional structure only averages adjacent visit records and cannot perceive missing values. Although StageNet and PrimeNet perform competitively on some tasks (such as heart disease and in-hospital mortality), their performance on the sepsis dataset is surprisingly poor. This is because the observation rate of the sepsis dataset is low, which makes it difficult for the model to learn effective representations. Some recursive models, including ConCare, GRASP, and PPN, perform robustly on all datasets. Nevertheless, the present invention has achieved significant performance improvement through its ability to encode missing values.

[0119] This experiment observes that on datasets with a high missing rate (such as heart disease and sepsis), the present invention shows a greater improvement than other methods, demonstrating its robustness to missing values. In the phenotype classification and length of stay prediction tasks, the present invention also achieves the best results, demonstrating its generalization ability in different clinical scenarios. The results also show that the performance of the present invention is more stable than other methods, such as having a smaller standard deviation.

[0120] Ablation experiment

[0121] To evaluate the effectiveness of each component in the present invention, this experiment conducts ablation experiments on the heart disease, sepsis, and in-hospital mortality tasks. First, this experiment investigates the impact of different pre-training strategies on model performance. This experiment introduces two variants of self-supervised pre-training strategies:

[0122] 1. Impute missing values in the input space (direct interpolation) like the method in previous studies.

[0123] 2. Directly train the model for patient health status prediction without using the proposed self-supervised pre-training (no pre-training).

[0124] As shown in Table 2, although the pre-training strategy in the input space can improve performance compared to the model without pre-training, it is still inferior to the strategy of reconstruction in the representation space proposed by the present invention. These findings highlight the necessity of the pre-training strategy that integrates the missing imputation ability by reconstructing the latent representation.

[0125] In addition, this experiment also explored the importance of different components and designs in the present invention, and the results are also shown in Table 2. Specifically, comparisons were made with the following simplified variants:

[0126] 1. Without masking, completely remove the masking information in the present invention, including the masking in the input encoder and the attention in the MART block;

[0127] 2. Without temporal attention, remove the temporal attention mechanism in the present invention;

[0128] 3. Without variable attention, remove the variable attention mechanism in the present invention;

[0129] 4. Without CLS vector, remove the CLS vector in the input encoder and use the last observation as the query in variable attention and prediction.

[0130] Table 2 Ablation experiments of the present invention on the tasks of heart disease, sepsis, and in-hospital mortality in MIMIC-III.

[0131]

[0132] It can be observed therefrom that all components contribute significantly to the performance improvement. In particular, integrating missing information is crucial for the model to learn high-quality representations. The results show that the temporal and variable attention mechanisms are crucial for capturing temporal and variable dependencies. The CLS vector also plays a key role in enhancing the pre-training effect, as it narrows the gap between the two training stages. Considering these components comprehensively, the present invention can capture the complex temporal relationships and features in sparse EHR data.

[0133] Effect of missing values

[0134] To further study the impact of missingness in the data on performance, this experiment conducted comprehensive experiments on the tasks of heart disease, sepsis, and in-hospital mortality by varying the observation rate from 10% to 100%. The results of AUPRC are as Figure 2As shown, AdaCare shows the most significant performance degradation when the observation rate decreases, which verifies the sensitivity of its convolutional architecture to missing values. Although Warpformer performs stably on the sepsis dataset, it cannot handle missing values well in heart disease and in-hospital mortality. Among all the baseline methods, StageNet and PPN show robustness to missing values. However, there is still a large gap compared with the present invention. Especially in the sepsis task, it is observed that the performance of the present invention hardly degrades even though only 10% of the available data is used. The present invention performs excellently in different missing rate scenarios, demonstrating its significant advantage in perceiving missing values.

[0135] This experiment also compared the running time and parameters of the present invention and the baseline methods in the prediction tasks of heart disease, sepsis, and in-hospital mortality. The number of parameters, the average training time on the GPU (calculated from the start of training), and the AUPRC of different methods are as Figure 3 shown. Since the sepsis dataset contains more patients, the average training time of the model trained based on this dataset is longer. As shown in the figure, RainDrop has the most parameters, and its training time is several times that of some models (such as AdaCare, ConCare, GRASP, Warpformer, and the present invention). However, its prediction performance on different datasets is unsatisfactory and unstable. StageNet, PPN, and PrimeNet achieve competitive performance on some datasets, but the training time is very long. Although AdaCare has the fewest parameters and the shortest training time, its performance is also one of the worst. Generally speaking, the present invention performs lightweight, fast, and reliable among all the models, achieving the best performance with fewer parameters and shorter training time, demonstrating its efficiency and effectiveness. However, it should be noted that since the time attention mechanism in the present invention processes in the visit dimension, when the number of visits increases, the training time may increase quadratically.

[0136] The above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit them. Those of ordinary skill in the art can modify or equivalently replace the technical solutions of the present invention without departing from the scope of the present invention. The protection scope of the present invention shall be subject to what is described in the claims.

Claims

1. A method for predicting the physical condition of a patient with loss perception, characterized in that: The method comprises: Step S1: Obtain the patient's EHR data x for each sampling t , and according to the EHR data x t The missing condition generates the corresponding binary mask m t ; Where t represents the number of sampling times; Step S2: For the EHR data x t and the binary mask m t Encode and generate the embedded representation h′ of the patient v ; Step S3: Use stacked MART blocks to embed the representation h′ v Perform attention calculation; wherein each MART block is used to perform time attention calculation and variable attention calculation; Step S4: Decode the attention calculation result to obtain the patient's health status representation s; Step S5: Classify the patient's health status representation s to obtain a prediction result of the patient's physical condition.

2. The method according to claim 1, characterized in that For the EHR data x t and the binary mask m t Encode and generate the embedded representation h′ of the patient v ,include: The EHR data x t and the binary mask m t Mapped to the latent representation h t ; The CLS vector v is combined with all the latent representations h in the time dimension. t Connect and get the embedded representation h v ; The embedding representation h is encoded by sinusoidal position v Introduce position information to obtain the embedded representation h′ with position information v .

3. The method according to claim 2, characterized in that The EHR data x t and the binary mask m t Mapped to the latent representation h t ,include: In EHR data t Connect a learned parameter to get the sequence x′ t ; In the binary mask m t Connect a True vector to get the sequence m″ t ; The sequence x′ is embedded in the hidden layer t and the sequence m″ t Connect and get the potential representation h t .

4. The method according to claim 1, characterized in that The process of temporal attention calculation includes: For the embedding representation h′ v Or the output of the previous MART block is linearly transformed to obtain the query Q in the temporal attention mechanism temp , key K temp Sum value V temp ; Construct a temporal attention bias B, where the elements in the temporal attention bias B are Among them, the mask Represents EHR data x i The nth variable in , bias Represents the relationship between the nth variable at the i-th sampling and the j-th sampling; According to the query Q temp , the key K temp and the temporal attention bias B, generating a temporal attention weight; The embedding representation h′ is performed based on the temporal attention weight v Or the attention calculation of the output of the previous MART block to obtain the representation generated by temporal attention.

5. The method according to claim 4, characterized in that The variable attention calculation process includes: Get the representation h generated by temporal attention temp ; For the said expression temp The vector for the first time step in Perform a linear transformation to obtain the query Q in the variable attention mechanism var ; For the said expression temp Mask with median True value Perform a linear transformation to obtain the key K in the variable attention mechanism var ; For the said expression temp Perform a linear transformation to obtain the value V in the variable attention mechanism var ; Based on the query Q var , the key K var and the value V var , generate variable attention weights; The representation h is performed based on the temporal attention weight temp The attention calculation is performed to obtain the output of the corresponding MART block.

6. The method according to claim 1, characterized in that Steps S2 to S4 are performed based on a neural network; wherein the neural network includes: an input encoder, a MART block unit and an embedding decoder, and the process of training the neural network includes: The neural network is pre-trained by generating enhanced data of the original data; wherein the original data includes: HER data x and a binary mask m, and the enhanced data includes: HER enhanced data x * and binary mask m * ; The embedding decoder in the pre-trained neural network is replaced with a specific task decoder, and the parameters of the neural network are fine-tuned based on the original data to obtain the trained neural network.

7. The method according to claim 6, characterized in that The neural network is pre-trained by generating augmented data of the original data, including: For a given raw data, a randomly generated mask Generate augmented data; The enhanced data is input into the neural network to obtain a reconstructed health status representation The original data is input into the label generation network to obtain the reconstruction target Wherein, the label generation network comprises: an input encoder and a MART block unit, and the label generation network parameters are updated by the exponential moving average of the neural network; Based on the mask The reconstructed health status indicates and the reconstruction target The pre-training phase loss is calculated to update the parameters of the neural network.

8. A device for predicting the physical condition of a patient with loss perception, characterized in that: The device comprises: Data collection module, used to obtain the patient's EHR data x t , and according to the EHR data x t The missing condition generates the corresponding binary mask m t ; Where t represents the number of sampling times; Input encoder for the EHR data x t and the binary mask m t Encode and generate the embedded representation h′ of the patient v ; MART block unit, for embedding h′ using stacked MART blocks v Perform attention calculation; wherein each MART block is used to perform time attention calculation and variable attention calculation; The embedded decoder is used to decode the attention calculation result to obtain the health status representation s of the patient; The classification module is used to perform classification based on the patient health status representation s to obtain a prediction result of the patient's physical condition.

9. An electronic device, characterized in that: The electronic device comprises: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the method for predicting the physical condition of a patient with loss of perception as described in any one of claims 1-7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the method for predicting the physical condition of a patient with loss of perception as described in any one of claims 1-7.

Images