An interpretable sequence prediction method based on spatiotemporal perception
By employing a spatiotemporally aware interpretable sequence prediction method, this approach utilizes the K-Means algorithm and Transformer encoder for disease classification and feature extraction. By combining the GRU model and self-attention mechanism, it addresses the shortcomings in accuracy and interpretability of existing disease prediction technologies, achieving more efficient disease prediction.
Patent Information
- Application Number
- CN202310456772.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-25
- Publication Date
- 2025-10-28
- Estimated Expiration
- 2043-04-25
AI Technical Summary
Existing technologies struggle to effectively combine deep learning and clinical expert experience in predicting medical events, resulting in insufficient accuracy and interpretability in disease prediction.
A spatiotemporally-aware interpretable sequence prediction method is adopted. Disease classification is performed using the K-Means algorithm, disease spatial features are extracted using a Transformer encoder, diagnostic embeddings are updated using a self-attention mechanism, and time-aware information is aggregated using a GRU model for prediction. The model parameters are trained using a binary cross-entropy loss function.
It improves the accuracy and interpretability of disease prediction, distinguishes between different diseases with the same diagnosis and the degree of contribution of different diagnoses to the prediction results, and enhances the interpretability of the model.
Smart Images
Figure CN116451102B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of data mining, specifically relating to an interpretable sequence prediction method based on spatiotemporal awareness. Background Technology
[0002] Sequence prediction is a research hotspot in the field of data mining. Among them, medical event prediction, as a subtask of time series prediction, aims to mine the evolutionary logic between medical events from electronic medical record text data, grasp the development trend of clinical events, and achieve disease prediction to support the diagnosis and decision-making of clinical experts.
[0003] With the development of disease prediction technology, how to use deep learning technology combined with the experience and knowledge of clinical experts to simulate diagnosis and treatment approaches and improve the accuracy and interpretability of disease prediction remains a current research trend. Summary of the Invention
[0004] The purpose of this invention is to provide an interpretable time series prediction method based on spatiotemporal awareness to improve the accuracy and interpretability of disease prediction.
[0005] The technical solution of the present invention is as follows:
[0006] A spatiotemporally aware interpretable sequence prediction method includes the following steps:
[0007] S1 acquires the original electronic medical record data and preprocesses the original electronic medical record data;
[0008] S2 classifies diseases based on the K-Means algorithm for preprocessed raw electronic medical record data;
[0009] S3 uses a Transformer encoder to encode the classified diseases, extracts spatial features between different diseases, and then obtains the diagnostic embedding representation of the patient.
[0010] S4 updates patient diagnostic embeddings based on self-attention mechanisms;
[0011] S5 combines time information and aggregates time-aware diagnostic embeddings through a GRU model, then predicts patient diagnostic outcomes through a prediction layer.
[0012] Furthermore, the original electronic medical record data is preprocessed, including:
[0013] Remove all characteristics except for the patient's consultation time and the diagnosed disease;
[0014] Replace the ICD codes after generating the disease code dictionary;
[0015] Sort the patients' diagnostic records in ascending order by the time of their visit;
[0016] Calculate the time interval between two consecutive patient visits, and set the time interval of the most recent visit to 0;
[0017] The processed data is divided into training set, validation set and test set.
[0018] Furthermore, the ratio of the training set, validation set, and test set is 8:1:1.
[0019] Furthermore, S2 specifically includes:
[0020] Calculate the number of times each disease occurs consecutively two or more times;
[0021] Diseases are classified into different frequency ranges based on the sliding window mechanism and the K-Means algorithm.
[0022] Furthermore, S3 specifically refers to:
[0023] The Transformer encoder input consists of disease embeddings and embeddings for three disease types, with 6 layers and 8 heads for multi-head attention. Finally, the disease embeddings from a single diagnosis are summed and averaged to obtain the patient's diagnostic embedding.
[0024] Furthermore, by combining time information and aggregating time-aware diagnostic embeddings through a GRU model, and then predicting patient diagnostic outcomes through a prediction layer, including:
[0025] Temporal embeddings generated based on time intervals and multilayer perceptrons;
[0026] The patient diagnostic embedding and temporal embedding are spliced together as input to the GRU model.
[0027] Furthermore, the method also includes:
[0028] The model parameters are trained by calculating the distance loss function between the predicted results and the true labels.
[0029] Furthermore, the loss function is a binary cross-entropy loss function.
[0030] Technical effects of the present invention:
[0031] The method described in this invention first preprocesses the raw data to remove redundant features; then classifies diseases based on clustering algorithms and disease occurrence features; uses a Transformer encoder to encode diseases and disease types, extracting spatial features between different diseases to obtain the patient's diagnostic embedding representation; updates the patient's diagnostic embedding based on a self-attention mechanism, extracting interaction information between different diagnoses; combines temporal information and aggregates time-aware diagnostic embeddings using a GRU model, then predicts the patient's diagnostic outcome through a prediction layer; finally, trains the model parameters by calculating the distance loss function between the predicted outcome and the true label. This innovative model proposes a disease classification method and applies it to a disease embedding model; it fully utilizes the spatiotemporal information of electronic medical record time-series data, fully exploring potential development patterns from both spatial and temporal dimensions, and can distinguish between different diseases with the same diagnosis and the degree of contribution of different diagnoses to the prediction results. This not only improves the accuracy of the model's time-series prediction but also further enhances the model's interpretability. Attached Figure Description
[0032] The accompanying drawings illustrate various embodiments generally by way of example rather than limitation, and are used, together with the specification and claims, to explain embodiments of the invention. Where appropriate, the same reference numerals are used in all drawings to refer to the same or similar parts. Such embodiments are illustrative and are not intended to be exhaustive or exclusive embodiments of the apparatus or method.
[0033] Figure 1 A schematic diagram of the method flow of the present invention is shown;
[0034] Figure 2 This invention illustrates a flowchart of obtaining a patient's diagnostic embedding representation using a Transformer encoder.
[0035] Figure 3 The diagram illustrates the process of obtaining prediction results using the GRU model in this invention. Detailed Implementation
[0036] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.
[0037] This invention provides an interpretable sequence prediction method based on spatiotemporal awareness;
[0038] First, the original electronic medical record data is preprocessed to calculate the frequency of consecutive occurrences of each disease. Disease classification is then performed using the K-Means clustering algorithm. A diagnostic embedding is obtained by combining disease classification and co-occurrence features. This is followed by updating the diagnostic embedding based on a self-attention mechanism, aggregating visit information based on temporal information, and finally, a prediction layer to identify potential diseases. A detailed flowchart is shown below. Figure 1 As shown, the following is a detailed explanation.
[0039] Specifically, the steps include the following:
[0040] S1. Preprocess the raw data to remove redundant features;
[0041] S2. Disease classification is completed based on clustering algorithms and disease occurrence characteristics;
[0042] S3. Use a Transformer encoder to encode diseases and disease types, extract spatial features between different diseases, and then obtain the diagnostic embedding representation of the patient;
[0043] S4. Update patient diagnostic embeddings based on self-attention mechanism;
[0044] S5. Combine time information and aggregate time-aware diagnostic embeddings through a GRU model, and predict patient diagnostic results through a prediction layer;
[0045] S6. Train the model parameters by calculating the distance loss function between the predicted results and the true labels.
[0046] In step S1, features other than the patient's consultation time and diagnosed disease are first removed, such as demographic information, medication and surgical data; a disease code dictionary is generated, and then the ICD encoding is replaced so that a high-dimensional embedding representation of the disease can be generated subsequently.
[0047] The patient's diagnostic records are sorted in ascending order by the time of visit; the time interval between two consecutive visits is calculated, and the time interval of the most recent visit is set to 0; the processed data is divided into training set, validation set and test set. In this embodiment, the training set, validation set and test set are divided in a ratio of 8:1:1.
[0048] In step S2, for each disease in the dataset, the number of consecutive occurrences of i is calculated, a feature matrix of disease occurrence is generated, and the matrix elements are mapped to the range of 0 to 1 using the min-max normalization method; the features of disease occurrence are calculated based on a sliding window mechanism at different frequency ranges. In this embodiment, i is greater than 1 and less than 10, and consecutive occurrences of more than 10 are calculated as consecutive occurrences of 10; the size of the sliding window is set to 3, and the step size is set to 3.
[0049] Diseases are classified using clustering algorithms and feature matrices, achieving disease classification from different perspectives. In this embodiment, the K-Means clustering algorithm is selected, and clustering is performed on feature matrices generated by different sliding windows, where K is set to 10.
[0050] In step S3, since the disease sequence in a diagnosis does not have a natural temporal relationship like the word sequence in a sentence, the input of the Transformer encoder needs to be modified. The original positional embedding is removed and replaced with the disease category embedding, and the number of category embedding layers is the same as the number of sliding windows in step S2. The disease embedding output by the Transformer encoder is aggregated based on the attention mechanism, and combined with spatial information to generate the patient's diagnostic embedding. The specific flowchart is shown below. Figure 2 In this embodiment, since there are 3 sliding windows, 3 category embedding layers are needed to resolve diseases from different frequencies, as shown in the formula below;
[0051] e ct =Tranformer(v ct ,v tt1 ,...,v ttW )
[0052]
[0053] Among them, e ct ,e vt Let v represent the disease embedding and diagnosis embedding of the t-th visit, respectively. ct ,v tt1 ,...,v ttW ω represents the patient's diagnosed disease at the t-th diagnosis and the corresponding category of the disease in different windows. i n represents the weight corresponding to each disease. t This indicates the number of diseases diagnosed in the patient during the t-th diagnosis.
[0054] In step S4, the patient diagnosis embedding is updated based on the self-attention mechanism, and the interaction information between different patient diagnoses is extracted. The specific formula is shown below:
[0055] e tt =MLP(Δt) t for t=1,2,...,T
[0056] e t =concat(e vt ,e tt )
[0057] e v1 ,e v2 ,...,evT = self-attention(e v1 ,e v2 ,...,e vT )
[0058] Here, self-attention(·) represents the self-attention function.
[0059] In step S5, firstly, a time-based embedding representation is generated based on the time interval information. The time interval is then concatenated with the patient's diagnostic embedding as a special medical event. Since the patient's diagnosis contains rich temporal information, this invention uses a recurrent neural network to store and update the patient's diagnostic information. Finally, based on the aggregation, the patient representation is used to predict possible diseases. A detailed flowchart is shown below. Figure 3 As shown. In this embodiment, a gated recurrent neural network—Gated Recurrent Unit (GRU)—is used; the output layer of the model consists of a fully connected layer and a softmax(·) function, which can be formalized as follows:
[0060] e tt =MLP(Δt) t fort=1,2,...,T
[0061] e t =concat(e vt ,e tt )
[0062] 1,2,...,T=GRU(e1,e2,...,e T )
[0063]
[0064] Among them, e tt ,e t Let represent the time interval embedding and the diagnostic embedding incorporating time information for the t-th visit, respectively; concat(·) represents the concatenation function; 1, 2, ..., T represent the updated hidden states for the t-th visit, respectively; and W represents the learnable parameters. This represents the prediction result output by the model.
[0065] In step S6, the model of the present invention uses the binary cross-entropy loss function to calculate the loss, and iteratively trains in the direction that reduces the loss to obtain the trained model, as shown below:
[0066]
[0067] Where y is the real label.
[0068] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the technical scope disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A spatiotemporally aware interpretable sequence prediction method, characterized in that, S1. Obtain the original electronic medical record data and preprocess the original electronic medical record data; S2 classifies diseases based on the K-Means algorithm for preprocessed raw electronic medical record data; In step S2, for each disease appearing in the original electronic medical record dataset, the number of times it appears consecutively i times is calculated, a feature matrix of disease occurrence is generated, and the matrix elements are mapped to the range of 0 to 1 using the min-max normalization method. Based on the sliding window mechanism, the features of disease occurrence are calculated in different frequency ranges. The K-Means clustering algorithm was selected to perform clustering on the feature matrices generated by different sliding windows; S3 uses a Transformer encoder to encode the classified diseases, extracts spatial features between different diseases, and then obtains the diagnostic embedding representation of the patient. S4 updates patient diagnostic embeddings based on self-attention mechanisms; S5 combines time information and aggregates time-aware diagnostic embeddings through a GRU model, and predicts patient diagnostic results through a prediction layer. Specifically, S2 is: Calculate the number of times each disease occurs consecutively two or more times; Diseases are classified from different frequency ranges based on the sliding window mechanism and the K-Means algorithm; Specifically, S3 is: The Transformer encoder input consists of disease embeddings and embeddings for three disease types, with 6 layers and 8 heads for multi-head attention. Finally, the disease embeddings from a single diagnosis are summed and averaged to obtain the patient's diagnostic embedding.
2. The prediction method according to claim 1, characterized in that, Preprocessing of the original electronic medical record data includes: Remove all characteristics except for the patient's consultation time and the diagnosed disease; Replace the ICD codes after generating the disease code dictionary; Sort the patients' diagnostic records in ascending order by the time of their visit; Calculate the time interval between two consecutive patient visits, and set the time interval of the most recent visit to 0; The processed data is divided into training set, validation set and test set.
3. The prediction method according to claim 2, characterized in that, The ratio of the training set, validation set, and test set is 8:1:
1.
4. The prediction method according to claim 1, characterized in that, By combining time information and aggregating time-aware diagnostic embeddings through a GRU model, and then predicting patient diagnostic outcomes through a prediction layer, the following is included: Temporal embeddings generated based on time intervals and multilayer perceptrons; The patient diagnostic embedding and temporal embedding are spliced together as input to the GRU model.
5. The prediction method according to claim 1, characterized in that, The method further includes: The model parameters are trained by calculating the distance loss function between the predicted results and the true labels.
6. The prediction method according to claim 5, characterized in that, The loss function is the binary cross-entropy loss function.