Time sequence health event prediction method and system based on graph learning and Transform framework
By constructing a GLT-Net model and combining graph learning and the Transformer framework, the problems of disease comorbidity and temporal irregularities in EHR data were solved, achieving more accurate health event prediction and improving the accuracy and reliability of prediction.
Patent Information
- Application Number
- CN202511355051.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-22
- Publication Date
- 2025-12-26
AI Technical Summary
Existing health event prediction methods based on EHR suffer from inadequate utilization of disease comorbidity, imbalance and irregular timing of admission records, and inaccurate representation due to sparse diagnostic coding, all of which affect the accuracy and reliability of predictions.
A GLT-Net model based on graph learning and the Transformer framework is constructed. By combining a patient-diagnosis encoding bipartite graph, a diagnosis encoding hierarchical structure tree, and time interval encoding with graph neural networks and self-attention mechanisms, a comprehensive representation of patient features and temporal information is achieved.
It significantly improves the accuracy and reliability of health event prediction, effectively utilizes comorbidity and temporal information, solves the problems of sparse coding and temporal irregularity in traditional methods, and provides more comprehensive patient characteristic support.
Smart Images
Figure CN121215228A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of medical data processing and deep learning, in particular to a time series health event prediction method and system based on graph learning and a Transformer framework. BACKGROUND
[0002] Electronic health records (EHR) as a modern tool for medical information management store rich health history and medical information of patients, and are of great significance to the development of personalized and precise medicine. Based on the patient's historical admission records in the EHR data, the future health status of the patient can be predicted to provide guidance for medical decision-making.
[0003] With the development of deep learning technology, most existing health event prediction methods focus on extracting disease progression patterns from structured temporal EHR data to predict clinical diagnosis results. The success of convolutional neural networks, recurrent neural networks (RNNs) and Transformer framework in the field of time series information and text processing has led to the emergence of many EHR prediction methods based on these technologies. However, current health event prediction based on EHR data still faces many challenges: 1. There are complex relationships between diagnosis codes, which are not independent medical concepts. However, the diagnosis codes in EHR data are sparse, and a large number of medical codes have low frequency, making it difficult to accurately learn the effective representation of rare codes. At the same time, comorbidity is common, and some diagnosis codes have interdependent relationships. When existing methods incorporate medical ontologies or knowledge graphs, there are problems such as code and entity mismatch and ignoring comorbidity relationships, which affect the reliability of disease relationship understanding and code embedding representation.
[0004] 2. The patient admission records in EHR data are imbalanced, with significant differences in patient admission frequency. And affected by factors such as disease outbreaks, severity of illness and personal circumstances of patients, the admission record time is irregular, and the time interval between adjacent admission records fluctuates greatly.
[0005] 3. The commonly used RNNs model has limitations in processing long EHR sequences, and simple time decay models cannot accurately reflect the complexity of chronic disease progression. Traditional methods also have problems such as relying on manual feature selection, poor generalization ability, insufficient ability to process sparse, high-dimensional and class distribution uneven data, ignoring EHR time series characteristics and high-dimensional correlation, etc. Deep learning models face challenges such as irregular time interval processing, insufficient comorbidity relationship mining, inaccurate multi-dimensional data description and insufficient clinical interpretability. SUMMARY
[0006] The present application aims to provide a time series health event prediction method and system based on graph learning and a Transformer framework, to solve the problems of insufficient utilization of disease comorbidity relationship, imbalance and irregularity of admission records, and inaccurate representation caused by sparse diagnosis coding in the existing EHR-based health event prediction method, and to improve prediction accuracy and reliability.
[0007] According to a first aspect of the embodiments of the present disclosure, a time series health event prediction method based on graph learning and a Transformer framework is provided, comprising: constructing a GLT-Net model and realizing prediction through the model, wherein the GLT-Net model comprises a representation initialization module, an admission record representation module, and a context encoder module, and the working process of each module is as follows: The representation initialization module processing: a patient-diagnosis coding bipartite graph is constructed, which is converted into a patient network association graph by using a single-mode projection technology, and a patient node representation is obtained by using a Node2Vec pre-training method; a diagnosis coding hierarchical structure tree is constructed, a shared vector and a local vector are set for each diagnosis coding, the vectors are updated through a message passing mechanism, and node distances are calculated in a hyperbolic space, so as to train the vectors to minimize the distance between connected nodes and maximize the distance between unconnected nodes, and obtain an initial embedding representation of diagnosis coding; The admission record representation module processing: based on the initial embedding representation of diagnosis coding, a diagnosis weighted undirected association graph is first constructed according to the diagnosis of the patient, which is converted into an adjacency matrix and optimized, and then a multi-layer graph neural network GNN is used to learn a hidden representation of diagnosis coding; at the same time, a time interval coding representation is obtained based on learnable parameters and interval periods; The context encoder module processing: the hidden representation of diagnosis coding and the time interval coding representation are combined into an input vector of a single admission record, a multi-layer Transformer-Encoder architecture is used to aggregate multiple admission record representations, attention weights and contribution coefficients are obtained through a self-attention layer, a final embedding representation of the patient is obtained, and time series health event prediction is realized based on the final embedding representation of the patient.
[0008] In one embodiment, the shared vector and the local vector together constitute the embedding representation of the diagnosis coding The shared vector of the embedding representation of the diagnosis coding reflects the information inherited from the parent node, and the local vector contains private information and details of the diagnosis coding itself; and the updating method of the embedding representation of the diagnosis coding is as follows: wherein, It is a trainable coefficient.
[0009] In one embodiment, the messaging mechanism includes downward flow and upward flow, where child nodes share vectors inherited from the parent node during downward flow, and the parent node aggregates all local vectors of child nodes during upward flow. in, and Each refers to and Number of offspring for Updated embedding representation, for Local vectors; yes The father generation, yes offspring.
[0010] In one embodiment, two diagnostic codes Embedded vector and The distance is calculated in hyperbolic space as follows: make for The loss function for reconstructing the hierarchical structure using the hyperbolic distance between the embedding vectors of two connected node pairs in a given set is defined as: in, To and The set of non-adjacent nodes; To reconstruct the encoding, the representation of the medical code is pre-trained using a loss function to obtain the embedding representation. .
[0011] In one embodiment, a diagnostic-weighted undirected association graph is used. To represent the association between diagnostic codes, The vertex is composed of Composition; for diagnosis If it has been diagnosed in a single patient, then in Construct edges between nodes; if an edge already exists, increment its weight by 1. The weight represents the number of people sharing the same disease. Then, [the graph is...]. Use adjacency matrix This indicates the relationship between quantitative diagnostic codes, and... The optimization process is as follows: wherein, ; optimized element values are used to measure the diagnostic performance on all disease pairs; A multi-layered graph neural network GNN is employed to learn the diagnostic code hidden representation Specifically, the th layer GNN aggregates features as wherein, is the input feature matrix of the th layer, , is the weight matrix of the th layer, and ReLU is the activation function.
[0012] In one embodiment, the time interval encoding is represented as: wherein, are learnable parameters, is the interval period.
[0013] In one embodiment, the self-attention layer obtains attention weights from the embedding representation of each admission record as follows: wherein, is a weight matrix, is the weight to calculate the th attention score ; then the coefficient is calculated to evaluate the contribution of each record, and the embedding representation of the patient is obtained: wherein, is the weight matrix to calculate the attention score , and element-wise multiplication.
[0014] According to a second aspect of the embodiments of the present disclosure, a time-series health event prediction system based on graph learning and a Transformer framework is provided, comprising: The representation initialization module constructs a patient-diagnosis code bipartite graph, converts it into a patient network association graph using a single-mode projection technology, and obtains patient node representations using a Node2Vec pre-training method; a diagnosis code hierarchical tree is constructed, a shared vector and a local vector are set for each diagnosis code, the vectors are updated through a message passing mechanism, node distances are calculated in hyperbolic space, and the training is performed to minimize the connected node distance and maximize the unconnected node distance, to obtain initial embedding representations of diagnosis codes; The admission record representation module, based on the initial embedding representations of diagnosis codes, first constructs a diagnosis weighted undirected association graph according to the patient's diagnosis, converts it into an adjacency matrix and optimizes it, and then learns hidden representations of diagnosis codes using a multi-layer graph neural network GNN; at the same time, based on learnable parameters and interval periods, time interval coding representations are obtained; The context encoder module combines the hidden representations of diagnosis codes and the time interval coding representations into a single admission record input vector, aggregates multiple admission record representations using a multi-layer Transformer-Encoder architecture, obtains attention weights and contribution coefficients through a self-attention layer, and obtains final embedding representations of patients, and realizes time-series health event prediction based on the final embedding representations of patients.
[0015] According to a third aspect of the embodiments of the present disclosure, an electronic device is provided, comprising a memory, a processor, and a computer program stored on the memory and running on the memory, wherein the processor implements the time-series health event prediction method based on graph learning and a Transformer framework when executing the program.
[0016] According to a fourth aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, which stores a computer program, and the program is executed by a processor to implement the time-series health event prediction method based on graph learning and a Transformer framework.
[0017] The above technical solutions adopted by the present application have the following advantages compared with the prior art: 1. Construct a patient network association graph and realize feature learning combined with graph neural network (GNN): On the one hand, the patient-diagnosis code bipartite graph is converted into a patient network association graph through single-mode projection technology, so that patients with common diagnosis codes form an association, effectively mining the potential health association rules among patient groups; on the other hand, based on the association graph, with the help of the powerful graph structure feature aggregation ability of GNN, the complex mapping relationship between patients and diagnosis codes is accurately captured, so that the individual feature of the patient not only contains its own historical diagnosis information, but also integrates the health feature association of similar patients, completely solving the defects of traditional methods that only rely on single historical data of patients and the one-sidedness of individual feature representation, providing more comprehensive patient feature support for subsequent health event prediction.
[0018] 2. Construct a hierarchical structure tree of diagnosis codes and conduct pre-training combined with hyperbolic embedding technology: First, according to the hierarchical logic of ICD codes from macro to micro (such as "mental diseases" containing "schizophrenia" and other subdivided diseases), a hierarchical structure tree is constructed, so that the codes naturally have medical semantic association; second, the embedding representation of the codes is optimized in the hyperbolic space, by minimizing the distance of connected nodes (codes with close semantic or clinical association) and maximizing the distance of unconnected nodes, so that sparse codes (especially rare codes) can inherit the effective features of the parent node based on hierarchical association, while retaining their unique semantics. This module greatly improves the embedding accuracy of sparse diagnosis codes, avoiding the problem of insufficient feature learning and large prediction bias caused by code sparsity in traditional methods.
[0019] 3. The time encoder discards the traditional simple time decay function, converts the time interval into a learnable embedding representation, so that the clinical significance difference between "short interval admission" and "long interval admission" is quantified; the self-attention mechanism can adaptively capture the importance of different admission records in the time dimension (such as the influence weight of recent admission records on future health events is higher), effectively aggregating the time dynamic features of historical admission records. Significantly improves the model's ability to handle "irregular admission record time" (such as the interval between adjacent admissions fluctuates from 36 days to 183 days) in EHR data, overcoming the defects of traditional models (such as RNN, LSTM) in accurately describing long-term, irregular time data.
[0020] In addition, the application is verified on real-world mental illness EHR datasets, and the results show that GLT-Net can not only make full use of comorbidity relationships and time information, but also realize high-precision prediction of patient health events through hierarchical coding feature learning and dynamic time feature selection, and the prediction accuracy and reliability are significantly improved compared with traditional methods. The model provides a new way with robustness (can cope with data sparsity and irregular time problems) and generalization ability (can be transferred to different types of disease prediction scenarios) for automated health event prediction, which can provide scientific and reliable decision support for clinical personalized diagnosis and treatment plan formulation and disease risk warning, and has important practical significance for promoting the development of precision medicine. BRIEF DESCRIPTION OF DRAWINGS
[0021] The accompanying drawings, which form a part of this specification, are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification. The embodiments of these drawings are set to explain the present application and do not constitute an improper limitation on the present application.
[0022] Figure 1 The overall architecture of GLT-Net is shown in the figure; Figure 2 The figure is a schematic diagram of the direction and hierarchy of information flow. DETAILED DESCRIPTION
[0023] The present disclosure will be further described below in conjunction with the drawings and embodiments.
[0024] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present application. Unless otherwise indicated, all technical and scientific terms used in the present application have the same meaning as generally understood by those skilled in the art to which the present application belongs.
[0025] It should be noted that the flowchart and block diagram in the drawings show the possible implementation architecture, function and operation of the method and system according to various embodiments of the present disclosure. It should be noted that each block in the flowchart or block diagram can represent a module, a program segment or a part of code, which can include one or more executable instructions for implementing the logical functions defined in various embodiments. It should also be noted that in some alternative implementations, the functions indicated in the blocks can also occur in an order different from that indicated in the drawings. For example, two blocks indicated in succession can actually be executed substantially in parallel, or they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the flowchart and / or block diagram, and the combination of blocks in the flowchart and / or block diagram, can be implemented using a dedicated hardware-based system that performs the specified functions or operations, or can be implemented using a combination of dedicated hardware and computer instructions.
[0026] Example 1: This embodiment provides a method for predicting time-series health events based on graph learning and the Transformer framework. It includes constructing a GLT-Net model and using this model to achieve prediction. The GLT-Net model includes a representation initialization module, an admission record representation module, and a context encoder module. The collaborative workflow of each module is as follows: The initialization module process involves: constructing a patient-diagnosis coding bipartite graph, converting it into a patient network association graph using single-mode projection technology, obtaining patient node representations using the Node2Vec pre-training method; constructing a diagnostic coding hierarchical structure tree, setting shared and local vectors for each diagnostic coding, updating the vectors through a message passing mechanism, calculating node distances in hyperbolic space, and training with the goal of minimizing the distance between connected nodes and maximizing the distance between disconnected nodes to obtain the initial embedding representation of the diagnostic coding. Specifically, the initialization module is mainly divided into two parts: patient representation initialization and diagnostic encoding hierarchy pre-training, as shown in the figure; (1) Patient indicates initialization Predicting a patient's health events requires considering not only their historical diagnoses but also the unique characteristics of each patient. Unlike previous studies that vectorized patient demographics as their representation, this invention utilizes a patient network graph to generate a unique representation for each patient. First, the patient's admission record is represented using a bipartite graph to construct a patient-diagnosis code association graph, thus better representing the relationship between the patient and the diagnosis code. However, the large size and sparsity of the bipartite graph significantly increase computational complexity, reducing processing efficiency. Then, to obtain the patient representation, after generating the association graph, a single-mode projection technique is used to construct the patient network association graph. This can allow patients with the same diagnostic codes to be linked more closely. Specifically, if patients If there is at least one common bipartite graph neighbor among the nodes (diagnostic coding), then... Connect the two edges and use the number of their common bipartite neighbors as the weight of the edge. This is done during construction and modification. Then, the Node2Vec pre-training method is used to obtain the representation of the patient node. .
[0027] (2) Diagnostic coding level pre-training ICD encoding consists of multiple layers, gradually refining from macro to micro. Each code is nested within a specific level, where a more specific code is a sub-item of the code at the level above it. This invention addresses... The encoding in the code is hierarchically divided to construct a hierarchical encoding tree for diagnostic coding. This is then reconstructed to learn an effective embedding representation of the diagnostic encoding. First, in the encoding tree... Message passing is executed. In order to capture... The information flow in the middle, encoding each diagnosis Two randomly initialized trainable embeddings are used, sharing a vector. and local vectors Together they form Embedded representation Shared vectors Reflects information inherited from the parent node, local vector Includes encoding Their own private information and more detailed information. Ultimately, Embedded representation The update method is as follows: in, It is a trainable coefficient. The direction and hierarchy of information flow are described as follows: Figure 2 As shown: Downward flow indicates that the shared vector of a lower-level medical code is inherited from its parent node, while upward flow refers to the parent node aggregating the local vectors of all child nodes under that medical code. The specific flow patterns of downward and upward flow are as follows: in, yes The father generation, yes offspring, and Each refers to and Number of offspring for Updated embedding representation, for Local vectors.
[0028] In order to reconstruct the code To minimize the distance between similar codes, this invention aims to minimize... The objective is the overall distance between all nodes. More specifically, the distance between two connected nodes should be as small as possible, while the distance between disconnected nodes should be as large as possible. Two diagnostic codes. Embedded vector and Distance is calculated in hyperbolic space as follows: make for The loss function of the hierarchy reconstruction is defined as the hyperbolic distance between the two node embedding vectors: where, is the set of non-adjacent nodes. Due to the restriction of the hierarchical ontology tree, it is not necessary to calculate all node pairs. Finally, the representation of the medical code is pre-trained to obtain the final embedding representation . .
[0029] Admission record representation module processing: based on the initial embedding representation of the diagnosis code, first, the diagnosis weighted undirected association graph is constructed according to the patient's diagnosis, and the adjacency matrix is converted and optimized, and then the multi-layer graph neural network GNN is used to learn the hidden representation of the diagnosis code; at the same time, based on the learnable parameters and interval period, the time interval coding representation is obtained; Specifically, the admission record information is composed of diagnosis information and time information. In order to consider the comorbidity relationship between diseases and make full use of time information, the graph learning and time encoder are designed to embed the representations of the two respectively.
[0030] (1) Graph learning Considering that different diseases are mutually influenced, the present application constructs a diagnosis weighted undirected association graph to represent the relationship between diagnosis codes, wherein, The vertices of are composed of . The projection construction rule is: for diagnosis pair , if both are diagnosed in a patient, an edge is constructed between , if the edge already exists, the weight is added by 1, and the weight represents the number of comorbidities. Then, the graph is represented by an adjacency matrix to quantify the relationship between diagnosis codes, and is optimized, the specific process is as follows: where, . The value of the element after optimization is used to measure the frequency of the diagnosis pair in all disease pairs in . The higher the frequency, the more likely the disease pair appears in the same patient.
[0031] The present application adopts a multi-layer GNN to deeply learn the latent representation of the diagnosis code. Using the graph convolution network architecture, the further hidden representation of the medical code is obtained . Specifically, the feature calculation formula of the first layer GNN aggregation is ,in, It is the first The input feature matrix of the layer, , For the first The weight matrix of the layer, The activation function is ReLU.
[0032] (2) Time encoder Previous time-series-based multi-disease classification models, while considering the importance of temporal information, mostly used it as input to a decay function to assess the retention of previous admission records. However, in reality, chronic diseases decay slowly, and a simple decay function may not accurately reflect the true changes in the condition over time. Therefore, this invention embeds temporal information as part of the model input, and considers shorter time intervals to indicate the importance of adjacent admission records. Time interval encoding representation... as follows: in, All are learnable parameters. This refers to the interval period.
[0033] The context encoder module processes the diagnostic coding hidden representation and the time interval coding representation into a single admission record input vector. It aggregates multiple admission record representations using a multi-layer Transformer-Encoder architecture, obtains attention weights and contribution coefficients through a self-attention layer, and obtains the final patient embedding representation. Based on the final patient embedding representation, it realizes the prediction of time-series health events.
[0034] Specifically, after obtaining the diagnostic coding representation and the time interval coding representation, the context encoder module is used to perform aggregation analysis on the admission information to obtain the patient's embedded representation.
[0035] (1) Contextual analysis Given a patient admission record matrix The diagnostic code embedding representation is obtained by using the diagnostic code and patient embedding representation initialization module. and the patient's initial expression Then, using the admission record representation module, the diagnosis code and admission record are processed separately to obtain the hidden representation of the diagnosis code. Encoding representation of time intervals Among them, regarding single admission records The embedded representation of the diagnostic code is as follows The final input vector is Then, an architecture containing multiple Transformer-Encoder layers is used to generate a comprehensive patient admission status, aggregating the representation of each admission: ,in, It is a hidden representation of each admission record. Dependencies in the admission records are aggregated using the multi-head self-attention mechanism in Transformer.
[0036] (2) Analysis of Admission Records The Transformer-Encoder calculates the patient's... The Embedded representation of secondary admission records Subsequently, this invention defines a self-attention layer to fuse them to learn patient representations for all admission records. This invention not only considers the overall saliency of each admission record but also introduces coefficients. This is used to measure the contribution of each record to the prediction result. The self-attention layer first measures the contribution of each admission record to the prediction result. Calculate attention weights As shown below: in, It is a weight matrix. Is calculating the first Sub-attention score The weights are then calculated; then the coefficients are calculated. To assess the contribution of each record and obtain the patient's embedded representation. : in, It calculates the attention score. The weight matrix, Multiply by elements.
[0037] This invention conducted extensive experiments on a dataset of medical record front pages. This dataset includes patient data from a city in southern Guangdong, China, from January 1, 2016 to December 31, 2022, with diagnoses including mental illnesses. The records include each patient's unique identifier (ID), age, gender, admission and discharge dates, and 16 diagnostic codes, including the primary diagnosis code.
[0038] The diagnostic codes are recorded according to the ICD-10 standard, but there are more than 70,000 unique and active diagnostic codes under the standard, and it is difficult to analyze each code one by one. To solve this challenge, the present application reduces the coding granularity level: the diagnostic codes are uniformly processed into four-digit ICD-10 code form, for example, I25.102 (coronary atherosclerosis) and I25.104 (coronary heart disease arrhythmia type) are both classified as I25.1 (atherosclerotic heart disease).
[0039] In addition, since the present application needs to be modeled based on time intervals, and in order to ensure that there is sufficient hospitalization record to support the prediction task, only the data of patients with more than 3 times of hospitalization in the past 7 years are retained. After preprocessing and cleaning, the basic statistical information of the data set is shown in Table 1.
[0040] The present application randomly divides the preprocessed data set into training set, validation set and test set in the ratio of 7:2:1, wherein the training set contains 19279 patients, the validation set contains 2755 patients, and the test set contains 5508 patients; and in the subsequent two prediction tasks, all diagnostic codes are trained in a self-supervised learning manner. Specifically, in the diagnosis prediction task, the goal is to predict the possible diagnostic codes of the patient's subsequent hospitalization; in the risk prediction task, schizophrenia and delusion are selected as target diseases, and the positive and negative sample proportions of the two diseases in the data set are 78.73% and 21.27% respectively.
[0041] Table 1 Basic statistical data of the data set Under the PyTorch deep learning framework, the present application trains the model using the Adam optimizer on a 16GB graphics memory NVIDIA Tesla P100, the model training number is set to 300 rounds, the batchsize size is 32, and the learning rate is 0.001. In order to improve the generalization ability of the model and prevent overfitting, the network weights are saved when the validation loss reaches the historical minimum.
[0042] The diagnosis prediction, which predicts the diagnosis codes of the next admission, is essentially a multi-label classification problem. The invention selects the weighted F1-score (w-f1) and top-k recall (R@k) as evaluation indicators to assess the comprehensiveness and accuracy of the model. The w-f1 refers to the weighted sum of the f1 scores of each class, which calculates the F1 score of each class and assigns a corresponding weight to reflect the importance of different classes, thereby obtaining a comprehensive score. R@k is a measure of the accuracy of the model in identifying positive samples in the top-k predicted results. It calculates the proportion of true positive samples in the total number of positive samples in the predicted results. This indicator is crucial for evaluating the predictive ability of the model on a specific subset of classes, especially in application scenarios where the accuracy of the top few predicted results is of concern. In this dataset, the average number of medical codes per admission and the maximum number of medical codes are 5.068 and 16, respectively. The invention extends the design of Sherbet and selects k = [10, 20] to calculate R@k.
[0043] The risk prediction, which predicts whether a patient will be diagnosed with a certain disease in the next admission, is essentially a binary classification problem. The evaluation indicators for risk prediction are Accuracy, Precision, Recall, F1-score, and the area under the ROC curve (AUC).
[0044] To verify the effectiveness of GLT-Net, it is compared with advanced methods in the research field. Since there are relatively few open-source electronic health record (EHR)-based health event prediction methods, the invention also selects some advanced methods widely recognized in time series data processing tasks as comparison models. The comparison methods mainly include two categories: EHR-based health event prediction methods and time series data processing methods. The EHR-based health event prediction methods mainly include Dipole, RetainEx, and HiTANet; the time series data processing methods include the widely recognized classic models RNN and LSTM, as well as the advanced methods BEHRT and Hi-BEHRT. For these models, if pre-trained weights are provided, pre-trained models will be used and fine-tuned.
[0045] Table 2 Comparison with advanced methods on benchmark datasets Table 2 shows the prediction performance of the model in the diagnosis prediction and risk prediction tasks in detail. Compared with the baseline model, GLT-Net shows certain improvement compared with all baseline models in the two tasks. For the diagnosis prediction task, GLT-Net achieves the highest w-F1 and R@10 values, which are 30.09 and 39.28, respectively. In the risk prediction task, GLT-Net achieves the highest Acc, Recall, F1 and AUC values.
[0046] The reason why GLT-Net produces such improvement: first, GLT-Net can more effectively represent and utilize the relationship between medical codes through graph learning method, which makes it perform better than models based on RNN and attention mechanism (such as Dipole, RetainEx, TimeLine and HiTANet) when dealing with complex medical data. At the same time, compared with Sherbet model, GLT-Net more comprehensively considers the influence of time information on health event prediction, and through the integration of time information, improves the prediction accuracy of disease development and patient health trend. Finally, compared with Chet and MCHN based on graph method, GLT-Net learns the effective representation of coding through the hierarchical pre-training of diagnosis coding and graph learning, which may pay more attention to the hierarchical structure and internal relationship of coding. And use the Transformer model combined with time information to capture the relationship between admission records, this combination of time series analysis and graph learning method further improves the ability of the model to predict health events. From the performance indicators of diagnosis and risk prediction tasks, GLT-Net based on graph learning and Transformer can effectively learn patient information, diagnosis coding hierarchical information and historical admission records, so as to improve the performance of time series health event prediction.
[0047] Embodiment two: The embodiment provides a time series health event prediction system based on graph learning and Transformer framework, comprising: The representation initialization module constructs a patient-diagnosis code bipartite graph, converts it into a patient network association graph using single mode projection technology, and obtains patient node representation using Node2Vec pre-training method; a diagnosis coding hierarchical structure tree is constructed, a shared vector and a local vector are set for each diagnosis coding, the vectors are updated through a message passing mechanism, node distances are calculated in hyperbolic space, and training is performed to minimize connected node distance and maximize disconnected node distance, to obtain initial embedding representation of diagnosis coding; The admission record representation module, based on the diagnosis code initial embedding representation, first constructs a diagnosis weighted undirected association graph according to the patient's diagnosis, converts it into an adjacency matrix and optimizes it, and then learns the diagnosis code hidden representation by using a multi-layer graph neural network GNN; at the same time, based on the learnable parameters and the interval period, the time interval coding representation is obtained. The context encoder module combines the diagnosis code hidden representation and the time interval coding representation into a single admission record input vector, aggregates multiple admission record representations by using a multi-layer Transformer-Encoder architecture, obtains attention weights and contribution coefficients through a self-attention layer, obtains a patient final embedding representation, and realizes the time series health event prediction based on the patient final embedding representation.
[0048] Embodiment three An electronic device includes a memory, a processor, and a computer program stored on the memory, and the processor executes the program to implement the above-mentioned time series health event prediction method based on graph learning and Transformer framework, including constructing a GLT-Net model and realizing prediction through the model, the GLT-Net model includes a representation initialization module, an admission record representation module, and a context encoder module, and the working process of each module is as follows: The representation initialization module processes: a patient-diagnosis code bipartite graph is constructed, a single-mode projection technology is used to convert it into a patient network association graph, and a Node2Vec pre-training method is used to obtain a patient node representation; a diagnosis code hierarchical structure tree is constructed, a shared vector and a local vector are set for each diagnosis code, the vectors are updated through a message passing mechanism, the node distance is calculated in a hyperbolic space, and the training is performed to minimize the connected node distance and maximize the unconnected node distance, to obtain a diagnosis code initial embedding representation; The admission record representation module processes: based on the diagnosis code initial embedding representation, first construct a diagnosis weighted undirected association graph according to the patient's diagnosis, convert it into an adjacency matrix and optimize it, and then learn the diagnosis code hidden representation by using a multi-layer graph neural network GNN; at the same time, based on the learnable parameters and the interval period, the time interval coding representation is obtained. The context encoder module processes: the diagnosis code hidden representation and the time interval coding representation are combined into a single admission record input vector, multiple admission record representations are aggregated by using a multi-layer Transformer-Encoder architecture, attention weights and contribution coefficients are obtained through a self-attention layer, a patient final embedding representation is obtained, and time series health event prediction is realized based on the patient final embedding representation.
[0049] Embodiment four A computer readable storage medium, having stored thereon a computer program, which when executed by a processor implements the above-mentioned time series health event prediction method based on graph learning and a Transformer framework, comprising constructing a GLT-Net model and realizing prediction through the model, the GLT-Net model comprising a representation initialization module, an admission record representation module and a context encoder module, and the modules work together in the following process: The representation initialization module processing: a patient-diagnosis code bipartite graph is constructed, which is converted into a patient network association graph by using a single-mode projection technology, and a patient node representation is obtained by using a Node2Vec pre-training method; a diagnosis code hierarchical structure tree is constructed, a shared vector and a local vector are set for each diagnosis code, the vectors are updated through a message passing mechanism, node distances are calculated in a hyperbolic space, and the initial embedding representation of the diagnosis code is obtained by training to minimize the connected node distance and maximize the unconnected node distance; The admission record representation module processing: based on the initial embedding representation of the diagnosis code, a diagnosis weighted undirected association graph is first constructed according to the patient's diagnosis, which is converted into an adjacency matrix and optimized, and then a multi-layer graph neural network GNN is used to learn the hidden representation of the diagnosis code; at the same time, a time interval coding representation is obtained based on learnable parameters and interval periods; The context encoder module processing: the hidden representation of the diagnosis code and the time interval coding representation are combined into a single admission record input vector, a multi-layer Transformer-Encoder architecture is used to aggregate multiple admission record representations, attention weights and contribution coefficients are obtained through a self-attention layer, and a final embedding representation of the patient is obtained, and the time series health event prediction is realized based on the final embedding representation of the patient.
[0050] Those skilled in the art should understand that the above-mentioned modules or steps of the present disclosure can be realized by a general computer device, and alternatively, they can be realized by program codes executable by a computing device, so that they can be stored in a storage device and executed by a computing device, or they can be respectively manufactured into individual integrated circuit modules, or a plurality of modules or steps among them can be manufactured into a single integrated circuit module. The present disclosure is not limited to any specific combination of hardware and software.
[0051] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Those skilled in the art can make various changes and modifications to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.
[0052] The specific embodiments of the present disclosure are described above with reference to the accompanying drawings, but are not intended to limit the protection scope of the present disclosure, and those skilled in the art should understand that various modifications or changes made on the basis of the technical solutions of the present disclosure without creative labor are still within the protection scope of the present disclosure.
Claims
1. A time series health event prediction method based on graph learning and a Transformer framework, characterized in that, The method comprises the following steps: The representation initialization module processes: a patient-diagnosis code bipartite graph is constructed, which is converted into a patient network association graph by using a single-mode projection technology, and patient node representations are obtained by using a Node2Vec pre-training method; a diagnosis code hierarchical structure tree is constructed, shared vectors and local vectors are set for each diagnosis code, the vectors are updated by a message passing mechanism, node distances are calculated in a hyperbolic space, and initial embedding representations of diagnosis codes are obtained by training to minimize connected node distances and maximize unconnected node distances. The admission record representation module processes: based on the initial embedding representations of diagnosis codes, a diagnosis weighted undirected association graph is first constructed according to the diagnosis conditions of patients, the graph is converted into an adjacency matrix and optimized, and then a multi-layer graph neural network GNN is used to learn hidden representations of diagnosis codes; at the same time, interval coding representations are obtained based on learnable parameters and interval periods. The context encoder module processes: the hidden representations of diagnosis codes and the interval coding representations are combined into a single admission record input vector, multi-admission record representations are aggregated by using a multi-layer Transformer-Encoder architecture, attention weights and contribution coefficients are obtained by a self-attention layer, final embedding representations of patients are obtained, and time-series health event prediction is realized based on the final embedding representations of patients. The representation initialization module processes: a patient-diagnosis code bipartite graph is constructed, which is converted into a patient network association graph by using a single-mode projection technology, and patient node representations are obtained by using a Node2Vec pre-training method; a diagnosis code hierarchical structure tree is constructed, shared vectors and local vectors are set for each diagnosis code, the vectors are updated by a message passing mechanism, node distances are calculated in a hyperbolic space, and initial embedding representations of diagnosis codes are obtained by training to minimize connected node distances and maximize unconnected node distances.
2. The method of claim 1, wherein the method is based on a graph learning and a Transformer framework. shared vector and local vector , together constitute the diagnostic code embedded representation whose shared vector reflects the information inherited from the parent node, the local vector contains the private information and details of the diagnostic code itself; the embedded representation of the diagnostic code is updated in the following way: wherein is a trainable coefficient. 3.The time-series health event prediction method based on graph learning and Transformer framework according to claim 2, wherein, The messaging mechanism includes a downflow and an upflow, the downflow when the child node The shared vector is inherited from the parent node, the upflow when the parent node Gathers all child local vectors; wherein, and respectively refer to and the number of offspring of is the updated embedding representation, is the local vector of is the parent of is the offspring of 4.The time-series health event prediction method based on graph learning and Transformer framework according to claim 1, wherein, Two diagnostic codes of embedding vectors and The distance in hyperbolic space is computed as follows: Let For the set of connected node pairs in, the loss function that reconstructs the hierarchy using hyperbolic distances between two node embedding vectors is defined as: wherein, is a set of non-adjacent nodes; is a set of non-adjacent nodes; is a reconstruction code; pre-training the representation of medical codes through a loss function to obtain an embedding representation .
5. The method of claim 1, wherein, Using diagnostic weighted undirected association graphs To represent the association between diagnostic codes, The vertex is composed of Composition; for diagnosis If it has been diagnosed in a single patient, then in Construct edges between nodes; if an edge already exists, increment its weight by 1. The weight represents the number of people sharing the same disease. Then, [the graph is...]. Use adjacency matrix This indicates the relationship between quantitative diagnostic codes, and... The optimization process is as follows: wherein, ; the values of the optimized elements are used to measure the diagnostic impact on all disease pairs in the frequency of Learning diagnostic coding hidden representations with multi-layer graph neural networks GNNs In particular, the first layer GNN aggregates features wherein, is an input feature matrix for the first layer, , is a weight matrix for the first layer, is an activation function ReLU.
6. The method of claim 1, wherein, The time interval encoding is represented as: wherein, are all learnable parameters, is an interval period.
7. The method of claim 1, wherein, Self-attention layers according to the embedding representation of each admission record Obtaining attention weights As follows: in, It is a weight matrix. Is calculating the first Sub-attention score The weights are then calculated; then the coefficients are calculated. To assess the contribution of each record and obtain the patient's embedded representation. : wherein, is a weight matrix, are multiplied element-wise. 8. A time-series health event prediction system based on graph learning and Transformer framework, characterized in that, The admission record representation module processes: based on the initial embedding representations of diagnosis codes, a diagnosis weighted undirected association graph is first constructed according to the diagnosis conditions of patients, the graph is converted into an adjacency matrix and optimized, and then a multi-layer graph neural network GNN is used to learn hidden representations of diagnosis codes; at the same time, interval coding representations are obtained based on learnable parameters and interval periods. The context encoder module processes: the hidden representations of diagnosis codes and the interval coding representations are combined into a single admission record input vector, multi-admission record representations are aggregated by using a multi-layer Transformer-Encoder architecture, attention weights and contribution coefficients are obtained by a self-attention layer, final embedding representations of patients are obtained, and time-series health event prediction is realized based on the final embedding representations of patients. The processor realizes the time-series health event prediction method based on graph learning and a Transformer framework according to any one of claims 1-7 when executing the program. 9. An electronic device comprising a memory, a processor, and a computer program stored on the memory to run on the processor, characterized in that, 10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the method of claim 1-7. The program is executed by the processor to implement the method of claim 1-7.