Disease prediction method, model, device and storage medium

Abstract

The invention relates to a disease prediction method, a model, equipment and a storage medium. The method comprises the following steps: modeling medical events and static demographic statistical data by adopting a time awareness attention mechanism and a time enhanced Transform to obtain a medical event level representation, and modeling laboratory detection data by using a hierarchical Transform to obtain a laboratory test data representation; constructing a diagnosis time sequence heterogeneous graph by integrating the external knowledge graph; fusing node embedding and edge semantic information of the diagnosis time sequence heterogeneous graph by utilizing a node-edge-type perception heterogeneous graph attention network to obtain a patient graph-level representation; multi-modal complementary features are extracted from the multi-modal representation by adopting multi-head attention and soft orthogonality constraints, and a disease prediction result is generated according to the multi-modal complementary features. According to the method, the complementary relationship between laboratory detection data and medical events can be fully mined, it is ensured that the model can extract really valuable complementary features, and the comprehensiveness and accuracy of patient characterization are improved.

Description

Technical Field

[0001] The present application belongs to the field of biomedical engineering technology, and in particular relates to a disease prediction method, model, device and storage medium. Background Art

[0002] The intensive care unit (ICU) is one of the most critical and complex environments in the modern healthcare system, where patients require continuous monitoring and timely intervention. In this high-risk environment, accurately predicting the patient's clinical trajectory is extremely important. Disease prediction models have made great progress in recent years. However, due to the inherent complexity of clinical data, developing robust disease prediction models for ICU patients faces major challenges. In early clinical predictive modeling research, the focus was on time series modeling of single-modal data. However, ICU data covers multiple modal data. Crucially, a considerable proportion of ICU patients (>68%) actually suffer from multiple comorbidities. Single-modal data cannot fully reflect the patient's health status, and it is necessary to effectively integrate multimodal data including laboratory test data, diagnostic codes, and medication usage records.

[0003] Currently, existing multimodal prediction methods can be roughly divided into two categories: (1) alignment-based methods, which use contrastive learning and self-supervised pre-training techniques to align different modalities into a shared latent space, focusing on reducing cross-modal semantic differences; and (2) attention-based methods, which achieve modality integration through cross-attention mechanisms, progressive information mining, and adaptive fusion networks. However, existing methods cannot effectively model the inherent complementary relationships between different modalities, for example, how laboratory tests provide physiological evidence for diagnosis and vice versa, resulting in poor modality fusion and limited prediction performance.

[0004] Furthermore, to enhance the modeling capabilities of medical events, researchers have introduced various external medical knowledge graphs, such as medical ontologies, knowledge graphs, and large language models (LLMs). However, these approaches typically incorporate all one-hop neighbors in the knowledge graph, indiscriminately introducing all related entities. This not only introduces a large amount of noise information but also significantly increases computational complexity. Furthermore, these approaches primarily focus on static semantic connections, ignoring the temporal dependencies between multiple patient visits and failing to effectively capture the temporal patterns of disease progression. Summary of the Invention

[0005] The present application provides a disease prediction method, model, device and storage medium, which aims to solve at least one of the above-mentioned technical problems in the prior art to a certain extent.

[0006] In order to solve the above problems, this application provides the following technical solutions:

[0007] A disease prediction method, comprising:

[0008] Input the patient's static demographic data, medical events for each visit, and laboratory test data into a pre-trained disease prediction model. The disease prediction model uses a time-aware attention mechanism and a time-enhanced Transformer to model the medical events and static demographic data to obtain a medical event-level representation, and uses a hierarchical Transformer to model the laboratory test data to obtain a laboratory test data representation;

[0009] Constructing a heterogeneous graph of the time series of medical visits of the medical event by integrating external knowledge graphs;

[0010] The node-edge-type-aware heterogeneous graph attention network is used to fuse the node embedding and edge semantic information of the visit time series heterogeneous graph to obtain the patient graph-level representation;

[0011] Multi-head attention and soft orthogonality constraints are used to extract multimodal complementary features from the medical event-level representation, laboratory test data representation and patient graph-level representation, and the patient's disease prediction results are generated based on the multimodal complementary features.

[0012] The technical solution adopted by the embodiment of the present application also includes: the static demographic data includes the patient's age, gender and race information, and the medical events include the patient's diagnosis code, medical procedure code and medication code for each visit, expressed as ME = {[D; P; M]}, where Indicates a diagnostic code, Indicates a medical procedure code, Indicates the drug code, |·| represents the number of elements in a set; the laboratory examination data is recorded as a time series in Indicates patient t i The number of discrete time windows of visits, Represents the laboratory examination data in the j-th time window.

[0013] The technical solution adopted by the embodiment of the present application also includes: the disease prediction model uses a time-aware attention mechanism and a time-enhanced Transformer to model the medical events and static demographic data to obtain a medical event-level representation, specifically:

[0014] The time-aware attention mechanism is used to model the medical events, learn the hospitalization-level representation of the medical events, realize the learning of the admission-level representation, and capture the long-term dependencies in the hospitalization sequence through the time-enhanced Transformer; i Medical events The calculation formula for the admission level representation is:

[0015]

[0016] Among them F me (·) represents the learnable medical event embedding map, F T (·) represents a nonlinear function encoding the time difference between the current visit and the previous visit, where Δt i,T =t T -t i ;

[0017] The hospitalization level indicates H ME The calculation formula is:

[0018] H ME =T-Transformer(H t ,t)=Transformer(H t ,F T ′(Δt i,i+1 )+Pos)

[0019] in Is the admission level, F T ′(·) is a nonlinear function used to encode the time interval between adjacent visits, Δt i,i+1 =t i+1 -t i ,Pos represents the position embedding of each visit;

[0020] The static demographic data are encoded using multi-hot encoding to obtain the patient-level representation H S =multi-hot(x static );

[0021] The hospitalization level is represented by H ME With patient level indicated H S Connect and get the medical event level representation H P =[H ME ;H S ].

[0022] The technical solution adopted in the embodiment of the present application also includes: using a hierarchical Transformer to model the laboratory test data to obtain a laboratory test data representation, specifically:

[0023] Given the tth i Laboratory test data Admission-level Transformer is used to simulate the temporal relationship between laboratory test data of all patients in the same visit:

[0024]

[0025] Aggregate all patients via patient-level converter The final laboratory test data representation H is obtained by coding the visit-level representation LT :

[0026]

[0027] Among them H LT For the final laboratory test data representation, Transformer(·) represents the encoder layer, and its input is and

[0028] The technical solution adopted in the embodiment of the present application also includes: constructing the heterogeneous graph of the time series of medical events by integrating the external knowledge graph, specifically:

[0029] The heterogeneous graph of the time series of medical consultations is represented as: G = (V, E), where V represents a node set. The node set V consists of Diagnosis d |v d ∈D}, surgery {v p |v p ∈P}, and medication {v m |v m ∈M}, where each represents the visit itself, v k The node embedding of ∈V is initialized by concatenating the entity embedding and the type embedding, i.e. Among them F e (·) and F type (·) represents mapping nodes and their types to the corresponding embedding space, Φ(·) represents mapping nodes to corresponding types; E is the edge that captures the node relationship, including the time sequence edge E time 、Association edge E has and semantic edge E rel Three types of key edge relationships, the timing edge E time To capture the time interval between adjacent visits, the associated edge E has Represents the association between medical visits and medical events, the semantic edge E rel Represents the semantic relationship between medical events obtained from the external knowledge graph, and the temporal edge Etime 、Association edge E has and semantic edge E rel The definitions are:

[0030]

[0031] E rel ={(v head ,r h,t ,v tail )|v head ,v tail ∈ME,r h,t ∈KG}

[0032] in represents the time interval between adjacent visits, Indicates the tth i Visits include medical events v k , r h,t Represents the semantic relationship between medical events defined in an external knowledge graph.

[0033] The technical solution adopted by the embodiment of the present application also includes: the node-edge-type aware heterogeneous graph attention network is used to fuse the node embedding and edge semantic information of the medical time series heterogeneous graph to obtain a patient graph-level representation, specifically:

[0034] The node-edge-type aware heterogeneous graph attention network is used to model the time series heterogeneous graph of medical consultations. The node-edge-type aware heterogeneous graph attention network is designed by designing a multi-layer heterogeneous GAT, in which the l+1 layer v k The embed update method is as follows:

[0035]

[0036] where σ(·) is a nonlinear activation function, By integrating node v k The node embedding and edge semantics of u are calculated:

[0037]

[0038] Among them F v (·) is a learnable function that calculates the attention weight by integrating node and edge information, W1 and W2 are transformation matrices, Ψ(·,·) represents the mapping of edges to their types, and F edge (·) represents the embedding edge type; after applying L layers, the patient graph-level representation is obtained

[0039] The technical solution adopted in the embodiment of the present application also includes: extracting multimodal complementary features from the medical event-level representation, laboratory test data representation, and patient graph-level representation using multi-head attention and soft orthogonality constraints, and generating a disease prediction result for the patient based on the multimodal complementary features, specifically:

[0040] Given a medical event level representation H P , patient graph level representation H G And laboratory test data show that H LT The disease prediction model uses multi-head attention to capture the interactive information H between multiple modalities v , and perform mean pooling operation:

[0041] H v =Mean(Multi-HeadAttention(H P ,H G ,H LT ))

[0042] A soft orthogonality constraint is imposed to explore the complementary relationship between the laboratory test data and medical events:

[0043]

[0044] where cos(·,·) represents the mutual information H v With H P 、H G and H LT The cosine similarity between .

[0045] Another technical solution adopted in the embodiment of the present application is: a disease prediction model, comprising:

[0046] Medical event modeling module: This module uses a time-aware attention mechanism and a time-enhanced Transformer to model input medical events and static demographic data to obtain medical event-level representations.

[0047] Laboratory test modeling module: used to model the input laboratory test data using a hierarchical Transformer to obtain laboratory test data representation;

[0048] A medical consultation time series heterogeneous graph construction module: used to construct a medical consultation time series heterogeneous graph of the medical event by integrating an external knowledge graph;

[0049] Heterogeneous graph learning module: used to fuse the node embedding and edge semantic information of the visit time series heterogeneous graph using a node-edge-type aware heterogeneous graph attention network to obtain a patient graph-level representation;

[0050] Complementary feature extraction module: used to extract multimodal complementary features from the medical event level representation, laboratory test data representation and patient graph level representation using multi-head attention and soft orthogonality constraints, and generate disease prediction results for patients based on the multimodal complementary features.

[0051] Another technical solution adopted by the embodiment of the present application is: a device, the device comprising a processor and a memory coupled to the processor, wherein:

[0052] The memory stores program instructions for implementing the disease prediction method;

[0053] The processor is configured to execute the program instructions stored in the memory to control a disease prediction method.

[0054] Another technical solution adopted in the embodiment of the present application is: a storage medium storing program instructions executable by a processor, wherein the program instructions are used to execute the disease prediction method.

[0055] Compared with the prior art, the beneficial effects produced by the embodiments of the present application are as follows: the disease prediction method, model, device and storage medium of the embodiments of the present application adopt a disease prediction model (KCIF) based on complementary feature fusion and medical visit time series heterogeneous graph learning. KCIF uses a time-enhanced Transformer to model discrete medical events, and at the same time constructs a medical visit time series heterogeneous graph to capture the semantic and temporal dependencies between medical events. It uses a hierarchical Transformer to model laboratory test data, effectively capturing multi-scale temporal patterns within and between visits, and can more comprehensively reflect changes in the patient's physiological state. By integrating external knowledge graphs, the semantic associations and temporal dependencies between multiple patient visits are effectively captured, thereby improving the accuracy and completeness of patient representation. The use of complementary feature fusion mechanism and soft orthogonal constraints can fully explore the complementary relationship between laboratory test data and medical events, ensuring that the model can extract truly valuable complementary features, and improving the comprehensiveness and accuracy of patient representation. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a flow chart of the disease prediction method according to an embodiment of the present application;

[0057] Figure 2 This is a schematic diagram of the disease prediction model framework of an embodiment of the present application;

[0058] Figure 3 This is a schematic diagram of the disease prediction model structure of an embodiment of the present application;

[0059] Figure 4 This is a schematic diagram of the device structure of an embodiment of the present application;

[0060] Figure 5 A schematic diagram of the structure of the storage medium of an embodiment of the present application. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0062] The terms "first," "second," and "third" in this application are used only for descriptive purposes and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first," "second," or "third" may explicitly or implicitly include at least one of such features. In the description of this application, "multiple" means at least two, for example, two, three, etc., unless otherwise specifically defined. All directional indications in the embodiments of this application (such as up, down, left, right, front, back...) are only used to explain the relative positional relationship, movement, etc. between the components under a specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indications also change accordingly. In addition, the terms "including" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products, or devices.

[0063] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0064] Specifically, see Figure 1 , is a flow chart of the disease prediction method of the embodiment of the present application. The disease prediction method of the embodiment of the present application includes the following steps:

[0065] S100: Obtain the patient's static demographic data, medical events of each visit, and laboratory test data respectively;

[0066] In this step, the static demographic data includes individual information such as the patient's age, gender, and race. For example, the static demographic data of patient p can be expressed as in and Represent the age, gender and race of patient p respectively. Medical events include diagnosis codes, medical procedure codes and medication codes of each visit of the patient. The present invention uses the standardized ICD-9 coding system to encode medical events. After encoding, the complete medical event of the patient can be expressed as ME = {[D; P; M]}, where Indicates a diagnostic code, Indicates a medical procedure code, Indicates the drug code, |·| represents the number of elements in a set. Laboratory examination data is recorded as a time series in Indicates patient t i The number of discrete time windows of visits, Represents the laboratory test data in the jth time window. Arranging the visits in time series can be expressed as t={t1,t2,…,t T} for each visit A t The corresponding timestamp for the i-th visit is represented as

[0067] S110: Input static demographic data and medical events into the pre-trained disease prediction model. The disease prediction model uses the time-aware attention mechanism and the time-enhanced Transformer to model the static demographic data and medical events and obtain medical event-level representations.

[0068] In this step, if Figure 2 The figure shows a framework diagram of the disease prediction model (KCIF) of an embodiment of the present invention. Considering the impact of the time interval between visits on the progression of a patient's disease, the disease prediction model first uses the time-aware attention mechanism (T-Attention) to model discrete medical events and learn the hospitalization-level representation of medical events, thereby achieving the learning of the admission-level representation; then, a time-enhanced Transformer (T-Transformer) is used to capture the long-term dependencies in the hospitalization sequence. Specifically, the t-th i Medical events The calculation formula for the admission level representation is:

[0069]

[0070] Among them F me (·) represents the learnable medical event embedding map, F T (·) represents a nonlinear function encoding the time difference between the current visit and the previous visit, where Δt i,T =t T -t i .

[0071] Hospitalization level indicates H ME The calculation formula is:

[0072] H ME =T-Transformer(H t ,t)=Transformer(H t ,F T ′(Δt i,i+1 )+Pos)(2)

[0073] in Is the admission level, F T ′(·) is a nonlinear function used to encode the time interval between adjacent visits, Δt i,i+1 =t i+1 -t i Pos represents the position embedding of each visit.

[0074] Finally, the static demographic data of the patients are encoded using multi-hot encoding to obtain the patient-level representation H S =multi-hot(x static ), and the hospitalization level is represented by H ME With patient level indicated H S Connect them to get the medical event level representation H P =[H ME ;H S ].

[0075] S120: The laboratory test data is input into the disease detection model. The disease detection model uses a hierarchical Transformer to model the laboratory test data, obtain the patient's multi-scale temporal patterns within each visit and between multiple visits, and aggregate all the patient's multi-scale temporal patterns to obtain a laboratory test data representation;

[0076] In this step, laboratory test data exhibits complex temporal dependencies within each visit and between multiple visits, which can supplement the quantitative indicators of discrete medical events by capturing multi-scale patterns such as clinical changes and early disease signals that have not yet been recorded in the medical code. In order to effectively capture these multi-scale patterns, the present invention proposes a hierarchical Transformer (H-Transformer) for laboratory test data modeling, which can simultaneously capture multi-scale temporal patterns within each visit and between multiple visits, thereby more comprehensively reflecting the patient's physiological state changes. Specifically, given the t-th i Laboratory test data First, we use the admission-level Transformer (A-Transformer) to simulate the temporal relationship between the laboratory test data of all patients in the same visit, as shown below:

[0077]

[0078] Then, all patients are aggregated through the patient-level transformer (P-Transformer) The final laboratory test data representation H is obtained by coding the visit-level representation LT :

[0079]

[0080] Among them H LT is the final laboratory test data representation, Transformer(·) represents the encoder layer of vanilla Transformer, and its input is and

[0081] S130: By integrating external knowledge graphs, a heterogeneous graph of medical event visits is constructed. The node-edge-type-aware heterogeneous graph attention network is used to dynamically fuse the node embedding and edge semantic information of the heterogeneous graph of medical event visits to obtain a graph-level representation of the patient.

[0082] In this step, in order to better capture the complex relationship between medical visits, the present invention integrates external knowledge graphs (KGs) to construct a medical event time series heterogeneous graph (THAG), which can effectively capture the semantic associations and temporal dependencies between multiple medical visits of patients, thereby improving the accuracy and completeness of patient representation. The time series heterogeneous graph of medical visits is represented as: G = (V, E), where V represents a node set, and the node set V consists of medical visits. Diagnosis d |v d ∈D}, surgery {v p |v p ∈P}, and medication {v m |vm ∈M}, where each Represents the visit itself (not the medical events within that visit). k The node embedding of ∈V is initialized by concatenating the entity embedding and the type embedding, i.e. Among them F e (·) and F type (·) represents mapping nodes and their types to the corresponding embedding space, and Φ(·) represents mapping nodes to corresponding types. E is the edge that captures the node relationship, including the time sequence edge E time 、Association edge E has and semantic edge E rel Three types of key edge relationships, among which the time sequence edge E time Used to capture the time interval between adjacent visits, the associated edge E has Indicates the association between medical visits and medical events, semantic edge E rel Represents the semantic relationship between medical events obtained from the external knowledge graph. Specifically, the temporal edge E time 、Association edge E has and semantic edge E rel is defined as follows:

[0083]

[0084] E rel ={(v head ,r h,t ,v tail )|v head ,v tail ∈ME,r h,t ∈KG} (7)

[0085] in represents the time interval between adjacent visits, Indicates the tth i Visits include medical events v k . r h,t Represents the semantic relationship between medical events defined in the external knowledge graph KG.

[0086] It should be noted that when integrating external knowledge graphs (KGs), large models in the medical field can be used to automatically extract medical entity relationships from literature and clinical records to achieve efficient construction and updating of knowledge graphs. It is also possible to build a unified knowledge graph covering the entire medical decision-making chain by integrating multi-source knowledge such as disease classification, drug instructions, and clinical guidelines.

[0087] Furthermore, in order to capture dynamic entity interactions, the present invention uses the node-edge-type aware heterogeneous graph attention network (NET-HGAT) to model the heterogeneous graph of medical visit time series. NET-HGAT designs a multi-layer heterogeneous GAT (HGAT), in which the l+1 layer v k The embed update method is as follows:

[0088]

[0089] where σ(·) is a nonlinear activation function, By integrating node v k It is calculated based on the node embedding and edge semantics of u, and the calculation formula is as follows:

[0090]

[0091] Among them F v (·) is a learnable function that calculates the attention weight by integrating node and edge information, W1 and W2 are transformation matrices, Ψ(·,·) represents the mapping of edges to their types, and F edge (·) represents the embedding edge type. After applying L layers, we get the final patient graph-level representation

[0092] S140: Use multi-head attention and soft orthogonality constraints to extract multimodal complementary features from medical event-level representation, patient graph-level representation, and laboratory test data representation, and generate patient disease prediction results based on the multimodal complementary features;

[0093] In this step, the disease prediction model (KCIF) is conducive to the loss function of medical event level representation H based on orthogonality constraint and contrastive learning. P , patient graph level representation H G And laboratory test data show that H LT Complementary feature extraction is performed to achieve more comprehensive patient representation learning and reduce the risk of information loss.

[0094] Specifically, given H P 、H G and H LT , the disease prediction model (KCIF) uses multi-head attention (MHA) to capture the interactive information between multiple modalities H v , and then perform mean pooling operation:

[0095] H v =Mean(Multi-HeadAttention(H P ,H G ,H LT ))(10)

[0096] In addition, the present invention ensures that the medical event level representation H P and patient graph-level representation H G The semantic consistency between them is calculated as follows:

[0097]

[0098] in Denotes the H of patients m and n P 、H G The cosine similarity between them, λ represents the weight of controlling the negative sample pair.

[0099] To ensure that the model can capture truly valuable complementary features, this paper imposes a soft orthogonality constraint to fully explore the complementary relationship between laboratory test data and medical events:

[0100]

[0101] where cos(·,·) represents the mutual information H v Cosine similarity with all modality representations.

[0102] In addition, based on the complementary feature fusion representation [H P ;H G ;H v ] loss L pred is minimized to ensure optimal performance of the model:

[0103] L total =L pred +λ1L orth +λ2L C (13)

[0104] Among them, L pred represents binary cross entropy or multi-class cross entropy loss, depending on the specific clinical prediction task.

[0105] It can be understood that the present invention adopts a complementary feature fusion mechanism, which ensures that the model can extract truly valuable complementary features through soft orthogonal constraints, fully explore the complementary relationship between laboratory test data and medical events, and improve the comprehensiveness and accuracy of patient characterization.

[0106] It should be noted that the present invention can also use an adversarial learning network instead of soft orthogonal constraints, extracting multimodal complementary features through a generator and distinguishing between unimodal features and multimodal complementary features through a discriminator, thereby achieving more accurate complementary feature extraction. Furthermore, by establishing a multimodal causal relationship model to distinguish between correlation and causality, clinically significant features can be extracted in a targeted manner, improving predictive interpretability.

[0107] In order to verify the usability and effectiveness of the present invention, experimental evaluations were conducted on the MIMIC-III and MIMIC-IV datasets for various disease prediction tasks. The results are shown in Tables 1 and 2:

[0108] Table 1: Experiments on the MIMIIC III dataset

[0109]

[0110] Table 2: Experiments on the MIMIIC IV dataset

[0111]

[0112] Experimental results show that the disease prediction model (KCIF) of the present invention shows significant improvements over all baseline methods in multi-disease prediction tasks. On MIMIC-III, w-F1 reached 32.41%, and on MIMIC-IV, it reached 31.99%, which are 1.06% and 2.54% higher than the strongest baseline method KGxDP, respectively. The R@20 index improved by 4.81% and 1.78% respectively on their respective datasets compared with the best-performing baseline methods. For cardiovascular disease prediction tasks, KCIF achieved w-F1 of 54.05% on MIMIC-III and 57.90% on MIMIC-IV. The R@20 index reached 83.69% and 81.64%, respectively, which are 1.15% and 0.78% higher than the best baseline methods, respectively. For binary classification tasks: predict whether a patient has a specific disease. In the prediction of hypertension, KCIF's AUC is improved by 9.0% (MIMIC-III) and 1.18% (MIMIC-IV) compared with the best baseline method, and the F1 score improvement is even more significant, increasing by 13.24% and 1.59% respectively. For the prediction of heart failure, KCIF's AUC values on each dataset reached 87.52% and 94.13%, respectively, which are 0.95% and 1.13% higher than the best baseline method. It is worth noting that the KCIF of the present invention also outperforms knowledge enhancement methods such as KGxDP, MMUGL and GraphCare, indicating that simply introducing external knowledge is not enough, even if part of the knowledge comes from a large language model (such as GraphCare). In contrast, KCIF focuses on both temporal graph modeling and complementary feature fusion, and can achieve consistent performance improvements in all experimental settings.

[0113] In addition, the following examples evaluate the contribution of each component in the KCIF of the present invention through comprehensive ablation experiments. The experimental results are shown in Table 3:

[0114] Table 3: Ablation experiment evaluation results

[0115]

[0116] Experimental results show that removing the temporal heterogeneous graph (THAG) leads to the most significant performance degradation in all metrics (w-F1 drops by 4.45% on MIMIC-III and 1.92% on MIMIC-IV), demonstrating its key role in encoding external medical knowledge and capturing temporal dependencies. C It will lead to a moderate performance degradation (w-F1 decreases by 0.94% and 1.22% respectively), which verifies its effectiveness. Similarly, removing the orthogonality constraint L orth This results in a 1.71% and 1.08% performance drop on the corresponding datasets, respectively, confirming its importance in extracting complementary features that cannot be derived from any single modality.

[0117] Based on the above, the disease prediction method of the embodiment of the present application adopts a disease prediction model (KCIF) based on complementary feature fusion and heterogeneous graph learning of medical visits. KCIF uses a time-enhanced Transformer to model discrete medical events, and at the same time constructs a heterogeneous graph of medical visits to capture the semantic and temporal dependencies between medical events. It uses a hierarchical Transformer to model laboratory test data, effectively capturing multi-scale temporal patterns within and between visits, and can more comprehensively reflect changes in the patient's physiological state. By integrating external knowledge graphs, the semantic associations and temporal dependencies between multiple visits of patients can be effectively captured, thereby improving the accuracy and completeness of patient representation. The use of complementary feature fusion mechanism and soft orthogonal constraints can fully explore the complementary relationship between laboratory test data and medical events, ensuring that the model can extract truly valuable complementary features, and improving the comprehensiveness and accuracy of patient representation.

[0118] See also Figure 3 , is a schematic diagram of the disease prediction model structure of an embodiment of the present application. The disease prediction model 40 of the embodiment of the present application includes:

[0119] Medical event modeling module 41: used to model the input medical events and static demographic data using the time-aware attention mechanism and the time-enhanced Transformer to obtain medical event-level representation;

[0120] Laboratory test modeling module 42: used to model the input laboratory test data using a hierarchical Transformer to obtain a laboratory test data representation;

[0121] A medical consultation time series heterogeneous graph construction module 43 is used to construct a medical consultation time series heterogeneous graph of the medical event by integrating an external knowledge graph;

[0122] Heterogeneous graph learning module 44: for fusing node embeddings and edge semantic information of the visit time series heterogeneous graph using a node-edge-type aware heterogeneous graph attention network to obtain a patient graph-level representation;

[0123] Complementary feature extraction module 45: used to extract multimodal complementary features from the medical event level representation, laboratory test data representation and patient graph level representation using multi-head attention and soft orthogonality constraints, and generate disease prediction results for the patient based on the multimodal complementary features.

[0124] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0125] The device provided in the embodiment of the present application can be applied in the aforementioned method embodiment. For details, please refer to the description of the aforementioned method embodiment, which will not be repeated here.

[0126] See also Figure 4 , is a schematic diagram of the device structure of an embodiment of the present application. The device 50 includes:

[0127] A memory 51 storing executable program instructions;

[0128] a processor 52 connected to the memory 51;

[0129] The processor 52 is used to call the executable program instructions stored in the memory 51 and perform the following steps: input the patient's static demographic data, medical events of each visit and laboratory test data into a pre-trained disease prediction model, the disease prediction model uses a time-aware attention mechanism and a time-enhanced Transformer to model the medical events and static demographic data to obtain a medical event-level representation, and uses a hierarchical Transformer to model the laboratory test data to obtain a laboratory test data representation; construct a time-series heterogeneous graph of the medical events by integrating an external knowledge graph; use a node-edge-type-aware heterogeneous graph attention network to fuse the node embedding and edge semantic information of the time-series heterogeneous graph to obtain a patient graph-level representation; use multi-head attention and soft orthogonality constraints to extract multimodal complementary features from the medical event-level representation, laboratory test data representation and patient graph-level representation, and generate the patient's disease prediction results based on the multimodal complementary features.

[0130] The processor 52 may also be referred to as a CPU (Central Processing Unit). The processor 52 may be an integrated circuit chip having signal processing capabilities. The processor 52 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The general-purpose processor may be a microprocessor or any conventional processor.

[0131] See also Figure 5 , is a structural diagram of the storage medium of an embodiment of the present application. The storage medium of the embodiment of the present application stores program instructions 61 that can implement the following steps: inputting the patient's static demographic data, medical events of each visit, and laboratory test data into a pre-trained disease prediction model, the disease prediction model uses a time-aware attention mechanism and a time-enhanced Transformer to model the medical events and static demographic data to obtain a medical event-level representation, and uses a hierarchical Transformer to model the laboratory test data to obtain a laboratory test data representation; constructing a time-series heterogeneous graph of the medical events by integrating an external knowledge graph; using a node-edge-type-aware heterogeneous graph attention network to fuse the node embedding and edge semantic information of the time-series heterogeneous graph of the medical visits to obtain a patient graph-level representation; using multi-head attention and soft orthogonality constraints to extract multimodal complementary features from the medical event-level representation, laboratory test data representation, and patient graph-level representation, and generating a patient's disease prediction result based on the multimodal complementary features. Among them, the program instructions 61 can be stored in the above-mentioned storage medium in the form of a software product, including a number of instructions for enabling a device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the various implementation methods of the present application. The aforementioned storage medium includes: various media that can store program instructions, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or a terminal device such as a computer, a server, a mobile phone, or a tablet. Among them, the server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0132] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0133] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. The above is only an implementation method of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the content of the description and drawings of this application, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A disease prediction method, characterized in that: include: Input the patient's static demographic data, medical events for each visit, and laboratory test data into a pre-trained disease prediction model. The disease prediction model uses a time-aware attention mechanism and a time-enhanced Transformer to model the medical events and static demographic data to obtain a medical event-level representation, and uses a hierarchical Transformer to model the laboratory test data to obtain a laboratory test data representation; Constructing a heterogeneous graph of the time series of medical visits of the medical event by integrating external knowledge graphs; The node-edge-type-aware heterogeneous graph attention network is used to fuse the node embedding and edge semantic information of the visit time series heterogeneous graph to obtain the patient graph-level representation; Multi-head attention and soft orthogonality constraints are used to extract multimodal complementary features from the medical event-level representation, laboratory test data representation and patient graph-level representation, and the patient's disease prediction results are generated based on the multimodal complementary features.

2. The disease prediction method according to claim 1, characterized in that: The static demographic data includes the patient's age, gender and race information, and the medical events include the patient's diagnosis code, medical procedure code and medication code for each visit, which is expressed as ME={[D; P; M]}, where Indicates a diagnostic code, Indicates a medical procedure code, Indicates the drug code, |·| represents the number of elements in a set; the laboratory examination data is recorded as a time series in Indicates patient t i The number of discrete time windows of visits, Represents the laboratory examination data in the j-th time window.

3. The disease prediction method according to claim 2, characterized in that: The disease prediction model uses a time-aware attention mechanism and a time-enhanced Transformer to model the medical events and static demographic data to obtain a medical event-level representation, specifically: The time-aware attention mechanism is used to model the medical events, learn the hospitalization-level representation of the medical events, realize the learning of the admission-level representation, and capture the long-term dependencies in the hospitalization sequence through the time-enhanced Transformer; i Medical events The calculation formula for the admission level representation is: Among them F me (·) represents the learnable medical event embedding map, F T (·) represents a nonlinear function encoding the time difference between the current visit and the previous visit, where Δt i,T =t T -t i ; The hospitalization level indicates H ME The calculation formula is: H ME =T-Transformer(H t ,t)=Transformer(H t ,F T ′(Δt i,i+1 )+Pos) in Is the admission level, F T ′(·) is a nonlinear function used to encode the time interval between adjacent visits, Δt i,i+1 =t i+1 -t i ,Pos represents the position embedding of each visit; The static demographic data are encoded using multi-hot encoding to obtain the patient-level representation H S =multi-hot(x static ); The hospitalization level is represented by H ME With patient level indicated H S Connect and get the medical event level representation H P =[H ME ;H S ].

4. The disease prediction method according to any one of claims 1 to 3, characterized in that: The laboratory test data is modeled using a hierarchical Transformer to obtain a laboratory test data representation, specifically: Given the tth i Laboratory test data Admission-level Transformer is used to simulate the temporal relationship between laboratory test data of all patients in the same visit: Aggregate all patients via patient-level converter The final laboratory test data representation H is obtained by coding the visit-level representation LT : Among them H LT For the final laboratory test data representation, Transformer(·) represents the encoder layer, and its input is and 5. The disease prediction method according to claim 4, characterized in that: The construction of the heterogeneous graph of the medical event's time series by integrating the external knowledge graph is specifically as follows: The heterogeneous graph of the time series of medical consultations is represented as: G = (V, E), where V represents a node set. The node set V consists of Diagnosis d |v d ∈D}, surgery {v p |v p ∈P}, and medication {v m |v m ∈M}, where each represents the visit itself, v k The node embedding of ∈V is initialized by concatenating the entity embedding and the type embedding, i.e. Among them F e (·) and F type (·) represents mapping nodes and their types to the corresponding embedding space, Φ(·) represents mapping nodes to corresponding types; E is the edge that captures the node relationship, including the time sequence edge E time 、Association edge E has and semantic edge E rel Three types of key edge relationships, the timing edge E time To capture the time interval between adjacent visits, the associated edge E has Represents the association between medical visits and medical events, the semantic edge E rel Represents the semantic relationship between medical events obtained from the external knowledge graph, and the temporal edge E time 、Association edge E has and semantic edge E rel The definitions are: E rel ={(in head ,r h,t ,in tail )∣v head ,in tail ∈ME,r h,t ∈KG} in represents the time interval between adjacent visits, Indicates the tth i Visits include medical events v k , r h,t Represents the semantic relationship between medical events defined in an external knowledge graph.

6. The disease prediction method according to claim 5, characterized in that: The node-edge-type aware heterogeneous graph attention network is used to fuse the node embedding and edge semantic information of the medical time series heterogeneous graph to obtain the patient graph-level representation, specifically: The node-edge-type aware heterogeneous graph attention network is used to model the time series heterogeneous graph of medical consultations. The node-edge-type aware heterogeneous graph attention network is designed by designing a multi-layer heterogeneous GAT, in which the l+1 layer v k The embed update method is as follows: where σ(·) is a nonlinear activation function, By integrating node v k The node embedding and edge semantics of u are calculated: Among them F v (·) is a learnable function that calculates the attention weight by integrating node and edge information, W1 and W2 are transformation matrices, Ψ(·,·) represents the mapping of edges to their types, and F edge (·) represents the embedding edge type; after applying L layers, the patient graph-level representation is obtained 7. The disease prediction method according to claim 6, characterized in that: The method uses multi-head attention and soft orthogonality constraints to extract multimodal complementary features from the medical event-level representation, laboratory test data representation, and patient graph-level representation, and generates a disease prediction result for the patient based on the multimodal complementary features, specifically: Given a medical event level representation H P , patient graph level representation H G And laboratory test data show that H LT The disease prediction model uses multi-head attention to capture the interactive information H between multiple modalities v , and perform mean pooling operation: H v =Mean(Multi-HeadAttention(H P ,H G ,H LT )) A soft orthogonality constraint is imposed to explore the complementary relationship between the laboratory test data and medical events: where cos(·,·) represents the mutual information H v With H P 、H G and H LT The cosine similarity between .

8. A disease prediction model, characterized in that: include: Medical event modeling module: This module uses a time-aware attention mechanism and a time-enhanced Transformer to model input medical events and static demographic data to obtain medical event-level representations. Laboratory test modeling module: used to model the input laboratory test data using a hierarchical Transformer to obtain laboratory test data representation; A medical consultation time series heterogeneous graph construction module: used to construct a medical consultation time series heterogeneous graph of the medical event by integrating an external knowledge graph; Heterogeneous graph learning module: used to fuse the node embedding and edge semantic information of the visit time series heterogeneous graph using a node-edge-type aware heterogeneous graph attention network to obtain a patient graph-level representation; Complementary feature extraction module: used to extract multimodal complementary features from the medical event level representation, laboratory test data representation and patient graph level representation using multi-head attention and soft orthogonality constraints, and generate disease prediction results for patients based on the multimodal complementary features.

9. A device, characterized in that The device includes a processor and a memory coupled to the processor, wherein: The memory stores program instructions for implementing the disease prediction method according to any one of claims 1 to 7; The processor is configured to execute the program instructions stored in the memory to control a disease prediction method.

10. A storage medium, characterized in that: The device stores program instructions executable by a processor, wherein the program instructions are used to execute the disease prediction method according to any one of claims 1 to 7.

Images