An electronic health record time series data generation method based on a diffusion model

Abstract

The application discloses an electronic health record time series data generation method based on a diffusion model, relates to the technical field of medicine and artificial intelligence, and comprises the following steps: S1, extracting and standardizing static variables and original dynamic time series data of a patient, processing missing values in the original dynamic time series data, and calculating row importance; S2, learning a joint distribution of the static variables based on a variational autoencoder, and generating a static variable group conforming to the joint distribution; S3, converting the original dynamic time series data into noise data by gradually adding Gaussian noise for diffusion noise addition, and denoising the noise data based on an improved diffusion model to generate synthetic dynamic time series data. The application is based on a diffusion model architecture, realizes generation of high-quality static data and effective condition guidance, accurately models complex dependence between physiological variables, realizes accurate control of a generation process and high fidelity of generated data, and enhances credibility and practicality of the model.

Description

Technical Field

[0001] This invention relates to the fields of medicine and artificial intelligence, and in particular to a method for generating time-series data of electronic health records based on a diffusion model. Background Technology

[0002] In practical applications, datasets often exhibit a significant imbalance between positive and negative examples, a situation particularly pronounced in electronic health record (EHR) datasets. Time-series data within EHRs is a key focus of medical artificial intelligence research. Therefore, generating high-quality EHR time-series data has become a critical task, with the core challenge lying in accurately capturing the complex dependencies within the data to generate synthetic data that conforms to medical principles while possessing individualized characteristics.

[0003] In recent years, data generation technologies, especially the synthesis of time-series data from electronic health records, have received increasing attention. However, existing data generation methods have significant limitations. Traditional methods, such as SMOTE, generate samples through linear interpolation but cannot capture the correlation between static features (such as age and gender) and dynamic time-series data in electronic health records, nor can they model complex time dependencies and inter-variable dependencies. Deep learning-based methods, such as generative adversarial networks (GANs), can generate time-series data, but suffer from training instability and pattern collapse. Furthermore, their generators typically employ recurrent or autoregressive structures, making it difficult to accurately characterize the complex relationships between different physiological indicators. In addition, existing methods generally lack effective conditional guidance mechanisms, failing to integrate medical knowledge into the generation process, resulting in insufficient medical rationale for the generated data. Simultaneously, the models have poor interpretability, making it difficult to gain clinical trust.

[0004] In summary, existing technologies generally suffer from the following common shortcomings: First, they fail to fully capture the intrinsic relationship between static variables and dynamic time-series data in electronic health records. Existing technologies often focus on the generation of time-series data itself, neglecting the guiding role of relatively fixed static variables (such as gender and age) on physiological data and their trends. Second, they ignore the dependencies between different physiological indicators in dynamic time-series data. While existing technologies attempt to capture the changing patterns of dynamic time-series data over time, the complex medical constraints between different physiological indicators at the same point in time are not effectively modeled. Third, in data generation based on advanced generative frameworks such as diffusion models, there is a lack of robust conditional guidance mechanisms. Existing technologies often generate data from a computer perspective, failing to deeply integrate rich medical knowledge as conditions into the generation process, thus limiting the medical rationality of the generated data. Fourth, they lack interpretable design. Most existing technologies focus on model performance optimization, and their generation process is like a "black box," lacking medical explanation and failing to gain the trust of clinicians, thereby hindering their application in real-world medical scenarios. Summary of the Invention

[0005] To address the problems in existing electronic health record (EHR) data generation methods, such as failing to fully capture the relationship between static variables and time-series data in EHRs, ignoring inter-column dependencies in time-series data, lacking a robust conditional guidance mechanism in diffusion model-based data generation, and lacking interpretable design, this invention proposes an EHR time-series data generation method based on a diffusion model to solve these problems.

[0006] This application discloses a method for generating time-series electronic health record data based on a diffusion model, comprising the following steps: S1. Extract and standardize the patient's static variables and original dynamic time-series data from the original electronic health record data, process the missing values ​​in the original dynamic time-series data, and calculate the row importance. S2. Learn the joint distribution of the static variables based on the variational autoencoder, and generate a set of static variables that conforms to the joint distribution; S3. The original dynamic time series data is converted into noise data by gradually adding Gaussian noise for diffusion and noise addition, and the noise data is denoised based on the improved diffusion model to generate synthetic dynamic time series data. The improved diffusion model includes an encoder that encodes multi-dimensional information and a decoder that guides multi-angle conditions. The encoder for multi-dimensional information encoding encodes static variables, diffusion time steps, variable types, time series values, and row importance in multiple dimensions and fuses them to generate a conditional vector. The multi-angle condition-guided decoder uses condition vectors to denoise noisy data through a multi-angle condition-guided mechanism, which includes static variable condition guidance, time-dependent condition guidance, and column-dependent condition guidance.

[0007] Preferably, the processing of missing values ​​in the original dynamic time series data includes: using linear interpolation for continuous missing segments, and filling discrete missing points with the mean of the corresponding physiological indicators of the patients.

[0008] Preferably, the row importance is calculated through the following steps: Suppose that for each patient, there are The original dynamic time-series data of the dimension, each time step The corresponding line importance is:

[0009] in, As for the importance of the line, For time steps, This represents the number of physiological indicators.

[0010] Preferably, S2 includes: Variational autoencoders include variational encoders and variational decoders. The variational encoder maps static variables to a low-dimensional latent space to obtain the mean vector and log-variance. Latent variables are obtained by sampling from the latent distribution using reparameterization techniques; The variational decoder reconstructs the latent variables into static variables; The total loss function of the variational autoencoder includes reconstruction loss and KL divergence.

[0011] Preferably, the reconstruction loss is as follows:

[0012] in, For cross-entropy loss, Mean square error, , , These are the weighting coefficients. The probability distribution for gender prediction. The probability distribution for predicting the outcome. This is a predicted value for age. , For the corresponding real tags, This corresponds to the actual value; The KL divergence is as follows:

[0013] in, Let be the dimension number of the potential space. For the first Standard deviation of each latent dimension For the first The mean of each potential dimension; The total loss function is as follows:

[0014] in, These are the weighting coefficients.

[0015] Preferably, the encoder for multi-dimensional information encoding includes a static variable encoder, a dynamic temporal encoder, and a feature fusion layer, with the static variable encoder and the dynamic temporal encoder arranged in parallel and both connected to the feature fusion layer. The static variable encoder is used to convert static variables into static conditional vectors through a self-attention mechanism and a deep network. The dynamic time-series encoder includes a time step encoding module, a variable type encoding module, a time-series value encoding module, and a row importance encoding module, which are used to encode the diffusion time step, physiological indicator type, time-series value, and row importance, respectively. The feature fusion layer is used to fuse the static condition vector and the output of the dynamic temporal encoder to generate a unified condition vector.

[0016] Preferably, the multi-angle condition-guided decoder includes a shared embedding layer, a unified cross-attention layer, and a fusion output layer. The shared embedding layer and the unified cross-attention layer are connected in sequence, and a static variable condition guidance module, a time-dependent condition guidance module, and a column-dependent condition guidance module are connected in parallel between the unified cross-attention layer and the fusion output layer. The shared embedding layer is used to perform time-aware normalization on noisy data to obtain shared embedding features; The unified cross-attention layer uses shared embedded features as queries and condition vectors output by an encoder that encodes multi-dimensional information as keys and values. Condition information is injected into the shared embedded features through attention computation. The static variable condition guidance module is used to integrate the static condition vector into the output of the unified cross-attention layer through a feature modulation mechanism; The time-dependent conditional guidance module is used to capture time dependencies through a bidirectional long short-term memory network and a conditionally guided temporal attention mechanism. The column dependency condition guidance module is used to model the dependency relationship between different physiological indicators within the same time step through a row importance-weighted multi-head self-attention mechanism. The fusion output layer is used to perform weighted fusion of the outputs of the static variable condition guidance module, the time-dependent condition guidance module, and the inter-column dependency condition guidance module, and obtain the denoised prediction value through linear mapping.

[0017] Preferably, the static variable condition guidance module achieves feature modulation through the following steps: Project the static conditions into a variable-specific modulation vector; For each variable, the modulation vector is decomposed into a scaling vector and an offset vector; The output features across the unified attention layer are modulated using scaling and offset vectors; The modulation intensity is dynamically adjusted through a global gating mechanism, and the modulated features are weighted and fused with the output features of a unified cross-attention layer. After fusing the static condition vector with the features through broadcasting and concatenation, it is concatenated with the weighted fused features in the last dimension to obtain the static condition guided branch output.

[0018] Preferably, in the column dependency guidance module, the row importance-weighted multi-head self-attention mechanism injects an additive bias matrix based on row importance when calculating the attention score. The additive bias matrix is ​​as follows:

[0019] in, As a time step-level importance scalar, It is a very small positive constant. It is a matrix of all 1s.

[0020] Preferably, the total loss of the improved diffusion model includes diffusion model reconstruction loss and auxiliary loss. The auxiliary loss includes at least one of temporal smoothing loss, physiological constraint loss, and missing pattern consistency loss, calculated as follows:

[0021] in, The total loss of the improved diffusion model, For the reconstruction loss of the diffusion model, For time-series smoothing loss, For physiological constraint loss, For the loss of consistency in missing patterns, , , These are the weighting coefficients for each type of loss.

[0022] The beneficial effects of this invention are: (1) By introducing a dedicated static variable generation module, high-quality static data generation and effective condition guidance were achieved. By introducing a dedicated static variable generation module, the joint distribution of static features was effectively learned and used as a strong condition to guide subsequent time series generation, making the generated data more consistent with the characteristics of a specific patient group.

[0023] (2) By proposing a column dependency guidance module based on row importance, the complex dependencies between physiological variables were accurately modeled. The concept of row importance was innovatively proposed, and a column dependency guidance module was constructed based on it to dynamically adjust the contribution of data with different integrity levels, ensuring that the generated physiological index values ​​meet medical constraints.

[0024] (3) By constructing a multi-angle condition-guided generation mechanism, the generation process is accurately controllable and the generated data is highly faithful. Through the three-way condition-guided mechanism of static, time and inter-column dependencies, the diffusion model generation process is finely controlled from multiple medical perspectives, which significantly improves the fidelity and consistency of the generated data.

[0025] (4) The credibility and practicality of the model are enhanced through interpretable modularization. Modularization corresponds to clear medical meanings, making the generation process interpretable and strongly supporting the practical application of generated data in medical research and clinical decision-making. Attached Figure Description

[0026] Figure 1This is a schematic diagram of the process for generating time-series electronic health record data based on a diffusion model, according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the encoder structure for multi-dimensional information encoding according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the decoder structure for multi-angle conditional guidance according to an embodiment of the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided with reference to the accompanying drawings and embodiments.

[0028] This application discloses a method for generating time-series data of electronic health records based on a diffusion model, the process of which is as follows: Figure 1 As shown, based on the diffusion model architecture, this paper addresses the problems in existing electronic health record (EHR) data generation, such as insufficient capture of the relationship between static variables and time-series data, neglect of column dependencies in time-series data, lack of a robust conditional guidance mechanism, and lack of interpretable design. The overall architecture follows the logic of static generation, conditional encoding, and conditional guided generation, forming a complete generation process. Specifically, it includes the following steps: S1. Data Preprocessing. Extract and standardize the patient's static variables and raw dynamic time-series data from the original electronic health record data, process missing values ​​in the raw dynamic time-series data, and calculate row importance.

[0029] This embodiment focuses primarily on the static variables and dynamic time-series data of patients in electronic health records. By extracting and standardizing multi-dimensional patient data from the original electronic health record data, it comprehensively describes the patient's static attributes and dynamic physiological indicators, and calculates the importance of each line.

[0030] Electronic health records contain multi-dimensional patient information. Static variables typically include variables that do not change over time or whose changes are not significant over time, such as gender and age. This embodiment expands the scope of static variables to include patient outcomes as well. Raw dynamic time-series data refers to time-series data generated by collecting multiple physiological variables during a patient's hospitalization at multiple time steps over a period of time.

[0031] Due to various reasons, actual electronic health record time-series data may be missing some data. To address the common problem of missing data in electronic health record data, a multi-strategy fusion imputation scheme is adopted. For continuous missing segments, linear interpolation is used to maintain the continuity of the time-series trend; for discrete missing points, the mean value of this indicator for the patient is used to fill in the missing data, ensuring that the filled data is more consistent with the patient's physiological condition.

[0032] During data preprocessing, row importance is calculated. Row importance is defined as follows: for each patient, there are... The original dynamic time-series data of the dimension. For time steps, Let be the number of physiological indicators, then for each time step... Define the row importance corresponding to this time step as:

[0033] in, This represents row importance. The calculated row importance will be used as weights in the column dependency guidance module of the subsequent decoder. Row Importance This reflects the completeness of the data in this row.

[0034] S2, Static variable generation. Based on Variational Autoencoder (VAE). A variational autoencoder (VAE) learns the joint distribution of static variables and generates a set of static variables that conforms to this joint distribution. A VAE consists of a variational encoder and a variational decoder, and its core challenge lies in capturing the complex correlations between static variables. For example, age and patient outcomes often have a non-linear relationship. VAEs can learn the probability distribution of data rather than a simple deterministic mapping, thus serving as a foundation. Through a variational inference process of "encoding-reparameterization-decoding," the joint distribution of static variables is captured, and static data that conforms to the actual joint distribution is generated.

[0035] The variational encoder is responsible for mapping real static variables to a low-dimensional latent space and learning the distribution parameters of the data. The input layer first enhances feature representation through an embedding mechanism. For categorical variables such as gender and outcome, the embedding layer maps them to a continuous vector space. For continuous variables such as age, normalization is performed first, followed by feature extraction through linear transformation and nonlinear activation functions. After fusion, all features are encoded as distribution parameters in the latent space, including mean and variance. Gender is mapped to an 8-dimensional vector using Embedding(2, 8), age is projected using Linear(1, 8) and activated by a Sigmoid function, and outcome is encoded using Embedding(2, 8). The three embedding vectors are concatenated to form a 24-dimensional joint feature.

[0036] The feature extraction network employs a two-layer fully connected structure and extracts hidden representations through nonlinear transformations.

[0037] in, and This is the weight matrix. The input is the concatenated static features. The bias direction of the first fully connected layer. This is the bias vector for the second fully connected layer. This is the activation function for the linear correction unit.

[0038] Hidden representations extracted by variational encoder Mean vectors are generated through two independent linear layers. Sum of logarithmic variance :

[0039]

[0040] in, The mean vector of the latent space. , The linear layer weight matrix, , This is the bias vector corresponding to the linear layer. Let be the standard deviation vector of the latent space.

[0041] The variational decoder is symmetrical to the variational encoder, but performs the opposite function. It starts from sampling points in the latent space and reconstructs the original static variables layer by layer. To ensure generation quality, each layer of the variational decoder is equipped with batch normalization and dropout mechanisms to enhance the model's generalization ability. The output layer is designed with appropriate activation functions for different types of variables to ensure the reasonableness of the generated values. The variational decoder converts latent variables... The data is reconstructed into static variables to generate specific values ​​from the distribution. The decoding network adopts a three-layer fully connected structure with progressively expanding dimensions (16-32-48-32), and enhances nonlinear expressive power through ReLU activation and batch normalization.

[0042] The output layer employs specialized prediction heads designed for different variable types. Gender prediction uses a 2D output activated by Softmax.

[0043] in, The probability distribution for gender prediction. The linear transformation weight matrix for the gender classification layer. The hidden representation obtained by the decoder. This is the linear transformation bias term for the gender classification layer.

[0044] Age prediction uses a one-dimensional output activated by a Sigmoid algorithm and scaled to [0, 100]:

[0045] in, This is a predicted value for age. The linear transformation weight matrix for the age prediction layer. The hidden representation obtained by the decoder. This is the linear transformation bias term for the age prediction layer.

[0046] Similar to gender prediction, the outcome prediction uses 2D Softmax output.

[0047] The training objective of VAE is to maximize the lower bound of evidence (ELBO), which is equivalent to minimizing the weighted sum of the reconstruction loss and the KL divergence. The reconstruction loss measures the difference between the generated variable and the true variable.

[0048] in, For cross-entropy loss, Mean square error, , , These are the weighting coefficients. The probability distribution for gender prediction. The probability distribution for predicting the outcome. This is a predicted value for age. , For the corresponding real tags, This corresponds to the actual value.

[0049] KL divergence regularization constraint: The latent distribution approximates the standard normal distribution.

[0050] in, Let be the dimension number of the potential space. For the first The standard deviation of each latent dimension For the first The mean of each potential dimension.

[0051] The total loss function is as follows:

[0052] in, These are the weighting coefficients.

[0053] S3. Conditional Diffusion Generation. The original dynamic time-series data is converted into noisy data through progressively added Gaussian noise diffusion, and then denoised using an improved diffusion model to generate synthetic dynamic time-series data. This step uses an improved diffusion model based on an encoder-decoder architecture, with improvements made to the encoder and decoder specifically tailored to the characteristics of electronic health record time-series data. At the encoder level, this embodiment introduces a multi-dimensional encoding mechanism, encoding information such as static variables, variable types, and time steps. At the decoder level, this embodiment constructs a multi-angle condition-guided network structure, including a static variable condition guidance module, a time-dependent condition guidance module, and an inter-column dependency condition guidance module.

[0054] The core idea of ​​the improved diffusion model in this embodiment is to transform the data distribution into a simple Gaussian distribution by gradually adding noise, and then achieve denoising and generation through an encoder-decoder structure. Compared with other generative models such as GANs, the diffusion model has advantages such as stable training, high generation quality, and solid theoretical foundation. In this embodiment, the diffusion process is applied to the generation of synthetic dynamic time-series data.

[0055] The forward diffusion process simulates the gradual contamination of data with noise. A small amount of Gaussian noise is added at each step, and after a sufficient number of steps, the original data is completely transformed into random noise. This process is predefined and does not require learning. The forward diffusion process transforms the original data by progressively adding Gaussian noise. Convert to pure noise Noise scheduling employs a linear scheme. It grows linearly from 0.0001 to 0.02. Definition ,in For the first The retention factor of the step, For the first The noise variance of each step. To facilitate the representation of cumulative noise intensity, a cumulative retention factor is introduced. The cumulative retention factor is defined as subscript This represents the intermediate index variable in a product. It will be used in subsequent formulas. Indicates the first The single-step retention coefficient of the step is used. Indicates the first The cumulative retention factor of the step.

[0056] Arbitrary time step The noisy data follows the following distribution:

[0057] in, The original data, For the first Noisy samples from each diffusion step It follows a Gaussian distribution. For the first The cumulative retention factor of the step, To and The identity matrix for dimension matching.

[0058] The improved diffusion model includes an encoder that encodes multi-dimensional information and a decoder that is guided by multi-angle conditions.

[0059] The encoder, which encodes multidimensional information, is responsible for converting static variables and other conditional information into a high-dimensional representation that can guide the generation process. Its structure is as follows: Figure 2 As shown, the architecture includes a static variable encoder, a dynamic temporal encoder, and a final feature fusion layer. The static variable encoder and the dynamic temporal encoder are parallel and both connected to the feature fusion layer. Conditional vectors are generated by multi-dimensional encoding and fusion of static variables, diffusion time steps, variable types, temporal values, and row importance. The static variable encoder focuses on encoding static variables. The dynamic temporal encoder encodes time steps, variables, values, and row importance. Finally, the multi-dimensional encoded information is passed through the feature fusion layer. The advantage of this parallel architecture of static variable encoder and dynamic temporal encoder is that it allows for the design of specialized encoding strategies for each data type while maintaining the independence between features.

[0060] A static variable encoder is used to transform static variables into static conditional vectors using a self-attention mechanism and a deep network. Specifically, it converts a patient's static variables into a conditional vector. This process is not a simple concatenation; instead, it uses a deep network to learn the interaction relationships between features. The network incorporates a self-attention mechanism, enabling different static variables to influence each other and capturing complex dependencies. The encoding process first encodes each static variable independently, and then generates a unified conditional representation through feature fusion.

[0061] Adaptive embedding strategies are used for different types of static variables:

[0062] in, static variable The vectorized representation of, ( ) is the embedding mapping function for categorical variables. For the first One static variable, For the embedding dimension of a single static variable, ( ) represents the mapping function of the multilayer perceptron. It is a binary feature embedding function.

[0063] Capture higher-order interactions between static variables:

[0064] in, The interaction features obtained through self-attention. For multi-head self-attention mechanism functions, This represents the total number of static variables. For the first An embedding vector of variables.

[0065] Self-attention mechanisms allow information exchange between different static variables, enabling the learning of complex dependencies.

[0066] Finally, the final static conditional vector is generated using a deep network:

[0067] in, This is a function for vector concatenation operations.

[0068] Output condition vector As global condition information.

[0069] Dynamic temporal encoders need to process information from multiple dimensions simultaneously: time dimension (time step), variable dimension (different physiological indicators), numerical dimension (actual measurement values), and quality dimension (row importance). To comprehensively capture this information, the dynamic encoder includes four dedicated encoding modules: a time step encoding module, a variable type encoding module, a temporal numerical encoding module, and a row importance encoding module. These modules encode the diffusion time step, physiological indicator type, temporal numerical value, and row importance, respectively.

[0070] For the time-step encoding module, the diffusion time step... The time step is a core element of the diffusion model, indicating the current noise level and denoising stage. The quality of the time step encoding directly affects the model's ability to handle different noise levels. Diffusion Time Step The encoding uses a Transformer-based sinusoidal positional encoding:

[0071]

[0072] in, This is the index for the diffusion step. For frequency dimension index, Embedding dimensions for time steps This is the time step encoding function.

[0073] Simple sine coding is not enough; it needs further processing via MLP to learn specific time representations.

[0074] in, This is the time step condition vector. For diffusion step Embedded.

[0075] For the variable type encoding module, medical time-series data contains various physiological indicators, each with its specific characteristics: normal range, variation pattern, clinical significance, etc. The goal of variable type encoding is to capture these characteristics, enabling the model to differentiate between different physiological indicators. Each different physiological indicator, each variable Each has its specific type embedding: Basic variable embeddings are implemented through a learnable embedding matrix:

[0076] in, An embedding mapping function for categorical variables. For variable index, It is a learnable embedding matrix.

[0077] Variable type coding can help models gain a deeper understanding of the characteristics of different physiological variables, thereby generating medically sound time-series data.

[0078] For the time-series numerical encoding module, since missing values ​​have already been handled in the data preprocessing stage, all time-series numerical values ​​are encoded directly here. For time-series numerical values, a projection function is used:

[0079] in, These are the preprocessed time series values. For the first Numerical projection function of variables.

[0080] For each physiological variable, the model is allowed to learn different numerical encoding methods.

[0081] For the row importance encoding module, the previously calculated row importance information is encoded as a conditional vector. This allows the model to consider differences in data quality during the generation process, giving more attention to high-quality time steps. Row importance weights To reflect the data quality at each time step, it needs to be encoded in a form that the network can utilize:

[0082] in, The importance of rows at each time step is listed in the column. For the embedding representation of row importance, It is a linear mapping function.

[0083] The introduction of row importance encoding enables the model to adaptively handle data of different qualities, learn accurate patterns on high-quality data, and remain robust on low-quality data.

[0084] The feature fusion layer is used to fuse the static conditional vector and the output of the dynamic temporal encoder to generate a unified conditional vector. After encoding by the static variable encoder and the dynamic temporal encoder, the feature fusion layer integrates the obtained encoded information. The fusion process first aligns the dimensions of the two features through linear projection:

[0085]

[0086] in, This is a static condition for projecting onto the alignment dimension. The weight matrix is ​​the static conditional linear transformation. It is the bias vector of the static conditional linear transformation. This is the time step condition vector projected onto the alignment dimension. The weight matrix is ​​the linear transformation matrix of the time-step conditional vector. It is the bias vector for the linear transformation of the time-step conditional vector.

[0087] Then, based on the attention mechanism, a weighted fusion of dynamic features under static conditions is calculated to obtain the conditional vector after attention fusion. :

[0088] in, This is the attention transformation function.

[0089] Finally, to further integrate multi-source feature information, a feature fusion function is used to obtain the final fused representation. :

[0090] in, For feature fusion function, The static condition for projection onto the alignment dimension. This is the time step condition vector projected onto the alignment dimension. This is the conditional vector after attention fusion. Embed the sequence for time steps. These are the projected time series values. An embedded representation of row importance.

[0091] The multi-perspective condition-guided decoder utilizes condition vectors to denoise noisy data through a multi-perspective condition-guided mechanism. This mechanism includes static variable condition guidance, time-dependent condition guidance, and column-dependent condition guidance.

[0092] The multi-dimensional condition-guided decoder is the core of the entire generative model (improved diffusion model), responsible for converting encoded conditional information and noisy data into high-quality electronic health record time-series data during the generation phase. Traditional decoders typically employ deep networks with a single path, where all information processing occurs sequentially along the same path. The problem is that different types of information processing needs are mixed together, making targeted optimization difficult. This embodiment uses three parallel condition-guided modules to process different conditional dimensions, improving both computational efficiency and model interpretability. The specific structure is as follows: Figure 3 As shown, it includes a shared embedding layer, a unified cross-attention layer, and a fusion output layer. The shared embedding layer and the unified cross-attention layer are connected in sequence. The unified cross-attention layer and the fusion output layer are connected in parallel by a static variable condition guidance module, a time-dependent condition guidance module, and a column-dependent condition guidance module.

[0093] The shared embedding layer is used to perform time-aware normalization on noisy data to obtain shared embedding features.

[0094] The shared embedding layer first processes noisy data. Standardization process:

[0095] in, For the first Noisy samples from each diffusion step and It is based on time steps Predictive statistics To prevent small constants from being divided by zero. This is a standardized noise sample.

[0096] The standardized data is then input into the embedding layer to obtain shared embedding features. :

[0097] in, For learnable embedding functions.

[0098] This time-aware normalization ensures numerical stability under different noise levels. The output of the shared embedding layer is the shared embedding feature. .

[0099] A unified cross-attention layer uses shared embedded features as queries, and conditional vectors output by an encoder encoding multi-dimensional information as keys and values. Conditional information is injected into the shared embedded features through attention computation. That is, using... For querying, inject the encoder output into In the middle, we get . This is shared across the subsequent three conditional guidance modules.

[0100] First, it is obtained through linear projection. / / (Query / Key / Value):

[0101] in, , , All are projection matrices.

[0102] Then, the attention weights are calculated using the scaled dot product:

[0103]

[0104] in, For masking. for Dimensions. It is a normalized exponential function.

[0105] Finally, output your attention. The projection back to the original dimension is then added to the input residual, resulting in an output infused with encoder information.

[0106] in, To inject features after the encoder context, Scalar gating factor This is a random deactivation operation.

[0107] The static variable condition guidance module integrates static conditional vectors into the unified cross-attention layer output through a feature modulation mechanism. It is responsible for deeply incorporating patients' static characteristics (such as age, gender, and patient outcomes) into the generation of synthetic dynamic time-series data. This module, through conditional modulation, enables the generated time-series data to reflect the physiological differences among different patient groups.

[0108] The static condition vector is first projected onto the variable-specific modulation vector:

[0109] in, For each variable, there is a set of modulation parameters (scaling / offset). The hyperbolic tangent activation function is used. It is a linear transformation function. The number of physiological indicators. Modulation dimension for each variable.

[0110] For each variable, the modulation vector is decomposed into scaling and offset vectors:

[0111] in, For the first A scaling vector of variables, For the first An offset vector of variables, Modulation dimension for each variable.

[0112] Feature modulation process:

[0113] in, These are the features after static condition modulation. This indicates element-wise multiplication.

[0114] The global gating mechanism further adjusts the modulation intensity:

[0115]

[0116] in, This is the global gating coefficient. For the Sigmoid function, These are features adjusted by a global gating mechanism.

[0117] Static conditional vectors are fused with features through broadcasting and concatenation:

[0118] in, This is the static condition vector after broadcasting. For tensor copy function, is the dimension of the static vector.

[0119] Finally, the static condition guided branch output is obtained. :

[0120] in, This is for splicing operations on the last dimension.

[0121] The time-dependent conditional guidance module is used to capture time dependencies through a bidirectional long short-term memory network and a conditionally guided temporal attention mechanism.

[0122] Medical time-series data exhibits strong time dependence—current physiological indicators are often influenced by historical conditions and may also predict future trends. This module captures these complex time dependencies through bidirectional time-series modeling and conditionally guided attention mechanisms.

[0123] Bidirectional time series modeling employs forward and backward LSTM:

[0124]

[0125] in, Forward hidden state sequence, This represents the forward long short-term memory network. This is a sequence of backward hidden states. It is a backward long short-term memory network. For along the time dimension Reverse the sequence.

[0126] Bidirectional feature fusion:

[0127] in, This represents the temporal features after bidirectional fusion. It is a linear transformation function. This is a splicing operation along the last dimension.

[0128] In the condition-guided temporal attention mechanism, the calculation of query, key, and value is as follows:

[0129]

[0130]

[0131] in, , , Used to generate independent linear transformations of queries, keys, and values. For real time step Position embedding, To embed the location into a mapping with Same dimension.

[0132] Attention score calculation includes time decay weights:

[0133] in, Attention scoring matrix elements, For the current time step, To calculate the time step of attention during attention calculation, This is the time decay coefficient.

[0134] Causal masking and softmax are applied during training:

[0135] in, Indicates masking operation. For attention scoring matrix, This is the final attention weight matrix.

[0136] Final temporal feature output :

[0137] in, Output for time-dependent condition-guided modules.

[0138] The inter-column dependency guidance module is used to model the dependencies between different physiological indicators within the same time step using a row importance-weighted multi-head self-attention mechanism. This module focuses on modeling the interdependencies between different physiological indicators. In medical scenarios, physiological indicators do not change independently but rather have complex interactions. For example, changes in blood pressure may affect heart rate, and an increase in body temperature may be accompanied by changes in respiratory rate. This module learns the interaction patterns between these variables through a Transformer architecture and a weighted attention mechanism. The implementation of this module introduces row importance weights, which can dynamically adjust the influence of different variables based on data quality. High-quality data (rows with high completeness) receives greater weight in interaction modeling, while the influence of low-quality data is appropriately reduced, which improves the model's robustness to noise and missing data.

[0139] Given the previously calculated row importance encoding, and then after feature modulation gating, we have:

[0140] in, The input features are after merging row importance. For the contextual features of the input, For variable-level modulation coefficients, This indicates that the importance of row encoding will be expanded to include variable dimensions.

[0141] In multi-head self-attention computation, the relationships between variables are processed independently at each time step. Extract variable dimensions input Then we have:

[0142]

[0143]

[0144] in, , , For querying, key, and value for attention between variables, Map row importance to the same dimension as the key. Encode the row importance for this time step.

[0145] To enhance the prior weights of high-quality variables, an additive partial matrix is ​​introduced. Defined as:

[0146] in, As a time step-level importance scalar, It is a very small positive constant. It is a matrix of all 1s.

[0147] Add this bias to the standard scaled dot product score at a single-head scale, then normalize and aggregate:

[0148]

[0149]

[0150] in, Scoring attention between variables The attention weights are normalized. For the context representation between variables, For attention, it is a single-head dimension.

[0151] Representation after attention convergence We then proceed with the variable-independent feedforward transform (FFN). To avoid excessive nonlinear amplification of low-quality variables, a variable-dependent transform is introduced. gating vector This is used for weighted interpolation between "identity preservation" and "feedforward transformation": variables with higher importance receive stronger nonlinear rewriting weights, while variables with lower importance tend to be more conservative.

[0152]

[0153] in, The gating coefficient, For the Sigmoid function, , These are learnable affine parameters. For gated fusion output, It is a feedforward network.

[0154] Finally, the gated results of all time steps are stacked back to their original shape in the time dimension, and a residual connection is made with the input to obtain the module output:

[0155] in, The output of the module is to guide the inter-column dependency conditions. This is a splicing operation along the time dimension.

[0156] The fusion output layer is used to perform weighted fusion of the outputs of the static variable condition guidance module, the time-dependent condition guidance module, and the inter-column dependency condition guidance module, and map them to obtain denoised prediction values.

[0157] The fusion output layer is responsible for integrating the features from the three conditional guidance modules to generate the final denoised prediction. First, channel normalization is performed on each input feature to enhance numerical stability and eliminate dimensional differences.

[0158] in, The features are normalized. This is a layer normalization operation.

[0159] To achieve adaptive fusion of multi-branch features, global statistical features (global mean aggregation) are first calculated for each branch:

[0160] in, For along Global mean convergence of dimensions.

[0161] Then, the fusion weights are calculated by combining static conditions with temporal context embedding:

[0162] in, For the weight of the three branches, , To fuse the parameters of the weighted linear layer, It is a static conditional mapping function. Embed the mapping function for the time step. This indicates that the output is normalized to a weighted distribution.

[0163] The three features are summed in a weighted manner, and a shared residual term is introduced to preserve the original contextual information.

[0164] in, This represents the joint feature representation after fusion. This is the learnable residual scaling factor.

[0165] After further normalization and channel compression of the fused features, the final denoised prediction is generated:

[0166] in, For linear mapping layer, Output for the model.

[0167] The total loss of the improved diffusion model includes diffusion model reconstruction loss and auxiliary loss. The auxiliary loss includes at least one of temporal smoothing loss, physiological constraint loss and missing pattern consistency loss.

[0168] The reconstruction loss of the diffusion model uses a simplified objective loss function:

[0169] in, These are real, noise-free time-series samples. The denoising result predicted by the model. In order to spread the first Step-by-step noisy samples, These are the model conditions, including static features, variable types, diffusion step embeddings, etc.

[0170] Temporal smoothing loss is used to encourage continuity between adjacent time steps:

[0171] in, For the prediction matrix The first in Each element.

[0172] Physiological constraint loss is used to ensure that the generated values ​​are within a reasonable range:

[0173] in, For the first The lower bound of each variable. For the first The upper bound of each variable.

[0174] Missing pattern consistency loss, maintaining the authenticity of missing patterns in generated data:

[0175] in, To generate a missing mask for the data, For true missing pattern, It is a binary cross-entropy function.

[0176] The formula for calculating total loss is as follows:

[0177] in, The total loss of the improved diffusion model, For the reconstruction loss of the diffusion model, For time-series smoothing loss, For physiological constraint loss, For the loss of consistency in missing patterns, , , These are the weighting coefficients for each type of loss.

[0178] In summary, existing technologies often input static and time-series data together into models more suited for generating time-series data. This embodiment proposes an auxiliary module, a static variable generation module, specifically designed to address the generation of static variables and prepare for using these generated static variables as guiding conditions in subsequent dynamic data generation. This embodiment expands the concept of static variables to include patient outcomes within its scope. Furthermore, this embodiment proposes a static variable generation method based on a variational autoencoder architecture, learning the joint distribution of static variables and generating a set of static variables that conforms to the learned joint distribution, making the generated static variables more realistic. This solves the problem of not fully capturing the relationship between static variables and time-series data in electronic health records.

[0179] Secondly, data in electronic health records often consists of measurements of multiple physiological variables at multiple time steps. In such time-series data, individual variables have time dependencies, and different physiological variables have inter-column dependencies at the same time step. Existing technologies have indeed attempted to capture time dependencies, but lack dependencies on inter-column relationships. This embodiment proposes a dedicated inter-column dependency condition guidance module to capture inter-column relationships. Furthermore, to improve the effectiveness of the inter-column dependency condition guidance module, this embodiment introduces the concept of row importance. Because electronic health time-series data often has missing data, for a patient at a given time step, not every physiological variable may be recorded, and rows with more complete records should be studied more extensively. Therefore, this embodiment defines row importance as the number of physiological indicators recorded at that time step divided by the total number of physiological indicators, calculating row importance for each row, and using row importance as the weight in the inter-column dependency condition guidance module. This solves the problem of ignoring inter-column dependencies in time-series data.

[0180] Furthermore, existing technologies often generate data based on computer-centric models or the technology itself, while electronic health record time-series data requires a more sophisticated conditional guidance mechanism. This embodiment proposes a multi-faceted conditional guidance generation mechanism that integrates static variables, time dependencies, and inter-column dependencies, better reflecting medical knowledge and the realities of electronic health record time-series data. This mechanism is integrated into the data generation process to guide the model in generating higher-quality electronic health record time-series data. This addresses the lack of a robust conditional guidance mechanism in diffusion-based model-based data generation.

[0181] Finally, existing technologies often rely on mathematical principles for generation, neglecting the medical interpretation required for time-series data in electronic health records. The static variable condition guidance module, time-dependent condition guidance module, and column-dependent condition guidance module proposed in this embodiment are based on certain medical knowledge and principles, resulting in a model with strong interpretability. This solves the problem of lacking interpretable design.

[0182] The method proposed in this embodiment is mainly applied to generating time-series electronic health record data generated during a patient's medical treatment, achieving the goals of medical data enhancement and privacy protection. This method is primarily based on a diffusion model, integrating the patient's static characteristics (gender, age, patient outcome, etc.), the changes in multiple dynamic physiological indicators over time, and the dependencies between these physiological indicators to generate privacy-preserving and valuable time-series electronic health record data.

[0183] Its applications cover the entire AI+healthcare field based on electronic health record time-series data, such as patient outcome prediction and hospitalization duration prediction. In data augmentation of electronic health record time-series data, this method can generate high-quality synthetic samples for data that is scarce, difficult to obtain, or extremely imbalanced, expanding the training dataset and improving model generalization performance. For example, in rare disease research, because rare diseases have extremely low incidence rates, rare disease cases account for a very small proportion of the dataset, hindering the application of machine learning or deep learning models. The method proposed in this embodiment can be used to generate rare disease case data for data augmentation. Regarding privacy protection, this method can generate synthetic data without the risk of privacy breaches, does not trace back to patients, poses no ethical risks, and can promote the development of medical research.

[0184] Furthermore, this method can be extended to the interpolation, prediction, and classification of electronic health record time-series data, providing strong support for the development of medical + AI research based on electronic health record time-series data.

[0185] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A method for generating time-series electronic health record data based on a diffusion model, characterized in that, Includes the following steps: S1. Extract and standardize the patient's static variables and original dynamic time-series data from the original electronic health record data, process the missing values ​​in the original dynamic time-series data, and calculate the row importance. S2. Learn the joint distribution of the static variables based on the variational autoencoder, and generate a set of static variables that conforms to the joint distribution; S3. The original dynamic time series data is converted into noise data by gradually adding Gaussian noise for diffusion and noise addition, and the noise data is denoised based on the improved diffusion model to generate synthetic dynamic time series data. The improved diffusion model includes an encoder that encodes multi-dimensional information and a decoder that guides multi-angle conditions. The encoder for multi-dimensional information encoding encodes static variables, diffusion time steps, variable types, time series values, and row importance in multiple dimensions and fuses them to generate a conditional vector. The multi-angle condition-guided decoder uses condition vectors to denoise noisy data through a multi-angle condition-guided mechanism, which includes static variable condition guidance, time-dependent condition guidance, and column-dependent condition guidance.

2. The method for generating time-series electronic health record data based on a diffusion model according to claim 1, characterized in that, The processing of missing values ​​in the original dynamic time series data includes: using linear interpolation for continuous missing segments, and filling discrete missing points with the mean of the corresponding physiological indicators of the patients.

3. The method for generating time-series electronic health record data based on a diffusion model according to claim 2, characterized in that, The row importance is calculated through the following steps: Suppose that for each patient, there are The original dynamic time-series data of the dimension, each time step The corresponding line importance is: in, As for the importance of the line, For time steps, This represents the number of physiological indicators.

4. The method for generating time-series electronic health record data based on a diffusion model according to claim 3, characterized in that, S2 includes: Variational autoencoders include variational encoders and variational decoders. The variational encoder maps static variables to a low-dimensional latent space to obtain the mean vector and log-variance. Latent variables are obtained by sampling from the latent distribution using reparameterization techniques; The variational decoder reconstructs the latent variables into static variables; The total loss function of the variational autoencoder includes reconstruction loss and KL divergence.

5. The method for generating time-series electronic health record data based on a diffusion model according to claim 4, characterized in that, The reconstruction loss is as follows: in, For cross-entropy loss, Mean square error, , , These are the weighting coefficients. The probability distribution for gender prediction. The probability distribution for predicting the outcome. This is a predicted value for age. , For the corresponding real tags, This corresponds to the actual value; The KL divergence is as follows: in, Let be the dimension number of the potential space. For the first Standard deviation of each latent dimension For the first The mean of each potential dimension; The total loss function is as follows: in, These are the weighting coefficients.

6. The method for generating time-series electronic health record data based on a diffusion model according to claim 5, characterized in that, The encoder for the multi-dimensional information encoding includes a static variable encoder, a dynamic temporal encoder, and a feature fusion layer. The static variable encoder and the dynamic temporal encoder are arranged in parallel and are both connected to the feature fusion layer. The static variable encoder is used to convert static variables into static conditional vectors through a self-attention mechanism and a deep network. The dynamic time-series encoder includes a time step encoding module, a variable type encoding module, a time-series value encoding module, and a row importance encoding module, which are used to encode the diffusion time step, physiological indicator type, time-series value, and row importance, respectively. The feature fusion layer is used to fuse the static condition vector and the output of the dynamic temporal encoder to generate a unified condition vector.

7. The method for generating time-series electronic health record data based on a diffusion model according to claim 6, characterized in that, The multi-angle condition-guided decoder includes a shared embedding layer, a unified cross-attention layer, and a fusion output layer. The shared embedding layer and the unified cross-attention layer are connected in sequence. A static variable condition guidance module, a time-dependent condition guidance module, and a column-dependent condition guidance module are connected in parallel between the unified cross-attention layer and the fusion output layer. The shared embedding layer is used to perform time-aware normalization on noisy data to obtain shared embedding features; The unified cross-attention layer uses shared embedded features as queries and condition vectors output by an encoder that encodes multi-dimensional information as keys and values. Condition information is injected into the shared embedded features through attention computation. The static variable condition guidance module is used to integrate the static condition vector into the output of the unified cross-attention layer through a feature modulation mechanism; The time-dependent conditional guidance module is used to capture time dependencies through a bidirectional long short-term memory network and a conditionally guided temporal attention mechanism. The column dependency condition guidance module is used to model the dependency relationship between different physiological indicators within the same time step through a row importance-weighted multi-head self-attention mechanism. The fusion output layer is used to perform weighted fusion of the outputs of the static variable condition guidance module, the time-dependent condition guidance module, and the inter-column dependency condition guidance module, and obtain the denoised prediction value through linear mapping.

8. The method for generating time-series electronic health record data based on a diffusion model according to claim 7, characterized in that, The static variable condition guidance module achieves feature modulation through the following steps: Project the static conditions into a variable-specific modulation vector; For each variable, the modulation vector is decomposed into a scaling vector and an offset vector; The output features across the unified attention layer are modulated using scaling and offset vectors; The modulation intensity is dynamically adjusted through a global gating mechanism, and the modulated features are weighted and fused with the output features of a unified cross-attention layer. After fusing the static condition vector with the features through broadcasting and concatenation, it is concatenated with the weighted fused features in the last dimension to obtain the static condition guided branch output.

9. The method for generating time-series electronic health record data based on a diffusion model according to claim 8, characterized in that, In the column dependency guidance module, the row importance-weighted multi-head self-attention mechanism injects an additive bias matrix based on row importance when calculating the attention score. The additive bias matrix is ​​as follows: in, As a time step-level importance scalar, It is a very small positive constant. It is a matrix of all 1s.

10. The method for generating time-series electronic health record data based on a diffusion model according to claim 9, characterized in that, The total loss of the improved diffusion model includes diffusion model reconstruction loss and auxiliary loss. The auxiliary loss includes at least one of temporal smoothing loss, physiological constraint loss, and missing pattern consistency loss, and the calculation formula is as follows: in, The total loss of the improved diffusion model, For the reconstruction loss of the diffusion model, For time-series smoothing loss, For physiological constraint loss, For the loss of consistency in missing patterns, , , These are the weighting coefficients for each type of loss.

Images