Controllable condition clinical data synthesis method and device supporting logical relationship perception

Abstract

The invention relates to the technical field of clinical data synthesis, provides a controllable condition clinical data synthesis method and device supporting logic relation perception, and aims to solve the problem of logic inconsistency in an existing synthesis method and flexibly generate data meeting user-defined conditions. The method comprises two key stages: a logical relationship perception stage: capturing inherent semantic information in original data, and learning a complex logical relationship between clinical data columns; and a conditional diffusion model generation stage: fitting data distribution characteristics by using a diffusion model, and integrating conditional semantic embedding to realize controllable clinical data synthesis. Through mutual cooperation of the two stages, a difficult electronic clinical data generation task is decomposed into two relatively simple problems, and the problems of logic inconsistency and lack of controllability are efficiently solved. According to the scheme, the quality and availability of the synthesized data are greatly improved, and a set of high-quality data synthesis scheme is provided for the medical field.

Description

Technical Field

[0001] The present invention relates to the technical field of clinical data synthesis, and in particular to a controllable conditional electronic clinical data synthesis method and device based on a diffusion model with logical relationship perception capability. Background Art

[0002] Electronic clinical data is an important carrier of patient health information, usually presented in a structured table format consisting of columns and rows. Columns represent various attributes of the domain, such as the patient's gender, age, medication records, and various laboratory indicators, while rows represent specific instances of these attributes. For example, intensive care unit patient data (MIMIC-III), human cancer clinical data (TCGA), and clinical data involving prediabetes and obesity. This digital clinical data can be used to train machine learning models, promoting the application of artificial intelligence technology in precision medicine, personalized risk prediction, and health trajectory analysis.

[0003] Electronic clinical data synthesis technology is crucial to the medical field. First, most clinical data contain sensitive information and cannot be made public due to privacy protection and social ethical principles. Electronic clinical data synthesis can solve the problem of privacy protection by generating high-quality virtual data without sensitive information to replace the real data that cannot be made public. Secondly, due to the difficulty of clinical data collection, there are usually missing values ​​and noise in electronic clinical data. This technology can be used to complete missing data and improve the quality of the data set. In addition, in some diseases or specific treatments, the available real electronic clinical data may be very limited. Synthetic data can expand these scarce data sets and improve the effect and generalization ability of model training. Therefore, the current medical field has an increasingly urgent demand for high-quality electronic clinical data synthesis technology, and its development is of great significance to promoting scientific and technological progress in the field of medical health.

[0004] In recent years, the application of deep learning models to the task of generating electronic clinical data has significantly improved the quality of synthetic data, and electronic clinical data synthesis schemes based on deep generative models have gradually become the mainstream scheme in this field. However, the difficulty in using deep generative models is that electronic clinical data usually contains mixed heterogeneous data types, including continuous variables (such as numerical data) and discrete variables (such as categorical data), which makes it difficult for deep learning models to learn the joint probability between columns. The standard input format of most deep learning models is continuous data, which poses a major challenge to processing discrete data in clinical data. For the above problems, a common solution is to use digital encoding techniques to convert discrete data into digital form to solve this problem, such as one-hot encoding or analog bit encoding. Some other methods use different models to process numerical features and categorical features respectively, or try to embed the original data into a unified high-dimensional space.

[0005] However, the above solution may lead to logical inconsistency. Logical inconsistency refers to the fact that the synthesized clinical data contains data that is logically inconsistent with the real data. For example, the above solution may synthesize a sample that is "never married" but the marital relationship is incorrectly marked as "husband". This contradictory relationship does not exist in the original data set, and this phenomenon is called logical inconsistency. In the field of medicine, this problem is particularly important. In addition to marital status, there is a high degree of dependence and logical relationship between various vital indicators in clinical data. Even a small logical inconsistency may lead to very serious consequences. Existing synthesis methods often require a large amount of preprocessing operations on the original data, resulting in serious information loss, thereby limiting the model's ability to capture the underlying semantic relationship and making it difficult for the model to learn the association relationship between columns. In addition, current research on clinical data synthesis mainly focuses on improving the performance of downstream classification models or enhancing the effect of privacy protection by synthesizing data, but ignores the importance of maintaining consistency between data features.

[0006] In addition, most current clinical data synthesis technologies do not support conditional generation, or only use a single feature as a simple conditional input, lacking effective control over the data generation process. However, in reality, it is often necessary to flexibly synthesize data that meets a variety of user-defined condition combinations. For example, when solving problems such as data imbalance and missing data, only data that meets the specified conditions needs to be synthesized. Therefore, the function of supporting controllable conditional synthesis is crucial. Summary of the invention

[0007] In order to solve the existing technical problems, the present invention proposes a controllable conditional electronic clinical data synthesis method and device based on a diffusion model with logical relationship perception ability to solve the problem of logical inconsistency in the synthesized data and enhance the conditional controllability of the data synthesis process.

[0008] In a first aspect of the present invention, a controllable conditional clinical data synthesis method supporting logical relationship perception is provided, comprising the following steps: Step S1, preprocessing the digital variable data and the categorical variable data; Step S2, constructing a logical relationship perception model, and using the preprocessed data to train the logical relationship perception model, and using the trained model to extract potential variables in the data; The processing process of the logical relationship perception model is as follows: first, the semantic features in the data are encoded into the original D-dimensional semantic vector representation using the tokenization module, then the latent variables rich in semantic logical relations are obtained using the variational autoencoder structure in the logical perception module, then the latent variables are decoded to generate a reconstructed D-dimensional vector, and the difference between the reconstructed D-dimensional vector and the original D-dimensional semantic vector is minimized, and finally the D-dimensional reconstructed vector is converted back to the form of data using the de-tokenization module; where D is a hyperparameter; Step S3, constructing a conditional diffusion model, and using the latent variables extracted in step S2 to train the conditional diffusion model; Step S4, using the trained conditional diffusion model, given any set of conditional inputs, starting from randomly sampled Gaussian noise, synthesizing data that meets the specified conditions; In step S5, the synthesized data is decoded from the latent space back to the real space, and the synthesized electronic clinical data is converted into a readable and usable form, completing the entire synthesis process.

[0009] Furthermore, the data preprocessing in step S1 is specifically as follows: for numerical variable data, a nonlinear quantile transformation function is used to achieve data normalization; for categorical variable data, a text operation is used to convert it from the original classification form to a text format.

[0010] Furthermore, in step S2, a differentiated processing strategy is adopted for the preprocessed digital variable data and categorical variable data to adapt to the characteristics of different data types. The processing process of the tokenization module is expressed by the following formula: ,

[0011] in, Represents the linear mapping layer, for each text-based categorical variable data , use the pre-trained BERT model to extract the semantic features of the text, and finally use a linear structure to encode the semantic features into a D-dimensional vector; for the normalized digital variable data , using two trainable parameters and Encode the normalized features into a unified D-dimensional vector space. and That is, the D-dimensional classification type and numerical semantic vector after tokenization. and They are the indexes of numeric variable data and categorical variable data respectively.

[0012] Furthermore, the de-tokenization module forms a corresponding relationship with the tokenization module, and the processing process is expressed by the following formula: ,

[0013] in, , , and are all learnable parameters. and They are D-dimensional reconstructed numerical vectors and categorical vectors, and are the restored numerical vectors and categorical vectors respectively; for digital variable data, a linear decoder is used to convert the vector Restore to ; For categorical variable data, after using a linear transformation, a Softmax structure is used to obtain the probabilities of all candidate values, and finally the candidate value with the largest probability is selected.

[0014] Furthermore, the variational autoencoder VAE is based on Transformer -VAE structure, consisting of an encoder and a decoder. The encoder accepts a unified original D-dimensional semantic vector representation as input and outputs two vectors: mean and variance , these two output vectors together define a Gaussian distribution ,in represents the latent variable, and the Gaussian distribution represents the given input data When the latent variable The decoder samples and generates data from this probability distribution; The encoder consists of two sets of Transformer structures, one for generating the mean , and the other group is responsible for generating the log variance , then, by the reparameterization technique, a latent variable is sampled from the Gaussian distribution , by introducing a standard normally distributed random variable , to achieve Sampling in process, i.e. Each Transformer structure contains a self-attention layer and a feedforward layer. The feedforward layer consists of two linear layers and a ReLU activation layer. The decoder is also an N-layer Transformer, matching the structure of the encoder. Its task is to transform the latent variables Mapping from the latent space back to the original data space to produce a reconstructed D-dimensional vector.

[0015] Furthermore, the overall loss function of the logical relationship perception model is:

[0016] Among them, the specific form of KL divergence is:

[0017] The reconstruction loss It consists of the following two parts:

[0018] Reconstruction loss Quantified reconstruction data and the original data For categorical variable data, the cross entropy loss function is used to measure the difference between the predicted value and the actual value, where Represents the original categorical variable data, Represents the reconstructed categorical variable data; for numerical variable data, the mean square error loss function is used To evaluate the deviation between the predicted value and the actual value, Represents the original numeric variable data, Represents the reconstructed numeric variable data; Divergence indicates latent variables The posterior distribution of With its prior distribution The KL divergence between Used to balance the reconstruction loss and The relative importance of divergences.

[0019] Furthermore, the training process of the conditional diffusion model in step S3 includes a forward denoising process and a reverse denoising process. The forward denoising process is performed at each time step. In the process of de-noising, predefined Gaussian noise is gradually added to the initial data, i.e., the latent variables obtained in step 2, gradually converting the data into noise; in the reverse denoising process, the original data is restored from the noise, thereby realizing data generation.

[0020] Furthermore, the inverse denoising process in the conditional diffusion model is implemented through a noise prediction network, and the input includes Noise data at time , the current time step , and control conditions , the output is the predicted noise The first layer of the noise prediction network is an additive embedding layer, whose function is to embed the time step information into the input data using the sinusoidal position encoding technique; following the additive embedding layer is a multi-head cross attention layer, and the specific structure of the cross attention layer is as follows: ,in,

[0021] in, represents the output of the cross attention layer, , , Represents query, key and value respectively; Responsible for embedding conditional semantic information , and Receive the intermediate data output by the embedding layer , does not participate in the introduction of conditional information, but through The interaction of can realize the fusion of conditional information; , , is a learnable weight matrix used to transform conditional information and intermediate data Map to the corresponding , , space, yes , , The dimension of is the function used to calculate the attention weight; Control conditions It is calculated by the feature combination module, which is used to extract the semantic features of any combination of conditions defined by the user. The calculation formula is as follows:

[0022] in, is the D-dimensional semantic vector obtained by the tokenization module for the categorical variable, is the index of the categorical variable, express Any subset of is the number of categorical variables, from which we can see that express The sum of the elements of any subset of , that is, the combination of the semantic features of any number of categorical variables.

[0023] Furthermore, the training objective function of the noise prediction network is as follows:

[0024] in, represents the standard normal distribution, represents the expectation, and this objective function minimizes the actual noise and prediction noise The difference between them can achieve the goal of noise prediction.

[0025] The present invention also provides a controllable conditional clinical data synthesis device supporting logical relationship perception, comprising the following units: A data preprocessing unit, used for preprocessing digital variable data and categorical variable data; A logical relationship perception model construction and training unit is used to construct a logical relationship perception model, train the logical relationship perception model using preprocessed data, and extract potential variables in the data using the trained model; The processing process of the logical relationship perception model is as follows: first, the semantic features in the data are encoded into the original D-dimensional semantic vector representation using the tokenization module, then the latent variables rich in semantic logical relations are obtained using the variational autoencoder structure in the logical perception module, then the latent variables are decoded to generate a reconstructed D-dimensional vector, and the difference between the reconstructed D-dimensional vector and the original D-dimensional semantic vector is minimized, and finally the D-dimensional reconstructed vector is converted back to the form of data using the de-tokenization module; where D is a hyperparameter; A conditional diffusion model construction and training unit, used to construct a conditional diffusion model and train the conditional diffusion model using the extracted latent variables; A controllable data synthesis unit is used to use the trained conditional diffusion model to synthesize data that meets the specified conditions from randomly sampled Gaussian noise given any set of conditional inputs; The data decoding and restoration unit is used to decode the synthesized data from the latent space back to the real space, convert the synthesized electronic clinical data into a readable and usable form, and complete the entire synthesis process.

[0026] The beneficial benefits brought by the implementation of the technical solution provided by the present invention are: 1. The present invention proposes to use a logical relationship perception model based on a pre-trained BERT model and a VAE structure, which can not only solve the problem of lossy preprocessing in previous solutions, but also efficiently capture and learn the semantic logical relationships inherent in the original data, and overcome the logical inconsistency problem in electronic clinical data synthesis technology, thereby greatly improving the quality and availability of synthesized data, and providing a set of high-quality data synthesis solutions for the medical field.

[0027] 2. This paper proposes a controllable conditional generation mechanism based on a diffusion model designed specifically for clinical data, which can cleverly integrate the embedding of multiple semantic conditions and flexibly generate data that meets multiple user-defined condition combinations. This mechanism can be widely used in specified class generation tasks and missing data completion tasks, overcoming the problem of the lack of controllability of previous data synthesis methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for describing the embodiments are briefly introduced below.

[0029] Figure 1 It is a flow chart of the controllable conditional electronic clinical data synthesis method with logical relationship perception capability proposed by the present invention.

[0030] Figure 2 is a schematic diagram of the structure of a logical relationship perception model in an embodiment of the present invention; Figure 3 Schematic diagram of the model structure of the variational autoencoder in an embodiment of the present invention; Figure 4 is a flow chart of the working process of the diffusion model in an embodiment of the present invention; Figure 5 Schematic diagram of the model structure of the noise prediction network in the embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to explain the technical solution, model structure, realization target and realization effect of the present invention in detail, the implementation method will be described in detail with reference to the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0032] The embodiment of the present invention provides a controllable conditional medical electronic clinical data synthesis method based on a diffusion model with logical relationship perception capability. The method includes two key stages: a logical relationship perception stage: this stage is responsible for extracting semantic feature vectors from raw data and mapping them to a high-dimensional latent space, so as to capture and utilize the inherent semantic information in the raw data, so that the model can learn the logical relationship between columns; a conditional diffusion model generation stage: this stage uses a diffusion model to fit the distribution characteristics of the latent variables extracted in the previous stage, and cleverly integrates conditional semantic embedding, aiming to achieve flexible multi-condition control generation tasks. Through the mutual cooperation of these two stages, the difficult electronic clinical data generation task is decomposed into two relatively simple problems, and the problems of logical inconsistency and lack of controllability are efficiently solved. Among them, the logical relationship perception stage mainly includes three core modules, namely, a tokenization module, a de-tokenization module, and a logical perception module.

[0033] like Figure 1 As shown, the method of the present invention specifically comprises the following steps: Step S1, data preprocessing. First, select the original training data from the target data set and preprocess it. Specifically, perform normalization on the numeric variables in the data set to eliminate the dimension effect and make them comparable on the same scale; perform text processing on the categorical variables, that is, convert the classification labels into text form so that the subsequent model can process them. After these processing steps, the obtained data can be directly used for model training.

[0034] Step S2, logical relationship perception model construction and training. Next, a logical relationship perception model is constructed and trained using the preprocessed data. This process aims to enable the model to learn and capture the inherent semantic logical relationships in the data. Subsequently, the trained model is used to extract the semantic logical relationships in the data and map these relationships into the latent space.

[0035] Step S3, conditional diffusion model construction and training. Further, a conditional diffusion model is constructed, and the model is trained using the latent variables and control condition information extracted in S2. The goal of this step is to enable the model to fully learn the distribution characteristics of the latent variables and the correspondence between samples and conditions, thereby ensuring the logical consistency and conditional relevance of the synthetic data.

[0036] Step S4, conditional controlled data synthesis. Using the trained conditional diffusion model, given any set of conditional inputs, starting from randomly sampled Gaussian noise, data that meets the specified conditions is controlled and synthesized. This step embodies the controllable conditional synthesis capability of the present invention, allowing users to generate data according to specific conditions.

[0037] Step S5, data decoding and de-tokenization. Finally, the decoder and de-tokenization module of the trained variational autoencoder (VAE) are used to decode the generated data from the latent space back to the real space. This process converts the synthesized electronic clinical data into a readable and usable form, completing the entire synthesis process.

[0038] Specifically, for the preprocessing operation in step S1, since the distribution types of numerical variables and categorical variables are different, different methods need to be used to process the two types of data respectively. An original data sample can be expressed as .in Represents numeric variables, which are mostly continuous numbers, such as age, weight, blood pressure and other numerical indicators. Represents categorical variables, which are mostly limited discrete data, such as gender, marital status, blood type, etc.

[0039] For digital features, the present invention adopts a nonlinear quantile transformation function , in order to achieve data normalization. The specific steps are as follows: First, by estimating the cumulative distribution function (CDF) of the feature, the original data value is mapped to a uniform distribution. This step can convert the original distribution of the data into a uniform distribution between 0 and 1, thereby eliminating the skewness of the original data distribution. Subsequently, the quantile function (QuantileFunction) of the inverse normal distribution is used to further map the uniform distribution value to the normal distribution. This process makes the distribution of the data closer to the standard normal distribution, that is, a distribution with a mean of 0 and a standard deviation of 1. For new data or feature values ​​that exceed the fitting range or unobserved data, they are mapped to the boundary values ​​of the output distribution. This processing method ensures the consistency of data preprocessing and prevents the influence of extreme values ​​on the model. Through this data preprocessing method, the influence of skewness and outliers in the data on the analysis results can be effectively reduced, making the distribution of the data more reasonable, thereby more accurately reflecting the true distribution of the data. This method is widely regarded as an important step to improve model performance and stability in the field of statistical analysis and machine learning.

[0040] For categorical features, text operations are required , converting it from its original classification form to text format. This text operation comprehensively considers the header information and context information, converting a single feature into a semantically rich text representation, which is crucial for the subsequent deep semantic analysis and feature extraction of the data. The core goal of this step is to ensure that the feature can fully capture and contain the rich semantic information contained in the dataset, laying the foundation for the subsequent semantic feature extraction of the pre-trained language model.

[0041] Specifically, step S2 is used to construct and train a logical relationship perception model, and its specific model structure is as follows: Figure 2 As shown, it mainly includes three core modules, namely, tokenization module, de-tokenization module and logic perception module.

[0042] In the process of semantic feature extraction, the tokenization module plays a vital role. Its main function is to encode the semantic features in the original data into a unified D-dimensional semantic vector representation. Here, D is the semantic vector dimension, which can be used as a hyperparameter of the model. Different data sets will have different optimal choices. This module adopts differentiated processing strategies for preprocessed numeric variables and categorical variables to adapt to the characteristics of different data types.

[0043] For preprocessed numeric variables , the tokenization module maps it to a D-dimensional vector through a linear transformation As a one-dimensional scalar, the original data can be significantly enhanced in semantic expression by encoding it into a high-dimensional space. The specific encoding process follows the following formula:

[0044] in, and are two learnable parameters. During the model training process, these two parameters will be continuously updated through optimization algorithms such as gradient descent in order to achieve the optimal solution of the model objective function. Through this encoding mechanism, the tokenization module can not only effectively convert the original information of digital variables into high-dimensional semantic vectors, but also gradually improve the model's ability to capture the semantic features of data by adjusting the learning parameters.

[0045] For the processing of categorical variables, the tokenization module uses a semantic feature extractor based on the pre-trained BERT model, which aims to extract deep semantic features from the text-processed categorical variables and map them into a D-dimensional semantic vector space. The pre-trained BERT model is well-known for its excellent semantic extraction capabilities and has shown outstanding performance in natural language processing fields such as sentiment analysis. The following is a detailed structural description of the semantic feature extraction module: set up Represents a categorical variable After text manipulation The text variable after . Input into the BERT model and extract the [CLS] tag of the text, which contains the global semantic features of the entire text. Then, this feature is mapped to a unified D-dimensional vector through a linear layer. The specific calculation process can be expressed as:

[0046] in, Represents a textual operation. Represents the semantic feature extraction process of the pre-trained BERT model, Represents the [CLS] token extracted by BERT. Represents the linear mapping layer, which is responsible for converting the high-dimensional semantic feature vector output by the BERT model Compressed into D-dimensional space. Through this structure, the tokenization module can effectively extract rich semantic information from categorical variables, providing strong support for the model to deeply understand the semantic structure of the data.

[0047] After obtaining a unified dimensional vector representation of all data, the logic perception module uses a variational autoencoder (VAE) structure to capture the complex logical relationship between data columns. The specific model architecture of VAE is as follows: Figure 3 As shown in Figure 1, it mainly consists of two parts: encoder and decoder. The encoder outputs a probability distribution (usually a Gaussian distribution), and the decoder samples and generates data from this distribution.

[0048] The function of the encoder is to map the input data into the latent space. Specifically, the input received by the encoder is the unified D-dimensional semantic vector obtained in the previous step. and The output is two vectors: the mean and variance The two output vectors together define a Gaussian distribution , which represents the distribution of given input data Latent variables Here, represents the latent variable, represents all D-dimensional semantic vectors, Indicates the mean The variance is Gaussian distribution.

[0049] The encoder proposed in this invention consists of N Transformer structures, each of which contains a self-attention layer and a feedforward layer. The feedforward layer consists of two linear structures with a dimension of 128 and a ReLU activation layer. The encoder contains two groups of Transformers, one for generating the mean , and the other group is responsible for generating the log variance .

[0050] Then, through the reparameterization technique, from the above Gaussian distribution Sampling a latent variable Specifically, we first introduce a standard normal distribution random variable , then calculate , thus achieving Sampling in process. That is, the latent space variables contain deep logical relationships. Mapping the original data to the latent space variables can enhance the semantic expression ability of the model and enable the model to fully capture the semantic logical relationship between features.

[0051] The task of the decoder is to transform the latent variables Map back to the original data space to generate a reconstructed D-dimensional vector and That is, the input to the decoder is , the output is and The goal of the decoder is to minimize the reconstruction of the D-dimensional vector and With the original D-dimensional vector and The decoder proposed in this invention is also an N-layer Transformer, which matches the structure of the encoder. Through this structure, the VAE model can not only learn the distribution characteristics of the data, but also capture and learn the complex logical relationships between data columns, providing strong support for subsequent data analysis and applications.

[0052] Furthermore, the function of this de-tokenization module is to reconstruct the D-dimensional vector and Convert back to the original data form. This module corresponds to the tokenization module and uses different processing methods for numeric variables and categorical variables.

[0053] For numeric variables, the de-tokenization module uses a linear transformation to reconstruct the vector from D dimensions Restored to a one-dimensional scalar, the specific decoding formula is as follows:

[0054] in, and These are all learnable parameters of the linear module, which are adjusted by the optimization algorithm during the model training process to ensure the accuracy of the reconstruction.

[0055] For categorical variables, the tokenization module also uses a linear encoding to reconstruct the vector from D dimensions. Decode to a dimensional vector, Represents the original categorical variable The number of candidate values. The vector of dimension is input into the Softmax activation function, and we get this The predicted probability distribution of candidate values. Finally, the candidate value with the largest probability is selected to restore the original data form. The specific decoding function is as follows:

[0056] in, and These parameters are all learnable parameters. These parameters are also optimized during the model training process to improve the accuracy of classification.

[0057] After detailed explanation, we have fully described the specific structure of the logical relationship perception model. After building the required model, the next step is to train the model using the preprocessed data obtained in step S1. Based on this, the overall loss function of the logical relationship perception stage is defined as follows: , Among them, the reconstruction loss It consists of the following two parts:

[0058] Reconstruction loss Quantified reconstruction data and the original data For categorical features, the cross entropy loss function is used. To measure the difference between the predicted value and the actual value; for numerical features, the mean square error loss function is used To evaluate the deviation between the predicted value and the actual value.

[0059] The specific form of KL divergence is:

[0060] Divergence Represents latent variables The posterior distribution of With its prior distribution The KL divergence between . Used to balance the reconstruction loss and The relative importance of divergences is determined by the model, thereby striking a balance between the fit of the model and the continuity of the latent space.

[0061] After sufficient training following the minimization principle defined by the loss function, the logic-aware model will be able to effectively learn and capture the intrinsic logical relationships between features. Through this training process, the model can not only identify explicit associations in the data, but also infer implicit, deep semantic connections. Using the trained model, we encode the original data into latent space variables containing rich semantic and logical relationships. This encoding process maps the data from its original high-dimensional space to a low-dimensional, dense latent representation space, where each latent variable contains the complex semantic information and logical structure of the data.

[0062] Specifically, step S3 is used to construct and train the conditional diffusion model. The model is responsible for fitting the latent variables obtained from the previous stage, while fully learning the distribution characteristics of the latent variables and the correspondence between samples and conditions, and realizing conditional generation in the latent space. The training process of the diffusion model includes the forward denoising process and the reverse denoising generation process, such as Figure 4 shown.

[0063] First, we transform the latent space variables obtained in step S2 into Recorded as , The subscript indicates the initial time step ,Right now represents the initial latent variable. Assume Obey probability distribution .

[0064] The forward noise addition process involves at each time step The process starts from the noise-free initial data. Start with, then gradually add noise to it, and get Intermediate data such as Indicates that at the final time step This continuous noise addition process simulates the evolution of data from a deterministic state to a random state, providing the necessary dynamic structure for the inverse denoising process of the diffusion model, thereby allowing the model to gradually recover the structure and characteristics of the original data from pure noise. To the time step The noise addition formula is as follows:

[0065] in, are predefined hyperparameters, Represents the standard normal distribution Gaussian noise extracted from . The role of is to adjust the proportional relationship between Gaussian noise and the previous state data. , the model can finely control the rate of noise introduction. The forward noise addition formula is written in the form of a probability transfer kernel:

[0066] This formula means that given the previous state , then the current state Subject to a mean , the variance is Normal distribution .in, Represents the identity matrix.

[0067] In the reverse process of the diffusion model, the model attempts to reverse the denoising process and gradually denoise from a complete noise to restore the original data and achieve data generation. The reverse denoising process involves training a Gaussian transfer kernel to achieve the goal of gradual denoising:

[0068] This formula means that given the previous state , then the current state Subject to a mean , the variance is Normal distribution . Specifically: ,

[0069] in, , .parameter Parameters of the noise prediction network for the diffusion model. is the output of the noise prediction network, i.e., the noise added to the previous state data. The input of the noise prediction network includes Noise data at time , the current time step , and control conditions , the output is the predicted noise By training the noise prediction network and obtaining its prediction output , we can calculate and . Then, according to the reverse transfer kernel formula , the model can gradually remove noise from pure Gaussian noise and achieve the data generation task.

[0070] The training objective function of the noise prediction network is as follows:

[0071] This objective function minimizes the actual noise and prediction noise The difference between them can achieve the goal of noise prediction.

[0072] The noise prediction network is responsible for predicting the noise added to the previous state data. , its model structure is the core part of the diffusion model and the key to support conditional generation. Its specific model structure is as follows Figure 5 As shown. The present invention proposes an innovative noise prediction network structure based on a multi-head cross attention mechanism, which cleverly integrates conditional semantic embedding.

[0073] The first layer of the noise prediction network model is an additive embedding layer, which embeds the time step information into the input data using the sinusoidal position encoding technique. Encoded into an embedding vector and added to the corresponding dimension of the input data, thus providing the model with time step information. The specific calculation formula of the embedding layer is as follows:

[0074] in, represents sinusoidal position encoding, Responsible for mapping the sinusoidally encoded data to the specified dimension. Specifically, sinusoidal position encoding is implemented by the following formula: ,

[0075] in, represents the time step, is the index of the embedding dimension, and is the dimension of the embedding vector. This encoding method not only captures the periodic changes of time steps, but also provides multi-scale feature representation, helping the model make more accurate denoising decisions during the generation process.

[0076] Following the additive embedding layer, the model architecture introduces a multi-head cross-attention layer, which is the core component for effectively integrating conditional information. The multi-head cross-attention mechanism allows the model to show higher flexibility and efficiency in processing conditional information. In this structure, the query sequence It can effectively transform semantic conditions Embedded into the model, ensure that the model can deeply understand and utilize the input conditional information, and achieve feature alignment between the conditional information and the data sample. The specific structure is as follows: ,in,

[0077] in, represents the output of the cross-attention structure, , , They represent query, key, and value respectively. Responsible for embedding conditional semantic information . and Receiving intermediate data of the model , does not participate in the introduction of conditional information, but through The interaction of two networks realizes the fusion of conditional information. , , is a learnable weight matrix used to transform conditional information and intermediate data Map to the corresponding , , space. yes , , The dimension of is a function used to calculate the attention weight. The design of the multi-head cross attention layer enables the model to flexibly integrate conditional control into the generation process, provides the model with the ability to capture conditional information at different scales, enhances the model's sensitivity to conditional changes, and improves the accuracy and applicability of data synthesis.

[0078] Specific, conditional semantic information It is calculated by the feature combination module. The function of this module is to extract the semantic features of any combination of conditions defined by the user. The calculation formula of this module is as follows:

[0079] in, is the D-dimensional semantic vector obtained by the tokenization module for the categorical variable, express Any subset of . From this we can see that express The sum of the elements of any subset of , that is, the combination of semantic features of any number of categorical variables. In this way, any combination of categorical variables can be specified as conditions to guide and control the synthesis process, thereby generating data containing specified features.

[0080] The above is an overview of the model structure and training method proposed in the present invention, which includes two core training stages, namely the logical relationship perception stage and the conditional diffusion model generation stage. Both stages have an objective function to be trained, which are responsible for different functions. After the model training of the above two stages is completed, the clinical data generation task can be carried out.

[0081] Furthermore, step S4 is responsible for data generation. In this step, the user can set a specific combination of conditions and use the conditional diffusion model trained in step S3 to generate data with randomly sampled Gaussian noise. As a starting point, follow the trained denoising transfer kernel , and follow the direction of conditional control to gradually perform denoising , and finally synthesize data that meets the preset conditions in a targeted manner .

[0082] Finally, step S5 uses the decoder and de-tokenization module of the trained variational autoencoder (VAE) to generate the generated data Decoding from latent space back to real space data and This process converts the synthesized electronic clinical data into a readable and usable form, completing the entire synthesis process. As a result, we have successfully achieved the task of generating conditional electronic clinical data, ensuring that the synthetic dataset is consistent with the real-world data in structure and semantics, and providing high-quality data resources for subsequent data analysis and applied research.

[0083] In order to comprehensively evaluate the effectiveness and practicality of the electronic clinical data synthesis method proposed in the present invention, a series of experimental tests were conducted, which included two levels: qualitative analysis and quantitative evaluation.

[0084] Qualitative test results: The experimental verification of the electronic clinical data synthesis method proposed in the present invention revealed positive results. The experimental results show that the proposed model performs well in capturing the logical dependencies between data columns and can significantly reduce the problem of logical inconsistency in the synthesized data, thereby ensuring the reliability and validity of the synthesized data. In addition, the experimental results further confirm that the conditional control mechanism introduced in the present invention exhibits excellent matching accuracy and can efficiently produce data that meets specific conditional combinations. These findings fully demonstrate the effectiveness and practicality of the present method in synthesizing electronic clinical data that meets specific conditions.

[0085] Quantitative test analysis: In order to evaluate the performance of the electronic clinical data synthesis method proposed in this paper, quantitative tests were conducted on the clinical obesity analysis dataset and compared with the three most widely used methods. These three methods are based on different generative model frameworks: TVAE, a data synthesis scheme based on variational autoencoder (VAE), CTGAN, a data synthesis scheme based on generative adversarial network (GAN), and TabDDPM, a data synthesis scheme based on diffusion model. Two key evaluation indicators were used in the experiment: F1 score (F1 Score) is used to quantitatively evaluate the classification performance of the classification model on the real test dataset after training on the synthetic dataset. The higher the value of this indicator, the better the performance. The correlation coefficient difference (CCD) measures the consistency of the logical relationship between the synthetic data and the real data. The lower the value of this indicator, the better the consistency. The data in Table 1 clearly show that the method proposed in this study performs well in improving the classification effect of the downstream classification model, confirming its significant practicality. At the same time, it also performs well in maintaining the consistency of the logical relationship between the synthetic data and the real data, confirming its high effectiveness. These quantitative results provide strong empirical support for the effectiveness and practicality of this method.

[0086] Table 1 The synthesis effects of multiple synthesis schemes on the clinical obesity analysis dataset

[0087] The present invention also provides a controllable condition electronic clinical data synthesis device based on a diffusion model and having a logical relationship perception capability, comprising the following units: A data preprocessing unit, used for preprocessing digital variable data and categorical variable data; A logical relationship perception model construction and training unit is used to construct a logical relationship perception model, train the logical relationship perception model using preprocessed data, and extract potential variables in the data using the trained model; The processing process of the logical relationship perception model is as follows: first, the semantic features in the data are encoded into the original D-dimensional semantic vector representation using the tokenization module, then the latent variables rich in semantic logical relations are obtained using the variational autoencoder structure in the logical perception module, then the latent variables are decoded to generate a reconstructed D-dimensional vector, and the difference between the reconstructed D-dimensional vector and the original D-dimensional semantic vector is minimized, and finally the D-dimensional reconstructed vector is converted back to the form of data using the de-tokenization module; where D is a hyperparameter; A conditional diffusion model construction and training unit, used to construct a conditional diffusion model and train the conditional diffusion model using the extracted latent variables; A controllable data synthesis unit is used to use the trained conditional diffusion model to synthesize data that meets the specified conditions from randomly sampled Gaussian noise given any set of conditional inputs; The data decoding and restoration unit is used to decode the synthesized data from the latent space back to the real space, convert the synthesized electronic clinical data into a readable and usable form, and complete the entire synthesis process.

[0088] The specific implementation method of each module is the same as the above steps and will not be repeated here.

[0089] The above details various embodiments of the present invention. It should be understood that these descriptions are exemplary and not exhaustive, and they should not be interpreted as the sole limitation on the possible applications of the present invention. Obviously, it is possible for a person skilled in the art to make various modifications and adjustments to the present invention without departing from the core scope and spirit of the present invention. If these modifications and adjustments fall within the scope of the claims of the present invention and their equivalents, they should also be considered as part of the present invention's intended protection.

Claims

1. A controllable conditional clinical data synthesis method supporting logical relationship perception, characterized in that: The steps include: Step S1, preprocessing the digital variable data and the categorical variable data; Step S2, constructing a logical relationship perception model, and using the preprocessed data to train the logical relationship perception model, and using the trained model to extract potential variables in the data; The processing process of the logical relationship perception model is as follows: first, the semantic features in the data are encoded into the original D-dimensional semantic vector representation using the tokenization module, then the latent variables rich in semantic logical relations are obtained using the variational autoencoder structure in the logical perception module, then the latent variables are decoded to generate a reconstructed D-dimensional vector, and the difference between the reconstructed D-dimensional vector and the original D-dimensional semantic vector is minimized, and finally the D-dimensional reconstructed vector is converted back to the form of data using the de-tokenization module; where D is a hyperparameter; Step S3, constructing a conditional diffusion model, and using the latent variables extracted in step S2 to train the conditional diffusion model; Step S4, using the trained conditional diffusion model, given any set of conditional inputs, starting from randomly sampled Gaussian noise, synthesizing data that meets the specified conditions; In step S5, the synthesized data is decoded from the latent space back to the real space, and the synthesized electronic clinical data is converted into a readable and usable form, completing the entire synthesis process.

2. The controllable conditional clinical data synthesis method supporting logical relationship perception according to claim 1, characterized in that: The data preprocessing in step S1 is specifically as follows: for numerical variable data, a nonlinear quantile transformation function is used to achieve data normalization; for categorical variable data, a text operation is used to convert it from the original classification form to a text format.

3. The controllable conditional clinical data synthesis method supporting logical relationship perception according to claim 1, characterized in that: In step S2, a differentiated processing strategy is adopted for the preprocessed digital variable data and categorical variable data to adapt to the characteristics of different data types. The processing process of the tokenization module is expressed by the following formula: ， in, Represents the linear mapping layer, for each text-based categorical variable data , use the pre-trained BERT model to extract the semantic features of the text, and finally use a linear structure to encode the semantic features into a D-dimensional vector; for the normalized digital variable data , using two trainable parameters and Encode the normalized features into a unified D-dimensional vector space. and That is, the D-dimensional classification type and numerical semantic vector after tokenization. and They are the indexes of numeric variable data and categorical variable data respectively.

4. The controllable conditional clinical data synthesis method supporting logical relationship perception according to claim 3, characterized in that: The de-tokenization module forms a corresponding relationship with the tokenization module, and the processing process is expressed by the following formula: ， in, , , and are all learnable parameters. and They are D-dimensional reconstructed numerical vectors and categorical vectors, and are the restored numerical vectors and categorical vectors respectively; for digital variable data, a linear decoder is used to convert the vector Restore to ; For categorical variable data, after using a linear transformation, a Softmax structure is used to obtain the probabilities of all candidate values, and finally the candidate value with the largest probability is selected.

5. The controllable conditional clinical data synthesis method supporting logical relationship perception according to claim 1, characterized in that: Variational Autoencoder VAE is based on Transformer -VAE structure, consisting of an encoder and a decoder. The encoder accepts a unified original D-dimensional semantic vector representation as input and outputs two vectors: mean and variance , these two output vectors together define a Gaussian distribution ,in represents the latent variable, and the Gaussian distribution represents the given input data When the latent variable The probability distribution of The decoder samples and generates data from this probability distribution; The encoder consists of two sets of Transformer structures, one for generating the mean , and the other group is responsible for generating the log variance , then, by the reparameterization technique, a latent variable is sampled from the Gaussian distribution , by introducing a standard normally distributed random variable , to achieve Sampling in process, i.e. ; Each Transformer structure contains a self-attention layer and a feedforward layer. The feedforward layer consists of two linear layers and a ReLU activation layer. The decoder is also an N-layer Transformer, matching the structure of the encoder. Its task is to transform the latent variables Mapping from the latent space back to the original data space to produce a reconstructed D-dimensional vector.

6. The controllable conditional clinical data synthesis method supporting logical relationship perception as claimed in claim 5, characterized in that: The overall loss function of the logical relationship perception model is: Among them, the specific form of KL divergence is: The reconstruction loss It consists of the following two parts: Reconstruction loss Quantified reconstruction data and the original data For categorical variable data, the cross entropy loss function is used to measure the difference between the predicted value and the actual value, where Represents the original categorical variable data, Represents the reconstructed categorical variable data; for numerical variable data, the mean square error loss function is used To evaluate the deviation between the predicted value and the actual value, Represents the original numeric variable data, Represents the reconstructed numeric variable data; Divergence indicates latent variables The posterior distribution of With its prior distribution The KL divergence between Used to balance the reconstruction loss and The relative importance of divergences.

7. The controllable conditional clinical data synthesis method supporting logical relationship perception according to claim 1, characterized in that: The training process of the conditional diffusion model in step S3 includes a forward denoising process and a reverse denoising process. The forward denoising process is performed at each time step. In the process of de-noising, predefined Gaussian noise is gradually added to the initial data, i.e., the latent variables obtained in step 2, gradually converting the data into noise; in the reverse denoising process, the original data is restored from the noise, thereby realizing data generation.

8. The controllable conditional clinical data synthesis method supporting logical relationship perception according to claim 1, characterized in that: The inverse denoising process in the conditional diffusion model is implemented through a noise prediction network, and the input includes Noise data at time , the current time step , and control conditions , the output is the predicted noise The first layer of the noise prediction network is an additive embedding layer, whose function is to embed the time step information into the input data using the sinusoidal position encoding technique; following the additive embedding layer is a multi-head cross attention layer, and the specific structure of the cross attention layer is as follows: , in, in, represents the output of the cross attention layer, , , Represents query, key and value respectively; Responsible for embedding conditional semantic information , and Receive the intermediate data output by the embedding layer , does not participate in the introduction of conditional information, but through The interaction of can realize the fusion of conditional information; , , is a learnable weight matrix used to transform conditional information and intermediate data Map to the corresponding , , space, yes , , The dimension of is the function used to calculate the attention weight; Control conditions It is calculated by the feature combination module, which is used to extract the semantic features of any combination of conditions defined by the user. The calculation formula is as follows: in, is the D-dimensional semantic vector obtained by the tokenization module for the categorical variable, is the index of the categorical variable, express Any subset of is the number of categorical variables, from which we can see that express The sum of the elements of any subset of , that is, the combination of the semantic features of any number of categorical variables.

9. The controllable conditional clinical data synthesis method supporting logical relationship perception according to claim 8, characterized in that: The training objective function of the noise prediction network is as follows: in, represents the standard normal distribution, represents the expectation, and the objective function minimizes the actual noise and prediction noise The difference between them can achieve the goal of noise prediction.

10. A controllable conditional clinical data synthesis device supporting logical relationship perception, characterized in that: The following units are included: A data preprocessing unit, used for preprocessing digital variable data and categorical variable data; A logical relationship perception model construction and training unit is used to construct a logical relationship perception model, train the logical relationship perception model using preprocessed data, and extract potential variables in the data using the trained model; The processing process of the logical relationship perception model is as follows: first, the semantic features in the data are encoded into the original D-dimensional semantic vector representation using the tokenization module, then the latent variables rich in semantic logical relations are obtained using the variational autoencoder structure in the logical perception module, then the latent variables are decoded to generate a reconstructed D-dimensional vector, and the difference between the reconstructed D-dimensional vector and the original D-dimensional semantic vector is minimized, and finally the D-dimensional reconstructed vector is converted back to the form of data using the de-tokenization module; where D is a hyperparameter; A conditional diffusion model construction and training unit, used to construct a conditional diffusion model and train the conditional diffusion model using the extracted latent variables; A controllable data synthesis unit is used to use the trained conditional diffusion model to synthesize data that meets the specified conditions from randomly sampled Gaussian noise given any set of conditional inputs; The data decoding and restoration unit is used to decode the synthesized data from the latent space back to the real space, convert the synthesized electronic clinical data into a readable and usable form, and complete the entire synthesis process.

Images