Systems and methods for generating synthetic datasets from original datasets

Abstract

A method for generating a synthetic dataset from an original dataset, comprising encoding categorical features of the original dataset, embedding the encoded dataset in a low-dimensional space, selecting seed records from the embedded dataset, identifying a plurality of nearest-neighbor records of the seed records, generating new records by randomly selecting features from the plurality of nearest-neighbor records, and concatenating the new records into a synthetic dataset. For a synthetic dataset containing n records, the selection, identification, generation, and concatenation operations on records in the embedded dataset are performed a total of N times, which can be the same as or different from the number of records in the original dataset.

Description

Technical Field

[0001] This application relates to a system and method for generating synthetic datasets from raw datasets. Background Technology

[0002] The widespread adoption of electronic health records (EHRs) in healthcare has led to a surge in the volume of private health data. With this growth, scientists and researchers have been able to use artificial intelligence and machine learning techniques to understand and predict, thereby advancing patient care and treatment. However, due to stringent privacy requirements for patient data, this data cannot be easily shared between organizations or widely made available to scientists and researchers.

[0003] Techniques have been developed to de-identify and anonymize this type of data, but these techniques still cannot fully guarantee patient privacy. These techniques may leave residual patterns in the anonymized data, which, along with other information sources, can help identify and diagnose individuals.

[0004] Data from clinical trials (sometimes called “clinical studies”) is also a major source of private health data. Like EHR data, this data is typically in electronic form from electronic data collection (EDC) systems. Clinical trial data typically has a much smaller sample size compared to EHR data (tens or hundreds of records compared to thousands), constrained not only by the strong privacy interests of participants (or patients), but also by the strong privacy interests of trial donors or data contributors, as well as regulatory and technological protection requirements. Clinical trial data is also valuable (and may be more valuable than EHR data) because it comes from participants who have consented to participate in controlled trials. Such trials aim to make high-value inferences (e.g., the effect of a drug versus placebo on participant survival for a specific disease under a particular treatment regimen). This type of data also helps estimate the efficacy of clinical trial designs, discover concomitant drug effects, and identify participant factors that may influence the success of clinical trials. Therefore, EDC data represents high-dimensional data collected periodically and repeatedly for each participant in a carefully controlled and standardized process. Summary of the Invention

[0005] A method for generating a synthetic dataset comprising n records from an original dataset includes: encoding categorical features of the original dataset; embedding the encoded dataset into a low-dimensional space; selecting a seed record from the embedded dataset; identifying multiple nearest neighbor records of the seed record; generating a new record by randomly selecting features from the multiple nearest neighbor records; concatenating the new record into the synthetic dataset; selecting a new seed record; and repeating the identification, generation, concatenation, and selection operations N-1 times for the records in the embedded dataset.

[0006] A system for generating a synthetic dataset comprising n records from a raw dataset includes: an encoder for encoding categorical features of the raw dataset; an embedder for embedding the encoded dataset into a low-dimensional space; a clusterer for selecting a seed record from the embedded dataset and identifying multiple nearest neighbor records of the seed record; and a synthetic record generator for generating new records by randomly selecting features from the multiple nearest neighbor records and concatenating the new records into the synthetic dataset, wherein the selection, identification, generation, and concatenation operations are performed N times for the records in the embedded dataset. Attached Figure Description

[0007] Figures 1A to 1B This is a block diagram of a system for generating synthetic datasets according to an embodiment of the present invention;

[0008] Figure 1C This is a flowchart illustrating how a synthetic dataset can be generated according to an embodiment of the present invention;

[0009] Figures 2A to 2E It shows about Figure 1C More details of the process;

[0010] Figures 3A to 3D Different aspects of the data involved in developing this invention are illustrated;

[0011] Figure 4 Scatter plots comparing raw and synthesized data using four different synthesizer methods are shown, including one synthesizer method used in an embodiment of the present invention.

[0012] Figure 5 The following are receiver operation characteristic curves for several cluster sizes according to embodiments of the present invention;

[0013] Figure 6 The illustration shows partial data exposed to data attackers according to embodiments of the present invention, as a function of compromised records and compromised features of several cluster sizes; and

[0014] Figures 7A to 7C This is a visualization of clinical trial data with different degrees of feature deletion according to embodiments of the present invention.

[0015] Where deemed appropriate, reference numerals in the accompanying drawings may be repeated to indicate corresponding or similar elements. Furthermore, some blocks depicted in the drawings may be combined to form a single function. Detailed Implementation

[0016] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of embodiments of the invention. However, those skilled in the art will understand that embodiments of the invention can be practiced without these specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to obscure the invention.

[0017] As mentioned above, techniques for de-identifying and anonymizing EHR data cannot fully guarantee patient privacy with this type of data. Other techniques, such as Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs), can provide privacy controls by generating synthetic data based on learned distributions using deep learning and machine learning. However, these techniques require large amounts of data from EHRs and are not suitable for the smaller EDC datasets from clinical trials. Therefore, privacy concerns regarding clinical trial participants and donors remain.

[0018] The inventors have developed a method and system to address these privacy challenges by generating synthetic datasets that still maintain high fidelity. This method combines low-dimensional embeddings of real-world data to identify the nearest neighbors of subjects in a compressed feature space, and then incorporates feature permutation and recombination methods to introduce controlled but realistic variations. This method generates high-fidelity, synthetic subject records and operates efficiently across various data mechanisms and data types, while concealing the identity and origin of the subjects (e.g., donors and contributors).

[0019] In this specification, "fidelity" includes "statistical fidelity," meaning that any analysis performed on synthetic data will produce the same or very similar results as those performed on real data. Furthermore, "privacy" refers to the inability to identify an individual in a given individual case (e.g., in the aggregation of data).

[0020] Figure 1A and Figure 1B An embodiment of the system is illustrated. System 100 includes a data synthesizer 10 that acquires raw real data 5 and generates synthetic data 95. A fidelity cross-validator 20 indicates the fidelity of the synthetic data to the real data. A privacy evaluator 30 evaluates the degree of data privacy protection provided by the present invention compared to other numerical methods.

[0021] Figure 1BSynthesizer 10 is shown in more detail. Synthesizer 10 includes encoder 11, embedder 13, clusterer 15, feature pairing detector 17, and synthetic record generator 19. Encoder 11 encodes categorical features, and embedder 13 embeds records into a low-dimensional space. Clusterer 15 selects a seed record and a “cluster” of records surrounding it. Feature pairing detector 17 detects relevant (or statistically relevant) features, such as age and date of birth, to separate them together when generating synthetic records. Synthetic record generator 19 takes the record clusters, notes highly relevant features, and arranges the features of the record clusters to generate synthetic records. Figure 1C The flowchart in the document provides more details about the operation of these blocks.

[0022] The inventor's method includes the following pseudocode:

[0023] 1. An original dataset X∈R with n records and m features. nxm ;

[0024] 2. Embed X into a space V of dimension p (e.g., t-SNE, UMAP, PCA) to generate embedding points r. s ;

[0025] 3. Use the following attributes:

[0026] • N > 0 (size of the synthetic dataset);

[0027] • k > 0 (the number of neighborhoods, also known as the number of "clusters");

[0028] • p (embedding dimension, m>p>0);

[0029] • T(.) feature arrangement mechanism.

[0030] 4. Generate a synthetic dataset R∈R Nxm :

[0031] When i≤N:

[0032] Generate random seeds to select some r s ;

[0033] • Choose r s k nearest neighbors (l 2 distance);

[0034] • Generate new records r by permuting features according to T(.). s' ;

[0035] • Storage r s' Combined with other generated records to form R;

[0036] ·i = i + 1;

[0037] Finish

[0038] Now refer to Figure 1C and Figures 2A to 2E . Figure 1C This is a flowchart illustrating how a synthetic dataset can be generated according to an embodiment of the present invention. In operation 105, the original source dataset is input. This dataset contains n records, where each record represents data from a subject (or patient), and each record is represented by m distinct features (sometimes called "variables"). Figure 2A As shown, a dataset can be represented as a set of tabular data with n records and m features. This method is designed to handle both large and small datasets, but it has the greatest advantage when dealing with small datasets (e.g., datasets from clinical trials). These features can be categorical or numerical, and can include demographic data (e.g., name, age, gender, country), vital signs data (e.g., height, weight, blood pressure, heart rate, etc.), and other types of data.

[0039] In operation 110, the dataset is processed by encoding categorical features. Categorical feature encoding can include label encoding or one-hot encoding, which converts categorical values ​​into numerical values. Missing values ​​are then estimated by padding the mean and pattern of the numerical and categorical features, respectively. This operation transforms the data into all numerical values ​​without missing values, preparing the data for the embedding operation, as described below.

[0040] In operation 115, the records are embedded in a low-dimensional space. This embedding involves mapping the records to a p-dimensional feature space V, where p is between 0 and m. Preferably, p is small, for example, 2 or 3, resulting in a low-dimensional feature space, which makes subsequent k-nearest neighbor clustering operations more effective. Embedding helps determine which records are similar to each other. Figure 2B The space of p=2 is shown. Embeddings can be performed using t-random neighborhood embedding (t-SNE), unified manifold approximation and projection (UMAP), or principal component analysis (PCA). Figure 2B As shown, T-SNE (sometimes called t-distribution SNE) calculates the probability of correlation between record pairs in a high-dimensional space and then selects low-dimensional embeddings that produce similar distributions. UMAP is similar to t-SNE, but it assumes that the data is uniformly distributed on a locally connected Riemannian manifold, and that the Riemannian metric is a local constant or approximately a local constant. PCA calculates the principal components (dimensions in the low-dimensional space) of a set of records and performs basis changes on the data using the principal components, sometimes using only the first few principal components and ignoring the rest. Figure 2C The differences between using these three types of dimensionality reduction are illustrated. Black dots represent the original data; gray dots show how each method generates slightly different synthetic datasets.

[0041] When a record is embedded in a low-dimensional space, a seed record r is randomly selected in operation 120. s Then, operation 125 identifies the k nearest neighbors of the seed record. The value of k is heuristically chosen based on a trade-off between fidelity and privacy, as well as the type of application. As k increases, fidelity decreases, but privacy increases. Figure 2D The case where k=3 is shown, thus determining the seed record r. s The three nearest neighbors. These are labeled r. 0 r 1 and r 2 .

[0042] In operation 130, a new synthetic record r is generated. s' For each record in the low-dimensional space, this method generates one or more synthetic records by arranging the features of its k nearest neighbors within the same cluster and a certain radius / distance. Several possible permutation algorithms are possible: cyclic, random with repetition, random without repetition, etc. The inventors typically use a cyclic permutation, which traverses all k neighbors in the cluster, taking the first feature value of the first neighbor, the second feature value of the second neighbor, and so on. In one embodiment of the invention, a small amount of noise can be added to the numerical features to ensure the new original feature values.

[0043] Several parameters are configurable. The ratio of the number of synthetic records (“N”) to the number of real records (“n”) is configurable. That is, N can be equal to n, N can be less than n, or N can be greater than n. In the latter case, generating multiple synthetic records (on average) for each real record can be useful, for example, if there is a specific data cluster (population) or if there is insufficient data in the cluster and more data needs to be generated for training and / or analysis. The cluster size k is also configurable and can be fixed or variable: the smaller the cluster, the closer the generated subjects are to real subjects. Another configurable parameter is used to protect the privacy of subjects with unique eigenvalues ​​(i.e., outliers), and the method can be configured to omit records for these subjects. Mathematically, an “outlier” is defined as a record whose distance to its nearest neighbor is greater than the qth percentile of the distances to its nearest neighbors of all records. The value of q is a configurable parameter, and an example of q is 95.

[0044] Figure 2E The generation of the synthetic record r is shown. s' Example. r s' Each feature is from the k+1 group r s It is randomly arranged in its k nearest neighbors. Therefore, in Figure 2E In the middle, the synthetic record r s' The first feature in r comes from sThe second and penultimate features come from r 0 The third and fifth features come from r 1 The fourth and last features come from r 2 .

[0045] When arranging features from their nearest neighbors, this method works better if the features are weakly coupled or correlated with each other. Figure 3A Three distributions are shown for two feature groups. Distribution 301 shows a high linear correlation between the two features in the first group, and distribution 302 shows a high non-linear correlation between the two features in the second group. Conversely, distribution 303 shows a weak correlation between the two features in the third group.

[0046] One novel aspect of Operation 130 is that it also works in conjunction with highly correlated features in the subject's record. It identifies these highly correlated features and separates them together to make the synthesized data more realistic. For example, a subject's record might contain eight features: date of birth, sex, address, age, height, weight, blood pressure, and body mass index (BMI). Date of birth and age are highly correlated with each other, while BMI, height, and weight are highly correlated. When randomly combining features from nearest neighbors, Operation 130 can first identify highly correlated features and then force them to separate together to preserve them together in the synthesized record, rather than processing all features in the subject's record individually. Therefore, the subject's age and date of birth will always be preserved together in the synthesized record, as will the subject's BMI, height, and weight (although these five features may not necessarily be preserved together), while sex, address, and blood pressure will be randomly arranged with their nearest neighbors in the record.

[0047] This method can also better handle the "high noise" and "low accuracy" characteristics that often occur in clinical trials. Figure 3B Examples of low-noise (311) and high-noise (312) feature distributions are shown. It is assumed that individual measurements (e.g., blood pressure) are noisy and inaccurate. Therefore, for most feature values, the between-subject and within-subject variances have similar magnitudes for continuous variables. In other words, the variance between nearest neighbors is no greater than the within-subject variance.

[0048] This method also outperforms gradient descent learning methods on small datasets, which often produce noisy and unstable solutions that are usually difficult to fit the observed distribution. Figure 3C An example of the instability of gradient descent learning methods with small sample sizes (e.g., 100 to 1000 records per sample) is shown.

[0049] This approach is also applicable to datasets requiring high source privacy (i.e., datasets where it is difficult to determine who contributed which data). Clinical trial data have fewer data sources (i.e., relatively easy to guess), so the data should not be separated in a way that allows someone to determine who the data contributors are. Figure 3D The diagram illustrates separable and non-separable distributions of data contributed by source A and source B. The method of this invention manages the data to obfuscate separability attacks.

[0050] When generating synthetic record r s' In operation 135, the synthesized record r will be... s' The data is stored as part of a synthetic dataset R'. The method then selects a new seed record in operation 140 and returns to operation 125 to generate another synthetic record using features from the k nearest neighbors of that new seed record. This loop is executed N times in total, concatenating each synthetic record with the previous synthetic record to form a synthetic dataset R'195 with n records.

[0051] Apart from Figure 1C In addition to the operations shown, other operations or series of operations are conceived as generating synthetic datasets from the original record dataset. Furthermore, Figure 1C The actual order of operations in the flowchart is not intended to be restricted, but can be performed in any actual order.

[0052] This method and system outperform existing machine learning methods for anonymizing electronic health records and other data, while remaining accurate and faithful to the original dataset. The method and system also offer better protection of privacy at the subject level and at the source, contributor, or donor level.

[0053] The inventors evaluated the performance of the techniques of this invention on a proprietary dataset consisting of three clinical trials in a clinically homogeneous study group of highly refractory multiple myeloma. To evaluate performance in a broader context, the techniques were also tested on four public datasets from the University of California, Irvine (UCI) Machine Learning Repository: UCI Cardiology, UCI Heart Failure, UCI Breast Cancer, and UCI Lung Cancer datasets.

[0054] Performance or fidelity evaluation

[0055] The inventors used the Synthetic Data Generation Benchmark (SDGym) to benchmark these techniques against other state-of-the-art synthesizers. The SDGym benchmark (Synthetic Data Vault Project (N. Patki, R. Wedge, K. Veeramachaneni, “The Synthetic Data Vault” (2018), https: / / sdv.dev / SDV / index.html)) is a library that provides a collection of real and simulated datasets, along with a set of classic and novel synthetic data generators for comparison. The benchmark uses a “likelihood fitness” metric and a “machine learning efficiency” metric. The “likelihood fitness” metric is used for synthetic data generated from simulated datasets. The “machine learning efficiency” metric is used for synthetic data generated from real datasets. Because simulated data comes from a known distribution, the likelihood fitness test checks whether features in the synthetic dataset follow the same joint distribution as in the original dataset. To test the machine learning performance on synthetic data generated from real data, the SDGym benchmark trains a model on the synthetic data to predict one feature given other features, to see if the model can achieve similar performance on the original test data. The classifier is evaluated by accuracy and F1 score, and the regression is evaluated by R-squared. Detailed information on the SDGym benchmark can be found at https: / / github.com / sdv-dev / SDGym.

[0056] To evaluate the fidelity of the synthetic dataset, cross-validation tests were performed on the synthetic and source datasets to check the degree to which the underlying attributes of the source were preserved. The Fisher Exact and Kolmogorov-Smirnov (KS) tests were used for both binary and non-binary features; the mean of numerical features was also compared to check univariate fidelity. For multivariate checks, the correlation of pairwise features in the synthetic data was examined and compared to the correlation in the original data. Furthermore, an unsupervised method similar to the bag-of-words (BoW) representation was used to compare the synthetic dataset with the original dataset. To measure the separability of the synthetic and original datasets, a silhouette coefficient and a random forest classifier were used. The silhouette coefficient is a multivariate, unsupervised metric that compares one cluster to another and quantifies the overlap between the real and synthetic datasets. A supervised random forest classifier predicted the subjects' outcomes at the end of the trial period. The area under the curve (AUC) was used to measure the accuracy of the random forest classifier.

[0057] Privacy assessment

[0058] The ability of the method of the present invention to protect privacy at both the subject / individual and source levels was also evaluated. Regarding subject-level privacy, in addition to examining whether the invention memorizes data from the source level and generates it in synthetic data, the robustness of the method to member disclosure risk and attribute disclosure risk was also examined.

[0059] As background, the risk of membership disclosure is defined as determining whether the “real” (i.e., training) dataset contains the target subject. See Z. Zhang, C. Yan, D. Mesa, J. Sun, and B. Malin, “Ensuring electronic medical record simulation through better training, modeling, and evaluation,” JAMIA 27(1):99-108 (2020). In clinical trial datasets, all subjects in the study have some condition common to all trial participants. If a “data attacker” can determine the membership of the target subject in the dataset, the attacker can determine that the subject has a trial-specific condition. If the attacker can prove that even a single subject is in the study, the attacker can discredit the institution and indicate a violation of patient privacy laws or regulations. To assess this risk, a test set similar to the attacker’s target records is generated by randomly sampling a small subset of records from the original dataset. K-Fold cross-validation is used, where k = 10. For each record, its Hamming distance to records in the synthetic dataset is determined, and if the distance is less than a threshold, the record is marked as a match from the original dataset. If the record is actually in the original dataset, the detection is marked as a true positive; otherwise, it is marked as a false positive. For numerical values, the modified Hamming distance considers two attribute values ​​to be equal if they differ from each other by less than 2.5%.

[0060] The risk of attribute leakage arises when a subject's sensitive characteristics can be inferred from a set of more general attributes known to that subject. To assess this risk, consider a scenario where an attacker has partial knowledge of the original data (e.g., a subset of records and / or attributes). For example, suppose that, along with a synthetic dataset, an attacker could access some demographic information about certain subjects in the original data. If r represents the proportion of original records known to the attacker, and p represents the proportion of attributes / features known to the attacker, parameters r and p can be varied to examine the method's performance in protecting / leaking source records. This is done by first generating a set of records similar to the compromised dataset by randomly sampling a small subset of records and features from the original dataset. Then, the nearest record (i.e., a nearest neighbor) for each compromised record is identified from the synthetic dataset, and missing values ​​are estimated to see how the method's synthetic data performs in protecting the privacy of the underlying source data.

[0061] Source-level privacy typically includes ensuring that data does not reveal its ownership attributes (e.g., ownership identity, number of sources, etc.). In one embodiment of the method of the present invention, the generation step includes subjects from multiple sources to reduce the identifiability of the sources.

[0062] result

[0063] fidelity and accuracy

[0064] Figure 4 Scatter plots were compared between MedGAN (Medical Generative Adversarial Network), RNN (Recurrent Neural Network), CTGAN (Conditional Generative Adversarial Network), and the present invention's original source multiple myeloma dataset (blue) and its synthetic data (orange) from 946 subjects with 40 features. (The scatter plots were visualized using t-SNE to reduce dimensionality.) Visually, the method of the present invention creates a synthetic dataset that overlaps well with the real dataset, even for small clusters.

[0065] Table 1 shows the method of the present invention (k=5) and Figure 4 The neural network method uses quantitative results of performance using several different univariate, bivariate, multivariate, supervised, and unsupervised metrics. As mentioned above, the silhouette coefficient is a multivariate, unsupervised metric that compares one cluster to another and quantifies the overlap between real and synthetic datasets. A coefficient closer to zero is better. The silhouette coefficient of the method in this invention is -0.001, which is significantly better than the other three methods.

[0066]

[0067] Table 1 also shows the AUC (Area Under the Curve) for real and synthetic data, which are multivariate supervision metrics. These metrics provide predictive accuracy—if one knows 39 out of 40 features, how well does the 40th feature predict? The closer the two AUCs are, the better. In this case, the AUC of the method of this invention differs by only 0.005, which is much better than the other three methods.

[0068] The next metric, the distance between BoW (Bag of Words) representations, is a multivariate, unsupervised indicator. In this case, "Bag of Words" corresponds to "Bag of Features," and a histogram is used to represent the frequency of all binned features in the data. Similar to the silhouette coefficient, the closer the distance between BoW representations is to zero, the better. In this case, the distance between the BoW representations of the present invention is 0.009, which is significantly better than the other three methods.

[0069] The next metrics are the Fisher Exact test and the KS (Kolmogorov-Smirnov) test. These tests are univariate metrics that compare the statistical distribution of each feature in the real and synthetic datasets. Fisher Exact is used for categorical features, and KS is used for numerical features. The p-value threshold for these tests is set to 5% (or 0.05). Table 1 shows the number of features with different statistical distributions in the real and synthetic datasets—the fewer the better. In this case, the 40 features tested using this invention do not have different statistical distributions in either the real or synthetic datasets, which is significantly better than any of the other three methods, where at least 12 features have different statistical distributions.

[0070] The last metric in Table 1 shows the accuracy of classifying synthetic datasets against real datasets. This metric measures whether the classifier is confused, where total confusion means a classification accuracy of only 50%, or 0.5. The metric ranges from 0.5 to 1.0, with closer to 0.5 being better. In this case, the accuracy of the method of this invention is only 0.61 (or 61%), which is significantly better than any of the other three methods, two of which (medGAN and CTGAN) are close to 100%.

[0071] Table 2 shows the average values ​​of various numerical features from the multiple myeloma dataset, comparing raw (e.g., real) data with synthetic data generated using medGAN, RNN, and the method of this invention. The baseline functional level of the subjects was their ECOG (Eastern Cooperative Oncology Group) performance status, an integer ranging from 0 (fully active) to 5 (dead), with intermediate level limits. For most features, the method of this invention yielded the best results.

[0072]

[0073] Table 3 shows the cross-validation results from the metrics in Table 1, illustrating how the method of this invention is performed on public and private datasets of varying sizes (number of records x number of features). The multiple myeloma dataset is the same as those appearing in Tables 1 and 2 (but the AUC and BoW values ​​may change each time a new synthetic set is generated). The breast cancer dataset is a proprietary dataset with the most records of all datasets. The other four datasets are publicly available from the UCI Machine Learning Repository. The method of this invention performs worst on the UCI lung cancer dataset because it is very small (i.e., <100 records) and the number of features is greater than the number of records, thus t-SNE and PCA perform poorly.

[0074]

[0075] Tables 4A and 4B use the SDGym benchmark to compare the performance of the method of this invention with 14 other methods on non-clinical, simulated, and real datasets. Table 4A includes seven simulated datasets, each with 10,000 records and 2 to 37 features. Table 4B includes eight real datasets with 22,500 to 481,000 records and 15 to 785 features. The method of this invention uses a clustering of k=5, and due to the large datasets, PCA is used to synthesize the datasets.

[0076] Green items indicate the best-performing synthesizer on each dataset, while red items indicate the worst-performing synthesizer; for green (red) items, the darker the shade, the better (worse) the performance. The method of this invention consistently outperforms other methods on most datasets: note that the results of the method of this invention are almost identical to those of the identity synthesizer. Although the data synthesized according to the method of this invention has high fidelity, a high degree of privacy is still preserved, as described in the following sections.

[0077]

[0078]

[0079]

[0080]

[0081]

[0082] Privacy at the subject level

[0083] Throughout the inventors’ experiments, the number of subjects who came from the original dataset and also appeared in the synthetic dataset was always zero; when generating synthetic data, the method of the present invention does not copy or remember records from the underlying source data.

[0084] Figure 5 The ROC (Receiver Operating Characteristic) curves for cluster sizes 2, 5, and 10 are shown. For cluster size 2, the AUC is 0.7, while for cluster sizes 5 and 10, the AUC is close to 0.5. Higher AUC generally indicates higher prediction accuracy. However, as mentioned earlier, privacy decreases as the cluster size decreases. For clusters of sizes 5 and 10, data attackers cannot distinguish between members and non-members (i.e., whether a piece of data is part of the original dataset), thus minimizing the possibility of leakage. Therefore, these are better values ​​to use if privacy is important.

[0085] Figure 6 The diagram illustrates the relationship between the proportion of data exposed to data attacks and r (the proportion of compromised records) and p (the proportion of compromised records containing features known to the attacker) for cluster sizes of 2, 5, and 10. The worst-case scenario is shown in the diagram above, where cluster size k = 2: even when 90% of the records are compromised (r = 0.9) and 90% of their features are compromised (p = 0.9), the attacker can uniquely recover up to 40% of the records, as indicated by reference numeral 602 in the figure. Figure 6 The intermediate figure, where k=5, shows that when 70% of the subjects are exposed (r=0.7) and 90% of their features are exposed (p=0.9), an attacker can uniquely recover up to 18% of the records, as shown in figure 605. Figure 6 The bottom plot, where k=10, shows that when 90% of the subjects are exposed (r=0.7) and 90% of their features are exposed (p=0.9), an attacker can uniquely recover up to 13% of the records, as indicated by reference numeral 610. Therefore, the privacy of synthetic data can be improved by increasing the cluster size. Since increasing the cluster size often reduces the quality (fidelity) of the generated synthetic data, we need to choose the k value that best balances fidelity and privacy based on the data type and application. The experiment used to generate... Figure 6 The data from the mid-curve (the multiple myeloma trial data discussed above) comprised 946 participants with 40 features. In practice, data attackers tend to know an average of 4-7 features about a participant, which, for this dataset, is approximately 10%-18% of the total feature size (or p-values ​​between 0.10 and 0.18). Therefore, for these p-values, Figure 6 It shows that if k=2, a very small amount of data can be shown, but if k=5 or 10, the amount is reduced to near zero.

[0086] Donor-level privacy

[0087] Figures 7A to 7C This is a t-SNE visualization of clinical trial data from 946 multiple myeloma participants. Figure 7A Including all 40 features from these subjects, the multiple myeloma data indicated that it came from three sources (i.e., three clinical trials). Visual separability of the data sources arose due to missing features in some datasets and differences in laboratory measurement units or patterns. To obfuscate the separability in the data, the inventors trained a random forest classifier to learn which source each record belonged to. Then, salient features were removed one by one until the data was no longer separable. Figure 7B The record shows the ability to partially separate the data source even after removing the eight most prominent features. Figure 7C The diagram shows records where the 23 most salient features were removed, rendering the data source unidentifiable on its own. Therefore, this invention provides a degree of protection at the source level, but involves a trade-off between reducing source identifiability and excluding features from the data.

[0088] In summary, this invention generates synthetic electronic health data based on valuable clinical trial data that has been unavailable until now. Clinical trial data typically resides in small datasets protected by laws and regulations and subject to privacy concerns. Due to the small size of these datasets, neural network algorithms cannot address all these challenges. Therefore, this invention enables those who lack direct access to the data due to these challenges to conduct research and innovation based on it.

[0089] This invention efficiently generates high-fidelity, synthetic, subject-level EDC data from all tabular data sizes, and is particularly suitable for smaller datasets (i.e., hundreds of records). The method is efficient, controllable, and traceable, capable of synthesizing any high-dimensional multivariate tabular data while highly protecting the privacy of subjects and donors / contributors / sources. Instead of learning high-dimensional conditional representations such as GANs or VAEs, this invention combines low-dimensional embeddings of the source data to identify the nearest neighbors of subjects in a compressed feature space and performs feature permutations and recombination among similar subjects. The synthesized data is statistically similar to the source data and captures the underlying dependencies of the source. This method does not memorize or retain records from the original data, nor does it expose any specific features of the original data, thus protecting subject privacy.

[0090] Furthermore, the generated data can be upsampled or downsampled from the original source data, or it can be precisely sized. Each generated record representing a subject can be mapped to a small cluster of subjects used to generate it. This allows for flexibility and the generation of data focused on the specific needs of the data user (e.g., researchers). It also allows for the flexible selection of the characteristics of the subjects for whom synthetic copies are made. Synthetic subjects can inherit characteristics from subjects from multiple contributors, thus protecting the privacy of contributors.

[0091] Another advantage is that the method of this invention makes no prior assumptions about the input data type (e.g., it can handle categorical and numerical features). The algorithm also makes no assumptions about the underlying distribution of the features. Furthermore, compared to other methods that require extensive fine-tuning, hyperparameter optimization, fitting, and / or setup, the method of this invention is lightweight, flexible, and easy to implement and deploy. It can automatically and efficiently handle features of mixed data types without any additional encoding and architectural changes, as required by some GANs. Its runtime is also much shorter compared to training a neural network.

[0092] This invention preserves the fundamental privacy of the real data evaluated by the aforementioned heuristic tests, while allowing for flexible data generation. For example, to customize or emphasize certain dependencies between specific characteristics in subjects—characteristics that may specifically reflect a subject having a rare disease—data users (e.g., researchers) may wish to synthesize subjects, and customized permutation functions can be designed after the embedding process to generate new subjects with the desired attributes.

[0093] Aspects of the invention may be implemented in the form of a system, a computer program product, or a method. Similarly, aspects of the invention may be implemented in hardware, software, or a combination of both. Aspects of the invention may be implemented as a computer program product stored on one or more computer-readable media in the form of computer-readable program code implemented on one or more computer-readable media.

[0094] A computer-readable medium can be a computer-readable storage medium. For example, a computer-readable storage medium can be an electronic, optical, magnetic, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any combination thereof.

[0095] The computer program code in these embodiments can be written in any suitable programming language. The program code can run on a single computer or on multiple computers. The computer may include a processing unit that communicates with a computer-usable medium, wherein the computer-usable medium contains a set of instructions, and wherein the processing unit is designed to execute that set of instructions.

[0096] The foregoing discussion is intended to illustrate the principles and various embodiments of the invention. Once the foregoing disclosure is fully understood, many variations and modifications will become apparent to those skilled in the art. This application is intended to interpret the claims as encompassing all such variations and modifications.

Claims

1. A method for generating a synthetic dataset comprising N records from an original electronic data acquisition dataset for clinical trials, wherein the original electronic data acquisition dataset for clinical trials contains n records of sensitive tabular data with mixed categorical and numerical features, wherein, The method includes: The classification features of the original clinical trial electronic data acquisition dataset are encoded into a numerical format and missing values ​​are estimated; Use dimensionality reduction techniques to embed the encoded dataset into a low-dimensional space; Select a seed record from the embedded dataset, and identify multiple nearest neighbor records of the seed record based on a distance metric; Statistical correlation analysis is used to detect one or more highly correlated feature groups among the features of the original clinical trial electronic data acquisition dataset, and the correlated features are jointly separated during synthetic record generation, such that the correlated features are inherited together. Synthetic records are generated by randomly arranging the features between the seed records and their nearest neighbor records within the same cluster, while retaining the related features that have been jointly separated. By determining the cluster size and arrangement parameters to balance privacy protection and statistical fidelity; The synthetic records are linked to the synthetic dataset, a new seed record is selected, and the identification, detection, generation, linking, and selection operations are repeated until N synthetic records are generated.

2. The method of claim 1, wherein the embedding includes using t-random neighborhood embedding.

3. The method of claim 1, wherein the embedding includes using uniform manifold approximation and projection.

4. The method of claim 1, wherein the embedding includes using principal component analysis.

5. The method of claim 1, further comprising adding noise to the numerical features to generate different feature values.

6. The method of claim 1, wherein the original clinical trial electronic data acquisition dataset contains n records and N = n.

7. The method of claim 1, wherein the original clinical trial electronic data acquisition dataset contains n records and N ≠ n.

8. A system for generating a synthetic dataset comprising N records from an original electronic data acquisition dataset for clinical trials, wherein the original electronic data acquisition dataset for clinical trials contains n records of sensitive tabular data with mixed categorical and numerical features, wherein, The system includes: The encoder is configured to encode categorical features into a numerical format and estimate missing values. Embedders are configured to embed encoded datasets into a low-dimensional space using dimensionality reduction techniques; A clusterer is configured to select a seed record from an embedded dataset and identify multiple nearest neighbor records of the seed record; A feature pairing detector is configured to use statistical correlation analysis to detect one or more highly correlated feature groups and to jointly separate the correlated features during synthetic record generation. A synthetic record generator is configured to randomly arrange the features between the seed record and its nearest neighbor record within the same cluster, while preserving the correlated features that have been jointly separated. A privacy-fidelity parameter selector is configured to determine cluster size and permutation parameters to balance privacy protection and statistical fidelity; and The controller is configured to connect the generated synthetic records to the synthetic dataset and repeatedly perform selection, identification, detection, generation and connection operations until N synthetic records are generated.

9. The system of claim 8, wherein the embedding includes using t-random neighborhood embedding.

10. The system of claim 8, wherein the embedding includes using a unified manifold approximation and projection.

11. The system of claim 8, wherein the embedding includes the use of principal component analysis.

12. The system of claim 8, wherein noise is added to the numerical features to generate different feature values.

13. The system of claim 8, wherein the original clinical trial electronic data acquisition dataset contains n records and N = n.

14. The system of claim 8, wherein the original clinical trial electronic data acquisition dataset contains n records and N ≠ n.

Images