Synthetic generation of data with many-to-many relationships

Abstract

Embodiments described herein relate to efficient generation of synthetic data sets representing many-to-many relationships. In particular, certain embodiments implement specific factorization for many-to-many generative models, which results in an extensible generative framework by combining random graph theses and representation learning. In other embodiments, we extend the framework to establish the concept of differential privacy within synthetic generated data. Embodiments described herein are thus able to efficiently generate a synthetic data set while preserving information within and across a many-to-many data set with improved accuracy.

Description

Technical Field

[0001] Embodiments described herein relate to computer systems and computer-implemented methods for generating synthetic data sets in a manner that preserves relationships within the original data while also protecting the privacy of information within the original data sets. Background Art

[0002] Advances in machine learning and data analytics have seen the value of raw data increase. As the value of raw data grows, maintaining data security becomes increasingly important, not only for security reasons, but also to maintain privacy and comply with local regulations.

[0003] Some computing systems have strict Chinese walls to protect information and comply with local regulations (for example, in the banking or healthcare sectors). As a result, many companies and research groups are unable to share raw, non-anonymous data due to the need to maintain user privacy. This can make data analysis across different data sources difficult. In addition, in some cases, personal data may be protected by law or regulation, and sharing this data between entities (for example, between businesses) or across borders may be prohibited.

[0004] One approach to address this problem is to create synthetic datasets based on the original datasets. Synthetic datasets contain information that is artificial rather than generated by real-world events. To be useful, synthetic datasets should have similar statistical properties to the original datasets. However, synthetic datasets should be generated in a way that preserves the privacy of user data in the original datasets.

[0005] In several applications, the original data set is contained in a database with multiple tables. Various relationships may exist between the attributes of each table. The attributes may have a one-to-one relationship, a one-to-many relationship, and / or a many-to-many relationship. Summary of the invention

[0006] The present application provides an improved method for efficiently generating synthetic data representing many-to-many relationships. This method can be adapted to ensure privacy, and in particular, differential privacy.

[0007] Previous methods for generating synthetic data from raw data sources containing many-to-many relationships have been found to lack scalability and / or applicability to a wide range of datasets. In particular, using previous methods, users have had to choose between ignoring the many-to-many relationships in the real data or sacrificing the level of privacy in the synthetic data. In view of this, a new method for generating synthetic data from datasets containing many-to-many relationships is desired.

[0008] In particular, when generating synthetic data with many-to-many relationships that are computationally more efficient, certain embodiments described herein apply a specific order to the generation of relationships and values. For example, an embodiment generates a relationship before generating values ​​in two related tables (via the generation of edges between the two tables). This change reduces the complexity of the generation process, thereby making the generation scalable and making the entire pipeline easier to implement in practice. It should be noted that this change is not simple because it requires the use of techniques that are completely different from those previously implemented, including random graph generation from graph theory and graph / node representations from the representation learning literature. In practice, the change results in the generation of synthetic data with many-to-many relationships up to 10 times faster than previous methods. In addition, the methods described herein can be implemented while maintaining control of fidelity and privacy.

[0009] According to a first aspect, a computer-implemented method for generating a synthetic data set from a first data set is provided, wherein: the first data set includes a first table (Table A) containing a first plurality of entries and a second table (Table B) containing a second plurality of entries; the synthetic data set includes a third table (Table A') containing a third plurality of entries and a fourth table (Table B') containing a fourth plurality of entries. The method includes: generating a composite graph based on multiple associations between the first multiple entries in the first table (Table A) and the second multiple entries in the second table (Table B), wherein the composite graph includes a first group of nodes (U'), a second group of nodes (V') and a first group of edges (L'); using a first conditional model (p(U|L))) conditioned on the composite graph to determine one or more attributes associated with each node in the first group of nodes (U'); and using a second conditional model (p(V|L,U))) conditioned on the composite graph and the one or more attributes associated with each node in the first group of nodes (U') to determine one or more attributes associated with each node in the second group of nodes (V'); wherein: the third multiple entries correspond to the one or more attributes associated with each node in the first group of nodes (U'), and the fourth multiple entries correspond to the one or more attributes associated with each node in the second group of nodes (V'). In an embodiment, conditioning on the composite graph and the one or more attributes associated with each node in the first set of nodes (U') may be conditioning on all or at least some of the attributes associated with the first set of nodes.

[0010] In an embodiment, generating the composite graph based on the multiple associations between the entries in the first table (Table A) and the entries in the second table (Table B) includes: generating the first set of edges (L') so that the first set of edges (L') has the same joint distribution of multiple pairs of node degrees as the second set of edges in the second graph representing the first data set.

[0011] In an embodiment, generating the synthetic graph based on the plurality of associations between the entries in the first table (table A) and the entries in the second table (table B) comprises: obtaining a bipartite joint degree distribution (P J ), where the bipartite joint degree distribution (P J ) represents the multiple associations between the entries in the first table (Table A) and the entries in the second table (Table B); modifying the bipartite joint degree distribution (P J ) to generate a modified bipartite joint degree distribution (P J ) + noise); from the modified bipartite joint degree distribution (P J )+noise) to sample a target number of edges to generate a joint degree matrix of the synthetic data; and generate the synthetic graph that conforms to the joint degree matrix of the synthetic data.

[0012] In an embodiment, the bipartite joint degree distribution is a joint degree matrix that gives the number of edges between degree i and degree j vertices for each pair (i,)j).

[0013] In an embodiment, generating the synthetic graph conforming to the joint degree matrix of the synthetic data comprises generating a synthetic graph having the same joint degree distribution as represented by the joint degree matrix of the synthetic data.

[0014] In an embodiment, generating the composite graph based on the joint degree matrix comprises using a 2-k generator.

[0015] In an embodiment, the bipartite joint degree distribution (P J ) includes: representing the multiple associations between the entries in the first table (table A) and the entries in the second table (table B) as a bipartite graph (B); and calculating the bipartite joint degree distribution (P J ).

[0016] In an embodiment, generating the synthetic graph that conforms to the joint degree matrix includes: generating a bipartite graph having a joint degree distribution that conforms to the joint degree matrix of the synthetic data; and changing the edges within the bipartite graph to remove loop edges and / or remove multiple edges while maintaining the joint degree distribution of the joint degree matrix that conforms to the synthetic data.

[0017] In an embodiment, the bipartite graph is generated based on random pairing.

[0018] In an embodiment, the bipartite joint degree distribution (P J ) to generate the modified bipartite joint degree distribution (P J )+noise) includes: injecting additive noise into the bipartite joint degree distribution (P J ).

[0019] In an embodiment, the bipartite joint degree distribution (P J ) is a matrix, and wherein the additive noise is injected into the bipartite joint degree distribution (P J ) further includes: using a Laplace mechanism to calculate a noise value for an entry in the matrix to ensure ε differential privacy, where ε is a first parameter associated with the differential privacy amount (ε); and adding the noise value to the entry.

[0020] In an embodiment, calculating the noise value using a Laplace mechanism includes: modeling the probability of the value of the noise value as a Laplace distribution; and sampling from the Laplace distribution to determine the value of the noise value.

[0021] In an embodiment, the noise value is based on Calculate, where S ij =4max(d i ,d j ), where i is the row of the entry in the matrix and j is the column of the entry in the matrix, (d i ,d j ) is the position of the entry in the matrix, and Lap is the Laplace mechanism.

[0022] In an embodiment, determining one or more attributes associated with each node in the first group of nodes (U') using the first conditional model (p(U|L)) conditioned on the synthetic graph includes: determining the node embedding of each node in the first group of nodes (U') based on the first node embedding model (β) and the synthetic graph; and determining the one or more attributes associated with each node in the first group of nodes (U') using the first conditional model ((p(U|L))) and the node embedding of each node in the first group of nodes (U').

[0023] In an embodiment, determining the one or more attributes associated with each node in the first set of nodes (U') using the first conditional model ((p(U|L)) and the node embedding of each node in the first set of nodes (U') includes inputting each node embedding into the first conditional model.

[0024] In an embodiment, determining the node embedding of each node in the first group of nodes (U') based on the first node embedding model (β) and the synthetic graph includes: obtaining the first node embedding model (β), wherein the first node embedding model (β) is configured to represent the nodes of the graph as vectors; and determining the node embedding of each node in the first group of nodes (U') using the first embedding model (β) and the synthetic graph.

[0025] In an embodiment, the first node embedding model (β) is based on one of: a) the number of neighbors in the synthetic graph within a given distance d of the embedded node; b) a DeepWalk model; or c) a node2vec model. In an embodiment, the node embedding model (β) may be an embedding neural network trained to generate embeddings of nodes of a graph.

[0026] In an embodiment, determining the one or more attributes associated with each node in the first group of nodes (U') using the first conditional model ((p(U|L))) conditioned on the synthetic graph includes: obtaining the first conditional model ((p(U|L)); and determining the probability distribution of each attribute associated with the node using the first conditional model ((p(U|L))) and the node embeddings associated with the nodes in the first group of nodes (U'); and sampling from the probability distribution to obtain each attribute associated with the node.

[0027] In an embodiment, obtaining the first conditional model (p(U|L)) includes: extracting a first set of attributes from the first table (Table A); representing the multiple associations between entries in the first table (Table A) and entries in the second table (Table B) as a second bipartite graph (B), wherein: the second graph includes a third set of nodes (U) and a second set of edges (L) associated with the first set of attributes; determining a third set of node embeddings including node embeddings of each node in the third set of nodes; and generating the first conditional model (p(U|L)) based on the first set of attributes and the third set of node embeddings.

[0028] In an embodiment, generating the first conditional model (p(U|L)) based on the first set of attributes and the third set of node embeddings includes: fitting the first conditional model on the first set of attributes conditioned on the third set of node embeddings.

[0029] In an embodiment, fitting includes training, i.e., adjusting the parameters of the conditional model to optimize the objective function. The training may include stochastic gradient descent. The stochastic gradient descent may be based on a condition that ensures differential privacy. The condition may specify ∈ differential privacy.

[0030] In an embodiment, determining one or more attributes associated with each node in the second set of nodes (V') using a second conditional model (p(V|L,U))) conditioned on the composite graph and the one or more attributes associated with each node in the first set of nodes (U') comprises: determining one or more attributes associated with each node in the second set of nodes (V') based on a second node embedding model (γ) and neighbors (ne(n v))) by determining a second node embedding for each node in the second set of nodes (V'); determining the one or more attributes associated with each node in the second set of nodes (V') using a second conditional model (p(V|L,U)) and the second node embedding for each node in the second set of nodes (V').

[0031] In an embodiment, based on the second node embedding model (γ) and the neighbors (ne(n v )) and determining the second node embedding of each node in the second set of nodes (V') includes: obtaining the second node embedding model (γ), wherein the second node embedding model (γ) is configured to represent the nodes of the graph as vectors; using the second embedding model (γ) and the neighbors (ne(n) of each node in the second set of nodes (V') v )) determines the second node embedding of each node in the second set of nodes, wherein: the neighbors (ne(n v The information associated with the second set of nodes (V') includes the neighbor (ne(n v )))'s one or more attributes associated with each node in the first set of nodes (U').

[0032] In an embodiment, determining a second node embedding of a third node in the second set of nodes (V') comprises: determining a third set of nodes (ne(n)) in the first set of nodes (U') that are neighbors of the third node v )); and based on the third set of nodes (ne(n v The second node embedding of the third node is determined based on the one or more attributes associated with each node in ))).

[0033] In an embodiment, the second embedding model (γ) and the neighbors (ne(n) of each node in the second set of nodes (V′) are used. v )))) Determining the second node embedding of each node in the second set of nodes includes determining the neighbors (ne(n) of each node in the second set of nodes (V') v )))'s aggregation of one or more attributes.

[0034] In an embodiment, said aggregation is said node embedding.

[0035] In an embodiment, said neighbours of a node in said second set of nodes are said nodes' "parent nodes", which are nodes in said first set of nodes that are (directly) linked / connected to said corresponding node in said second set of nodes.

[0036] In an embodiment, the second node embedding of each node in the second set of nodes is a fixed size vector.

[0037] In an embodiment, the second node embedding model (γ) is an aggregator based on one of the following: a) summary statistics including one or more of counts, sums, n-th order moments, or q% quantiles of a selected q set spanning [0,)1]; b) a DP copula model; or c) a deep set model.

[0038] In an embodiment, determining the one or more attributes associated with each node in the second group of nodes (V') using a second conditional model (p(V|L,U)) and the second node embedding of each node in the second group of nodes (V') includes: obtaining the second conditional model (p(V|L,U)); determining a second probability distribution for each attribute associated with the node using the second conditional model (p(V|L,U)) and the second node embedding associated with the nodes in the second group of nodes (V'); and sampling from the second probability distribution to obtain each attribute associated with the node.

[0039] In an embodiment, obtaining the second conditional model (p(V|L,U)) includes: extracting a second set of attributes from the second table (Table B); representing the multiple associations between entries in the first table (Table A) and entries in the second table (Table B) as a third bipartite graph (B), wherein: the third bipartite graph includes a fourth set of nodes (V) associated with the second set of attributes; determining a fourth set of node embeddings including node embeddings of each node in the fourth set of nodes (V), wherein the node embeddings of each node in the fourth set of nodes (V) represent attributes of neighbors of each node in the fourth set of nodes (V); and generating the second conditional model (p(V|L,U)) based on the second set of attributes and the fourth set of node embeddings.

[0040] In an embodiment, the neighbor is a parent node of the node.

[0041] In an embodiment, determining the fourth set of node embeddings further includes: determining a multiset representation of each node in the fourth set of nodes (V) based on the attributes of the neighbors of each corresponding node in the fourth set of nodes (V); and calculating the fourth set of node embeddings based on the multiset representation of each node.

[0042] In an embodiment, generating the second conditional model (p(V|L,U)) based on the second set of attributes and the fourth set of node embeddings includes: fitting the second conditional model on the second set of attributes conditioned on the fourth set of node embeddings.

[0043] In an embodiment, fitting includes training, i.e., adjusting the parameters of the conditional model to optimize an objective function. The training may include stochastic gradient descent according to a condition that ensures differential privacy. The condition may specify ∈ differential privacy.

[0044] In an embodiment, the first conditional model and the second conditional model are each differentially private (DP) conditional models.

[0045] In an embodiment, generating the synthetic graph comprises generating the synthetic graph based on a first learnable parameter (θ1), and the method further comprises determining a value of the first learnable parameter (θ1).

[0046] In an embodiment, the first learnable parameter affects the edges in the graph associated with the first dataset to the bipartite joint degree distribution (P J )’s mapping.

[0047] In an embodiment, the method further includes: learning the value of a second learnable parameter (θ2) so that the first conditional model (p(U|L))) satisfies a criterion, wherein: the second learnable parameter (θ2)) includes parameters of the first conditional model (p(U|L)); and the criterion is that the error between the output of the first conditional model p(U|L) and the node value associated with the first data set is minimized according to a differential privacy criterion.

[0048] In an embodiment, the second learnable parameters (θ2) further include a third set of parameters associated with the first node embedding model (β).

[0049] In an embodiment, the method further includes: learning the value of a third learnable parameter (θ3) so that the second conditional model (p(V|L,U))) satisfies a criterion, wherein: the second learnable parameter (θ3)) includes parameters of the second conditional model (p(V|L,U))); and the criterion is that the error between the output of the second conditional model (p(V|L,U)) and the node value associated with the first data set is minimized according to a differential privacy criterion.

[0050] In an embodiment, the third learnable parameters (θ3) further include a fourth set of parameters associated with the second node embedding model (γ).

[0051] In an embodiment, the conditional model comprises a conditional generative model.

[0052] In an embodiment, the attributes of the first table and the second table are associated through a many-to-many relationship.

[0053] In an embodiment, the synthetic graph models relationships in the synthetic dataset.

[0054] In an embodiment, the first data set comprises real data generated by humans.

[0055] According to a second aspect, a computer-implemented method for generating a synthetic data set from a first data set is provided, wherein the first data set includes multiple tables, and the method further includes: generating synthetic data for each combination of tables in the multiple tables in the first data set according to the method described in any of the aforementioned technical solutions.

[0056] According to a third aspect, there is provided a non-transitory computer-readable medium comprising computer program instructions adapted to be executed by a processor, the instructions being configured to perform the above-described method when executed by the processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The arrangement of the present invention will be more fully understood and appreciated from the following detailed description, which is given by way of example only and taken in conjunction with the accompanying drawings, in which:

[0058] Figure 1 presenting a method for generating a synthetic data set from a first data set;

[0059] Figure 2 presenting a method for generating synthetic data according to an embodiment;

[0060] Figure 3 Demonstrates a method for generating a composite graph for a pair of tables in an input dataset; and

[0061] Figure 4 A computing system is shown for implementing the embodiments described herein. DETAILED DESCRIPTION

[0062] In synthetic data generation (SDG), a generative model is fitted on the original data, and samples are extracted from the model to generate synthetic data. The synthetic data can then be published as protected synthetic data, reflecting the relationships within the original data and protecting the privacy within the original data. Although real-world datasets often consist of multiple tables with potential many-to-many relationships (i.e., many-to-many datasets), some SDG methods focus on modeling tables independently. Other methods only consider generating datasets with special cases (e.g., one-to-many) of many-to-many relationships.

[0063] Embodiments described herein relate to efficient generation of synthetic datasets representing many-to-many relationships. Specifically, certain embodiments implement a specific factorization for many-to-many generative models, which results in a scalable generative framework by combining random graph theory and representation learning. In other embodiments, we extend the framework to establish the concept of differential privacy within synthetically generated data. Embodiments described herein are therefore able to efficiently generate synthetic datasets while preserving information within and across many-to-many datasets with improved accuracy.

[0064] Figure 1 A method for generating a synthetic data set from a first data set is presented.

[0065] The first data set includes a first plurality of entries (Table A) and a second plurality of entries (Table B). The first plurality of entries (Table A) and the second plurality of entries (Table B) have a plurality of associations therebetween.

[0066] In an example, the first data set is a real data set (e.g., a data set containing "real" data that is observed rather than algorithmically generated). The first data set may be referred to as a raw data set. In this context, the terms "real" and "raw" may mean that it is not synthetic data (e.g., it is real-world data). The real or raw data may have had previous operations or calculations performed on it and may be the result of previous operations or calculations (e.g., filtering). The first data set may be stored in a memory or may be received (e.g., such as from an external source via a network).

[0067] There are various use cases for the methods described in this article for generating synthetic datasets.

[0068] In a first use case, a first data set includes customer banking data, including but not limited to: customer location, customer age, and / or customer financial information (including account balances). Using customer banking data to train machine learning models can be challenging due to the need to protect the privacy of user data and the need to comply with various regulatory requirements. Applying the methods described herein enables the generation of a synthetic data set that is statistically similar to the first data set (e.g., includes the same or similar attributes, relationships, or correlations between data in the first data set), but protects the privacy of the customer. Furthermore, applying the methods described herein enables many-to-many relationships within the first data set to be captured in the synthetic data set.

[0069] Advantageously, clients such as banks and consulting firms require accurate synthetic data sets in order to move their data around securely and quickly (either internally between different branches or to share it with external partners).

[0070] In a second use case, the first data set includes customer order information, including but not limited to: information identifying the items ordered and information identifying the supplier of the order. Applying the methods described herein enables the generation of a synthetic data set that is statistically similar to the first data set but protects the privacy of the data set. Relationships in real data can also leak private information if you are not careful. As a simple example, in a customer-merchant data set, the number of links to a merchant may leak its identity due to its unusual popularity. Therefore, while it is possible to generate synthetic data to allow further analysis (such as training of additional machine learning models), preserving relationships within the synthetic data too accurately can lead to private data leakage. Therefore, it is important to make this process differentially private in order to avoid leakage of private information. In addition, applying the methods described herein enables the capture of many-to-many relationships within the first data set in the synthetic data set.

[0071] In a third use case, the first data set contains manufacturing data, including but not limited to sensor readings obtained by measuring physical objects and / or processes. Applying the methods described herein enables the generation of a synthetic data set that can be shared with a third party organization for the purpose of analysis and process improvement. Furthermore, applying the methods described herein enables the capture of many-to-many relationships within the first data set in the synthetic data set.

[0072] In a fourth use case, the first dataset contains personal medical information, including but not limited to: patient number, patient age, medical condition, and / or patient test / measurement results. Applying the methods described herein enables the generation of a synthetic dataset that can be shared with a third party for the purpose of analyzing trends in the medical information and identifying treatments for the patient while also protecting the patient's privacy. Furthermore, applying the methods described herein enables the capture of many-to-many relationships within the first dataset in the synthetic dataset.

[0073] It should be appreciated that the methods described herein may be applied to other use cases.

[0074] Using the techniques described in this article, clients can privately publish sensitive data with many-to-many relationships without having to give up the many-to-many relationships or risking leaking private information.

[0075] In an example, a database stores a first data set (e.g., a real data set containing values ​​obtained by direct measurement). The database represents the first data set as a plurality of tables. A table is a data structure that organizes data into columns and rows. Each row in a table is associated with a record and has a unique row index. Each column in a table is associated with a data field and each column is typically associated with a label (e.g., "first name," "last name," etc.).

[0076] The following shows an example database that stores the original dataset in two tables:

[0077]

[0078] In this example: The rows of table A are: {1,) A}, {2,) B}, {3,) C}, {4,) A}, {5,) C}. The rows of table B are: {1,) car}, {2,) book}, {3,) table}, {4,) book}, {5,) car}.

[0079] Entries (or records) in Table A are associated with entries (or records) in Table B through columns of the table (e.g., the "Order Number" field). The links (or "associations") between Table A and Table B are: {A,) car}, {B,) book}, {C,) table}, {A,) book}, {C,) car}. As can be seen from Table A, the Order Number field is a unique identifier in the sense that no orders share the same order number. However, the Customer ID is not unique because two separate orders can be placed by the same customer (e.g., Orders 1 and 4 were both placed by Customer 'A').

[0080] Each row of a table is associated with an attribute. The attributes associated with a row of a table are the values ​​of the data fields / columns. For example, the attributes associated with the first row of table A are: {A}. The attributes do not include an index for uniquely identifying each row in the table or a field for generating links between tables (which, in the above example, is the "Order Number" field). In instances where a table includes more than one data field / column, then the attributes associated with each row take the form of a D-dimensional matrix, where D is the number of data fields associated with each row / index in the table.

[0081] Attributes from a pair of tables can be represented by a one-to-one relationship, a one-to-many relationship, a many-to-one relationship, or a many-to-many relationship.

[0082] In a one-to-one relationship, an attribute in the first table is associated with one and only one attribute in the other table. In a one-to-many relationship, one attribute in the first table is associated with multiple attributes in the second table, but each of the multiple attributes in the second table is uniquely associated with one attribute in the first table. In a many-to-many relationship, multiple attributes in the first table are associated with multiple attributes in the second table.

[0083] In the example shown above, there is a many-to-many relationship between the 'Customer ID' field and the items ordered, because a single customer ID can order different items (e.g. customer ID 'A' orders 'Car' and 'Book'), and a single item can be ordered by different customers (e.g. 'Car' is ordered by customers 'A' and 'C').

[0084] like Figure 1As shown in , a synthetic data set can be generated that has a distribution that matches the distribution of the first data set (eg, shares some properties of the first data set). The synthetic data set can include a third plurality of entries (table A), a fourth plurality of entries (table B'), and a first set of edges L'.

[0085] In the embodiments described herein, a graph is used to represent a data set. A graph is a mathematical structure used to model relationships between objects. A graph includes nodes and edges connecting nodes. In the examples discussed herein, reference is made to a synthetic graph. A synthetic graph is a graph that models a synthetic data set. Thus, a synthetic data set may be represented by a synthetic graph, where each entry is a node and where the relationship between the third plurality of entries (table A') and the fourth plurality of entries (table B') is represented by an edge.

[0086] Certain embodiments described herein utilize bipartite graphs. A bipartite graph is a graph whose vertices (ie, nodes) can be partitioned into two disjoint and independent sets U and V, such that each edge connects a vertex in set U to a vertex in set V.

[0087] Therefore, if Figure 1 , the third plurality of entries (table A') may be the first set of nodes U', and the fourth plurality of entries (table B') may be represented by the second set of nodes V'. A first set of edges L' may connect the first set of nodes U' to the second set of nodes V', such that the edges connect vertices in set U' to vertices in set V'.

[0088] In the following description, reference is made to one or several neighbors of a node in a set of nodes. It should be understood that in a bipartite graph, the neighbors of a node in a set of nodes (e.g., in V') are the nodes in another set (i.e., in U') that are connected to that node by edges. These neighboring nodes may also be referred to as "parent nodes" of the nodes in V' (which are "child nodes").

[0089] In the embodiments described herein, reference is made to node embedding. Node embedding is a way of representing the nodes of a graph as vectors. The node embedding model includes the process of transforming (or mapping) the information associated with the nodes of the graph into a vector identifying the node.

[0090] In the examples described herein, reference is made to conditional models (also called conditional generative models or conditional probability models). A conditional model is a model that predicts the probability of an event occurring given that another event has occurred.

[0091] In some examples, the method uses multiple differentially private condition models.

[0092] In an example, the method can be configured to generate data according to a given privacy level. This adjusts the resolution of the data output and may be affected by the amount of randomness introduced when generating the synthetic data. Differential privacy provides robust, mathematically provable privacy protection guarantees. In the context of differential privacy, a process or function is private if removing an individual user's data from the input data set does not result in a loss of privacy beyond a predefined amount.

[0093] In the example, the method is used to establish (∈,)δ) differential privacy, also known as (∈,)δ)-DP or approximate DP. If the following is the case, then there is a synthetic algorithm A)(where A2 is the sampling process, A1 is the learning algorithm) conditional model p θ It is (∈,)δ)-DP:

[0094] For all And for all B, B' that differ in a single element:

[0095]

[0096] In the examples described herein, fitting the probabilistic model includes training the probabilistic model.

[0097] The difference between the method described in this article and previous methods is the order in which the table rows and links are generated: some examples described in this article first generate random synthetic links that unconditionally share the same joint degree distribution as the real data by sampling the target bipartite joint degree matrix fed to the 2K generator and then condition the links through the graph / node embedding to generate rows. In contrast, alternative methods implement these steps in the opposite order. It should be noted that this change is not simple because it requires the use of completely different techniques, including random graph generation from graph theory and graph / node representations from the representation learning literature. This change makes the generation scalable and the entire pipeline easy to implement in practice. For customers, they can generate 10 times more data quickly while controlling fidelity and privacy; for software manufacturers, the code is easy to maintain due to this easy-to-implement design.

[0098] Factorization

[0099] To simplify the discussion, we first focus on modeling bipartite graphs and will expand them to multipartite graphs later. For the joint distribution p(U, V, L) we are interested in, where U and V are two tables with rows linked by an edge in L, considering that the roles of U and V are interchangeable, the joint p(U, V, L): = p joint There are three possible factorizations:

[0100] Factorization S: p joint,S=p(U)p(L|U)p(V|U,L)

[0101] Factoring E:

[0102] p joint,E =p(U)p(V|U)p(L|U,V)

[0103] Factoring A:

[0104] p joint,A =p(L)p(U|L)p(V|U,L)

[0105] Embodiments described herein utilize factorization A in order to more efficiently model multiple dependencies.

[0106] A total of 6 sub-models are considered within all factorizations:

[0107] p(U) (single table model) can be any generative model for single table generation. This model generates table U.

[0108] p(L|U) (semi-conditional edge model) generates edge L based on table U.

[0109] • p(V|U, L) (conditional table model) generates each node in the second table V based on the first table U and the edge L. The model may generate each node in V to be connected via L to a respective subset of nodes in U.

[0110] p(V|U) (table conditional model) generates a second table V conditioned on the first table U. The implementation of this model can be computationally challenging. While it can be implemented by directly modeling the union of the two tables p(U,V) and applying a generative model on the union, this can result in tables of unmanageable size, and it can be difficult to reverse the union operation to recover the individual tables from the synthetic data.

[0111] p(L|U,V) (edge ​​prediction model) predicts the existence of an edge between node pairs from two tables U and V.

[0112] p(L) (edge ​​model) generates edges unconditionally. This form of random graph model is studied in random graph theory.

[0113] p(U|L) (edge-conditional model) generates a table U given a set of edges L. This can be implemented using node embeddings.

[0114] Factoring S and E has several drawbacks. For example, for factoring S, there is no simple way to extend p(L|U) to the general many-to-many case. Factoring E has the scalability problem of p(V|U)) as described above (or similarly p(U,V)).

[0115] In view of the above, embodiments described herein implement a factorization A that provides an efficient method for modeling many-to-many relationships for synthetic data generation.

[0116] Figure 2 A method 100 for generating synthetic data according to an embodiment is shown. First, a synthetic graph is generated 110 based on associations between entries in a first table A and a second table B. The graph includes a first set of nodes U', a second set of nodes V', and a first set of edges L'. Next, one or more attributes of each node in the first set of nodes U' are determined 130 using a first conditional model p(U'|L') conditioned on the synthetic graph. Next, one or more attributes of each node in the second set of nodes V' are determined using a second conditional model p(V'|L',U') conditioned on the synthetic graph and one or more attributes associated with each node in the first set of nodes.

[0117] Method 100 may be repeated for each pair of tables in the data set to be synthesized. Three steps within method 100 shall be described in more detail below.

[0118] Stage 1) (Step 110)

[0119] During the first phase, a graph is generated based on a pair of tables A, B modeled (from real world / raw data). The graph contains nodes of a first synthetic table A', nodes of a second synthetic table B', and edges L' connecting the two sets of nodes. In this phase, only the graph topology is generated, so that no attributes are associated with each node. The pseudocode for generating the synthetic graph is detailed in Algorithm 1 below. Algorithm 1 is a revised version of the bipartite 2K generator described in Asma Azizi Boroojeni, Jeremy Dewar, Tong Wu, and James M. Hyman, “Generating bipartite networks with a prescribed joint degree distribution,” Journal of complex networks, 5(6): 839-857, 2017, which is hereby incorporated by reference in its entirety.

[0120] To generate a synthetic graph, an embodiment obtains the bipartite joint degree distribution of the input data and generates a random graph that satisfies this distribution.

[0121] Figure 3 Shows a method for generating a composite graph for a pair of tables in an input dataset. This method can be repeated for each pair of tables.

[0122] First, the bipartite joint degree distribution J112 of the input data is obtained. This can be obtained by extracting the links between the rows of the table, representing the links as a bipartite graph, and calculating the bipartite joint degree distribution (BJDD) of the graph.

[0123] If the synthetic data is to be anonymized, noise may be injected into the BJDD 114. For example, additive noise may be injected into the distribution (eg, using a Laplace mechanism). The noise added to each entry may be conditioned on the maximum of the row and column marginal distributions.

[0124] Next, a target number of edges are sampled from the joint degree distribution based on the size of the expected data set to generate a joint degree matrix 116 .

[0125] Next, a synthetic graph 118 is generated that satisfies the sampled joint degree matrix. The target number of edges may be sampled from a bipartite joint degree distribution (PJ). This may include extracting n edges from Pj, where n is the target number (e.g., defined by a user or based on the size of the desired data set (i.e., the size of the synthetic data set)). In an example, the sampling is random sampling.

[0126] Synthetic graphs can be generated using the dK generator method to model the probability p(L') of the edge. Specifically, a 2k distribution can be used. This involves a dK distribution, where d)=)2. In this method, d represents the size of the graph being modeled. A dK graph is a set of graphs constrained by a given value of the dK distribution. Generating a series of 0K graphs for a given input graph only requires reproducing the average node degree of the original graph. Generating a series of 1K graphs requires reproducing the node degree distribution of the original graph, i.e., a 1K distribution. A 2K graph reproduces the joint degree distribution of the original graph, i.e., a 2K distribution. This is the probability of randomly selecting a node with a link connection degree k, and k is defined as the (joint) degree distribution of all d node pairs in the graph.

[0127] All graphs with a given dK distribution are called dK graphs. For example, 0K graphs share the same graph density, 1K graphs share the same node degree distribution, 2K graphs share the same joint distribution of node pairs, and so on. The model that samples dK graphs is called a dK generator. This method applies the same concept of bipartite graphs, defines a bipartite dK graph, and learns to generate the distribution p of the bipartite 2K graph of the input data. θ .

[0128] For a bipartite graph B, its bipartite joint degree distribution (BJDD) is defined as the empirical distribution of samples where n(i) denotes the node with index i (similarly, i(n) is the node index of node n) and deg(n) is the degree of node n. This empirical distribution can be represented as the matrix P J And the two-dimensional support d u ,d vThe (marginal) degree portion of the upper or lower node can also be calculated from the BJDD. For example, the upper node p u The (marginal) degree distribution of is a probability vector (a vector whose elements sum to 1) where its i-th element p u [i] = ∑ j P J [i,j] / d u [i].

[0129] The composite graph can be generated based on an iterative process. First, a bipartite graph can be generated based on random pairings to match the desired joint distribution. This step involves first creating a stub for each node based on the node degree; the stub for a node with index i and degree d is a list After the stubs for all nodes are created, they are randomly paired based on the joint degree distribution (to match the desired joint degree distribution). This can be achieved by enumerating the joint degree counts. At each iteration of a pair of degrees d1, d2, a d1 stub can be randomly sampled at degree d1, and a d2 stub can be randomly sampled at degree d2. These stubs can be paired accordingly. After this, the graph matches the distribution, but potentially has duplicate edges.

[0130] Next, a rewiring process can be performed to make the graph simple while preserving the joint degree distribution. A graph is called simple if there are no cycles or multiple edges.

[0131] For a given data set with multiple tables, the method described herein can be repeated to generate edges for each pair of tables. An arbitrary order can be applied to the generation of each pair of tables. The edges of each pair of tables can be generated according to this order. When generating a graph involving tables with more than one connection, there is a risk that an infinite loop may be implemented during the rewiring process. To avoid this, the method can terminate after a predefined maximum number of iterations.

[0132] Learning the model for generating synthetic graphs involves only computing the BJDD matrix from the data graph. If the table is involved in multiple edges, then the joint distribution of all degrees of each node is learned. Sampling from the model is done by sampling from P J Sampling is then defined by Algorithm 1.

[0133]

[0134] Phase 2) (Step 130)

[0135] In the second stage 130, a first set of properties for a first set of nodes is generated based on the generated composite graph. In one embodiment, this involves modeling the upper node U' conditioned on the edge L'. This distribution essentially models a set of nodes given a specific topology, which can be implemented by node embedding as:

[0136]

[0137] where β is a node encoder that computes the node embedding of node i in the graph L'. Its range is Among them, H β is the dimension of the node embedding. Intuitively, β computes a representation of the local topology of node i in L'; if the representation is rich enough, then given the representation of any i, j, and is conditionally independent. Thus, once β is given, this distribution can be modeled by any existing conditional generative model. Several for β are:

[0138] Node statistics: d-step neighbor count (number of neighbors with distance d for a chosen d). As a simple example, d)=)1 means using the node degree as 1d embedding. This can be generalized to using β The set of counts of different d, given by size H β Embedding.

[0139] Node embedding: An embedding model can be fit to the data and used for β. Examples of potential embedding models include DeepWalk and node2vec. The model can be fit using stochastic gradient descent. To ensure privacy, differentially private stochastic gradient descent (DP-SGD) can be used.

[0140] Thus, a method for generating a first set of attributes may include:

[0141] 1. For a pair of tables (A and B), extract the attributes of A and the links between the rows of A and B and represent the links as a bipartite graph

[0142] 2. Compute the node embedding for each node in table A of the graph using any method that computes node embeddings, i.e. a method that accepts a graph and nodes and outputs a fixed-size vector representing the topology of the nodes in the graph. This gives a synthetic node representation. Optionally, this can be done in a differentially private way to ensure privacy.

[0143] 3. Conditioned on the composite node representation, sample rows of Table A using a generation pattern configured to generate attributes conditioned on the composite node representation.

[0144] The generative model may fit attribute A) (e.g., may be trained based on attribute A), conditioned on the computed node representation. The conditional generative model may be a conditional differentially private (DP) generative model. Fitting may involve adjusting one or more parameters of the model to optimize an objective function. For example, the objective function may aim to minimize the error (or difference) between the generated attribute A' and attribute A. If this is implemented in a differentially private manner, then this optimization may include a condition on a minimum differential privacy level.

[0145] Stage 3) (Step 150)

[0146] In stage 3, a second attribute 150 is generated based on the first attribute and the composite graph.

[0147] Consider node n v ∈V, where the neighbors It can be seen that given ne(n v ), n v Conditionally independent of any other n′ v ∈V. Therefore, the properties of the second node conditioned on the composite graph and the first node can be generated based on the probability distribution:

[0148]

[0149] Where p(n v |ne(n v ) is a conditional distribution whose condition is a multiset—a set that allows repeated elements. v ) imposes any order, one way to condition this is via a fixed-size set embedding computed by the aggregator γ. This yields the following model:

[0150]

[0151] This model requires an aggregator: where pow(U′) is the power set of U′, and H γ is the dimension of the set embedding. If the set embedding is rich enough, then any two nodes n v and n′ v are conditionally independent. Some candidates for aggregators are:

[0152] 1. Summary statistics: Summary statistics (e.g., counts, sums, n-th moments, q% quantiles, etc.) can be used. For example, q% quantiles is a partition of a set into subsets that fall within a certain range (e.g., for a selected q spanning [0,)1], e.g., (0.1,)0.3,)0.5,)0.7,)0.9). If both counts and sums and d different moments and quantiles are used, then for a node with D-dimensional attributes, the total dimension of the embedding is D+D+d×D+d×D.

[0153] 2. Distribution parameters: A distribution can be fitted to a set and the distribution parameters can be used as an embedding. This can be done using Gaussian copula. For example, a Gaussian copula model can be fitted using neighbors as a training set and model parameters (e.g., margins as histogram vectors and covariance matrices) as representations. In this case, for a node with D-dimensional attributes, the dimension of the embedding is D)+D 2 .

[0154] 3. Set embeddings: Set embeddings can be utilized (e.g., the deep set architecture detailed in Zaheer et al., "Deep Sets", Advances in neural information processing systems, 30, 2017, the entire contents of which are incorporated herein by reference). For example, two neural networks can be utilized, one for computing the embedding of each element in the set and one for aggregating the element embeddings into a set embedding. The set embedding can be trained based on the input data set (e.g., using stochastic gradient descent, and optionally using differentially private stochastic gradient descent). This only works if the generative model used to generate the attributes of B based on the set embedding is also based on a neural network.

[0155] Therefore, the second set of properties can be generated using the following method:

[0156] 1. For a pair of tables (A and B), extract the attributes of B and the links between the rows of A and B and represent the links as a bipartite graph.

[0157] 2. Compute the set embedding—a representation of the neighbors (e.g., parent nodes) of each node in Table B in the graph—using any method that computes a multiset representation, i.e., a method that accepts a multiset and outputs a fixed-size vector representing the neighbors. This method can achieve this in a differentially private manner.

[0158] 3. Conditioned on the set embeddings, sample rows of Table B using a generative model configured to generate attributes.

[0159] Depending on the type of set embedding used, it may be necessary to learn the set embedding (e.g. based on the attributes of B and the composition graph).

[0160] The generative model can be trained (fitted) based on the properties of B, conditioned on the computed set embeddings. The generative model can be trained to be differentially private. Training can be performed in a similar manner to the training of the generative model in stage 2; however, it is instead based on the properties of table B and the set embeddings.

[0161] If both the set embedding and the conditional generative model utilize neural networks, then both can be learned in an end-to-end manner using the learning objective of the conditional generative model.

[0162] Differential Privacy

[0163] The models described herein can be trained to synthesize data that reflects the distribution of an input data set while maintaining a given privacy level. As discussed above, a differential privacy level can be ensured.

[0164] Each sub-model in can be trained so that the learned parameters θ=(θ1,θ2,θ3) are differentially private (DP). Each set of parameters θ1,θ2,θ3 can be learned as (∈,δ)-DP based on corresponding privacy budgets ∈1, ∈1, and ∈3 (∈1+∈2+∈3=∈). These privacy budgets can be predefined (e.g., set by the user).

[0165] for θ1 is the BJDD matrix P J According to the definition of differential privacy given above, it is important to mask the presence or absence of a single node of U' or V', which can be a stronger privacy notion than edge protection. To achieve this, the Laplace mechanism can be implemented so that P J ∈-DP. For each entry in BJD)J, Laplace noise can be added. Specifically, after measuring the BJD matrix J from the data graph, noise can be added to each entry, where for the i-th row and j-th column entry of J, the added noise is Where Lap() is the Laplace operator and S ij =4max(d i ,d j ). After injecting noise into BJDD)J, the BJDD matrix P J Can be sampled from J.

[0166] for θ2 includes the underlying parameters of the generative model used to generate the first set of attributes U'. Therefore, to ensure differential privacy, a differentially private generative model trained using differentially private stochastic gradient descent can be selected.

[0167] for θ3 includes parameters of the set embedding model and parameters of the generative model for generating the second set of attributes V'. For the former, if the set embedding is fixed in advance (e.g., summary statistics or Gaussian copula dependencies), then there are no learnable parameters of the embedding model. If the set embedding is based on a neural network (e.g., a deep set architecture), then it can be trained using differentially private stochastic gradient descent. The differentially private generative model can be selected for the generative model trained using differentially private stochastic gradient descent. If the set embedding is based on a neural network, then both the generator model and the set embedding can be trained jointly using differentially private stochastic gradient descent.

[0168] Given the factorization implemented in this paper: By sequential combination, it can be seen that if each of the sub-models is differentially private according to ∈1, ∈2, and ∈3, then the overall process is (∈1+∈2+∈3) differentially private.

[0169] The proposed formula is flexible enough to allow privacy protection for nodes from one of the generated tables U', V' but not the other. This can be implemented by setting ∈ for the generation of a particular table to be equal to infinity. For example, if the first generated table U' is chosen to protect, then when learning ,∈3 can be set to∈3=∞, and when learning When , we can choose d for all j j =0, forming the overall synthetic data generation process (∈,δ)-DP)(∈1+∈2=∈).

[0170] In an example, the conditional model and / or the node embedding model is implemented using an (artificial) neural network. An artificial neural network is a structure inspired by the human brain. An example neural network includes multiple node layers, including an input layer, an output layer, and one or more intermediate layers. Each layer includes a node (also called a neuron or perceptron). A node is a computing unit that has one or more weighted input connections, a transfer function that combines inputs by a mathematical function to produce an output, and an output connection. Nodes in different layers are connected to each other via weighted connections so that the output connection of one node is coupled to the input connection of another node. The neural network can be parameterized by a set of one or more parameters. Training a neural network can include updating the parameters of the neural network to optimize a specific objective function. This can be in the form of stochastic gradient descent, where the parameters of the neural network are updated based on the gradient of a loss function. The loss function can be based at least in part on an error (the difference between the output (prediction) of the neural network and a predefined value (training value)).

[0171] Although the above embodiments assume that there are no edge attributes, the methods described herein can be extended to consider edge attributes (attributes associated with each association between entries). This can be achieved by adding an edge attribute model that is similar to p(L|U,V) but generates edge attributes instead. Any form of edge prediction model can be used for this purpose. In this case, the edge attribute model generates edge attributes based on the generated attributes U' and V', rather than predicting the existence of edges. In this case, the edge attribute model can be implemented after all other data has been generated (e.g., after stage 3 discussed above). A neural network can be used to predict these edge attributes. The neural network can be trained to predict edge attributes based on training data that includes labeled edge attributes (e.g., by stochastic gradient descent). As with the other steps, this can be implemented in a differentially private manner (e.g., by DP-STG).

[0172] While the reader should appreciate that the general methods described herein are applicable to any computing system, a typical computing system will now be described that provides methods that enable the embodiments described herein to be effective.

[0173] Figure 4A computing system is shown for implementing the embodiments described herein. As illustrated, computing system 200 includes a processor 210 coupled to a mass storage unit 220 and accessing a working memory 230.

[0174] Conventional procedures for loading software into memory and storing data in the mass storage unit 220 apply. The processor 210 also accesses, via the bus 240, a communication interface 250 configured to receive data from and output data to an external system (e.g., an external network or a user input or output device). The communication interface 250 may be a single component or may be divided into a separate input interface and a separate output interface.

[0175] The processor is configured to implement the methods described herein based on executable software stored in the mass storage unit 220. The software may be embedded in the original equipment, or may be provided in whole or in part after manufacture. For example, the software may be introduced in its entirety as a computer program product in a downloadable form or via a computer program storage medium such as an optical disk or a connectable memory drive (e.g., a universal serial bus flash drive). Alternatively, modifications to an existing controller may be made by way of an update or plug-in to provide the features of the above-described embodiments.

[0176] The computing system 200 may implement the methods described herein to generate synthetic data. This may utilize a pre-trained model. Alternatively, the computing system 200 may train a model based on an input data set to generate synthetic data.

[0177] The subject matter and implementation of the operations described in this specification may be implemented in a digital electronic circuit system or in computer software, firmware or hardware (including the structures disclosed in this specification and their structural equivalents) or in a combination of one or more of them. The implementation of the subject matter described in this specification may be implemented using one or more computer programs (i.e., one or more modules of computer program instructions) encoded on a computer storage medium to be executed by a data processing device or to control the operation of the data processing device. Alternatively or in addition, the program instructions may be encoded in an artificially generated propagation signal, such as a machine-generated electrical, optical or electromagnetic signal generated to encode information for transmission to a suitable receiver device for execution by a data processing device. The computer storage medium may be a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them or included therein. In addition, although the computer storage medium is not a propagation signal, the computer storage medium may be the source or destination of the computer program instructions encoded in the artificially generated propagation signal. The computer storage medium may also be one or more separate physical components or media (such as multiple CDs, disks or other storage devices) or included therein.

[0178] Although certain arrangements have been described, the arrangements have been presented only by way of example, and it is not intended to limit the scope of protection. The inventive concepts described herein may be implemented in various other forms. In addition, various omissions, substitutions, and changes to the specific embodiments described herein may be made without departing from the scope of protection defined in the following claims.

Claims

1. A computer-implemented method for generating a synthetic data set from a first data set, wherein: The first data set includes a first table (Table A) containing a first plurality of entries and a second table (Table B) containing a second plurality of entries; The synthetic data set includes a third table (Table A') including a third plurality of entries and a fourth table (Table B') including a fourth plurality of entries; and The method comprises: generating a composite graph based on a plurality of associations between the first plurality of entries in the first table (table A) and the second plurality of entries in the second table (table B), wherein the composite graph comprises a first set of nodes (U'), a second set of nodes (V'), and a first set of edges (L'); determining one or more attributes associated with each node in the first set of nodes (U') using a first conditional model (p(U'|L')) conditioned on the composite graph; and determining one or more attributes associated with each node in the second set of nodes (V') using a second conditional model (p(V'|L',U')) conditioned on the composite graph and the one or more attributes associated with each node in the first set of nodes (U'); wherein: The third plurality of entries corresponds to the one or more attributes associated with each node in the first set of nodes (U'), and the fourth plurality of entries corresponds to one or more attributes associated with each node in the second set of nodes (V').

2. The method of claim 1, wherein generating the composite graph based on the plurality of associations between entries in the first table (table A) and entries in the second table (table B) comprises: The first set of edges (L') is generated so that the first set of edges (L') has the same joint distribution of multiple pairs of node degrees as a second set of edges in a second graph representing the first data set.

3. The method according to any of the preceding claims, wherein generating the composite graph based on the plurality of associations between entries in the first table (Table A) and entries in the second table (Table B) comprises: Obtain the bipartite joint degree distribution (P J ), where the bipartite joint degree distribution (P J ) represents the plurality of associations between the entries in the first table (table A) and the entries in the second table (table B); Modify the bipartite joint degree distribution (P J ) to generate a modified bipartite joint degree distribution (P J + noise); From the modified bipartite joint degree distribution (P J + noise) to sample a target number of edges to generate a joint degree matrix of the synthetic data; and The synthetic graph is generated to conform to the joint degree matrix of the synthetic data.

4. The method according to claim 3, wherein the bipartite joint degree distribution (P J )include: Representing the plurality of associations between entries in the first table (table A) and entries in the second table (table B) as a bipartite graph (B); and Calculate the bipartite joint degree distribution (P) of the bipartite graph (B) J ).

5. The method according to any one of claims 4 or 5, wherein generating the composite graph conforming to the joint degree matrix comprises: generating a bipartite graph having a joint degree distribution conforming to the joint degree matrix of the synthetic data; and Edges within the bipartite graph are altered to remove loop edges and / or to remove multiple edges while maintaining a joint degree distribution of the joint degree matrix conforming to the synthetic data.

6. The method according to any one of claims 3 or 5, wherein the bipartite joint degree distribution (P J ) to generate the modified bipartite joint degree distribution (P J + noise) include: Additive noise is injected into the bipartite joint degree distribution (P J ).

7. The method according to claim 6, wherein the bipartite joint degree distribution (P J ) is a matrix, and wherein the additive noise is injected into the bipartite joint degree distribution (P J ) further comprises: computing noise values ​​for entries in the matrix using a Laplace mechanism to ensure ε differential privacy, where ε is a first parameter associated with a differential privacy amount (ε); and The noise value is added to the entry.

8. The method according to claim 7, wherein the noise value is based on Calculate, where S ij =4max(d i ,d j ), where i is the row of the entry in the matrix and j is the column of the entry in the matrix, (d i ,d j ) is the position of the entry in the matrix, and Lap is the Laplace mechanism.

9. The method of any of the preceding claims, wherein determining one or more properties associated with each node in the first set of nodes (U') using the first conditional model (p(U|L)) conditioned on the composite graph comprises: determining a node embedding for each node in the first set of nodes (U') based on a first node embedding model (β) and the composite graph; and The one or more attributes associated with each node in the first set of nodes (U') are determined using the first conditional model ((p(U|L)) and the node embedding of each node in the first set of nodes (U').

10. The method of claim 9, wherein determining a node embedding of each node in the first set of nodes (U') based on the first node embedding model (β) and the composite graph comprises: obtaining the first node embedding model (β), wherein the first node embedding model (β) is configured to represent nodes of the graph as vectors; and The node embedding of each node in the first set of nodes (U') is determined using the first embedding model (β) and a composite graph.

11. The method of any one of claims 9 or 10, wherein determining the one or more attributes associated with each node in the first set of nodes (U') using the first conditional model ((p(U|L)) conditioned on the composite graph comprises: Obtain the first conditional model ((p(U|L)); and determining a probability distribution for each attribute associated with a node in the first set of nodes (U') using the first conditional model ((p(U|L))) and the node embeddings associated with the node; and Each attribute associated with the node is sampled from the probability distribution.

12. The method according to claim 11, wherein obtaining the first conditional model (p(U|L)) comprises: Extracting a first set of attributes from the first table (Table A); The plurality of associations between the entries in the first table (table A) and the entries in the second table (table B) are represented as a second bipartite graph (B), wherein: The second graph includes a third set of nodes (U) and a second set of edges (L) associated with the first set of attributes; determining a third set of node embeddings comprising the node embedding of each node in the third set of nodes; The first conditional model (p(U|L)) is generated based on the first set of attributes and the third set of node embeddings.

13. The method of claim 12, wherein generating the first conditional model (p(U|L)) based on the first set of attributes and the third set of node embeddings comprises: The first conditional model is fit on the first set of attributes, conditioned on the third set of node embeddings.

14. The method of any of the preceding claims, wherein determining one or more attributes associated with each node in the second set of nodes (V') using a second conditional model (p(V|L,U)) conditioned on the composite graph and the one or more attributes associated with each node in the first set of nodes (U') comprises: Based on the second node embedding model (γ) and the neighbors (ne(n v )) to determine a second node embedding for each node in the second set of nodes (V'); The one or more attributes associated with each node in the second set of nodes (V') are determined using a second conditional model (p(V|L,U)) and the second node embedding of each node in the second set of nodes (V').

15. The method according to claim 14, wherein the method further comprises: v )) and determining the second node embedding of each node in the second set of nodes (V') comprises: obtaining the second node embedding model (γ), wherein the second node embedding model (γ) is configured to represent nodes of the graph as vectors; Using the second embedding model (γ) and the neighbors (ne(n v )) determines the second node embedding of each node in the second set of nodes based on information associated with the second node embedding, wherein: The neighbors (ne(n v The information associated with the second set of nodes (V') includes the neighbor (ne(n v ))'s one or more attributes associated with each node in the first set of nodes (U').

16. The method of claim 15, wherein the second node embedding of each node in the second set of nodes is a fixed-size vector.

17. The method of any one of claims 14 to 16, wherein determining the one or more attributes associated with each node in the second set of nodes (V') using a second conditional model (p(V|L,U)) and the second node embedding of each node in the second set of nodes (V') comprises: Obtain the second conditional model (p(V|L,U)); determining a second probability distribution for each attribute associated with a node using the second conditional model (p(V|L,U)) and second node embeddings associated with nodes in the second set of nodes (V'); and The second probability distribution is sampled to obtain each attribute associated with the node.

18. The method according to claim 17, wherein obtaining the second conditional model (p(V|L,U)) comprises: extracting a second set of attributes from the second table (table B); The plurality of associations between the entries in the first table (table A) and the entries in the second table (table B) are represented as a third bipartite graph (B), where: The third bipartite graph includes a fourth set of nodes (V) associated with the second set of attributes; determining a fourth set of node embeddings comprising node embeddings of each node in the fourth set of nodes (V), wherein the node embeddings of each node in the fourth set of nodes (V) represent properties of neighbors of each node in the fourth set of nodes (V); and The second conditional model (p(V|L,U)) is generated based on the second set of attributes and the fourth set of node embeddings.

19. The method of claim 18, wherein determining the fourth set of node embeddings further comprises: determining a multiset representation for each node in the fourth set of nodes (V) based on the attributes of the neighbors of each respective node in the fourth set of nodes (V); and The fourth set of node embeddings is computed based on the multiset representation of each node.

20. The method of any of claims 18 to 19, wherein generating the second conditional model (p(V|L,U)) based on the second set of attributes and the fourth set of node embeddings comprises: The second conditional model is fit on the second set of attributes, conditioned on the fourth set of node embeddings.

21. The method of any of the preceding claims, wherein the first conditional model and the second conditional model are each differentially private (DP) conditional models.

22. A method according to any one of the preceding claims, wherein: Generating the synthetic graph includes generating the synthetic graph based on a first learnable parameter (θ1), and the method further includes determining a value of the first learnable parameter (θ1).

23. The method according to any one of the preceding claims, further comprising: The value of the second learnable parameter (θ2) is learned so that the first conditional model (p(U|L)) satisfies the criterion, where: The second learnable parameters (θ2) include parameters of the first conditional model (p(U|L)); and The criterion is that the error between the output of the first conditional model p(U|L) and the node value associated with the first data set is minimized according to a differential privacy criterion.

24. The method according to any one of the preceding claims, further comprising: The value of the third learnable parameter (θ3) is learned so that the second conditional model (p(V|L,U)) satisfies the criterion, where: The second learnable parameters (θ3) include parameters of the second conditional model (p(V|L,U)); and The criterion is that the error between the output of the second conditional model (p(V|L,U)) and the node value associated with the first data set is minimized according to a differential privacy criterion.

25. A computer-implemented method for generating a synthetic data set from a first data set, wherein the first data set comprises a plurality of tables, and the method further comprises: Synthetic data for each combination of tables in the plurality of tables in the first data set is generated according to the method of any one of the preceding claims.

26. A non-transitory computer readable medium comprising computer program instructions adapted for execution by a processor, the instructions being configured to perform the method according to any one of the preceding claims when executed by the processor.

Images