Method, system and computer readable products for generating synthetic data

Abstract

Described embodiments comprise generating a synthetic dataset from a set of input datasets, each comprising a plurality of data items. A plurality of logical expressions is applied to at least one data item in each input dataset to derive a plurality of logical signatures that collectively characterise the value of each of the at least one data item in each input dataset. The logical signatures are compared to derive at least one logical constraint from the logical signatures, and a synthetic dataset is determined by applying the at least one logical constraint to a logic solving engine. The synthetic dataset satisfies the at least one logical constraint.

Description

Method, system and computer readable products for generating synthetic dataCross-Reference to Related Application

[0001] The present application claims priority from Australian Provisional Patent Application No. 2024900406 filed on 20 February 2025, the contents of which are incorporated herein by reference in their entirety.Technical Field

[0002] Aspects of the disclosure relate generally to systems and methods for generating synthetic data and, more specifically, to generating synthetic data based on the constraints derived from the characteristics of an input dataset.Background

[0003] Synthetic data refers to artificially created data that mimics data that may be collected from observations or measurements.

[0004] The provision of synthetic data can enable the development of research and techniques where real data is unavailable, limited, restricted, biased, expensive to collect and / or private.

[0005] Users may be motivated to create synthetic datasets to address challenges such as privacy concerns, data scarcity, or the need for data on which to perform controlled experiments.

[0006] Synthetic data can also be beneficial for validating mathematical models and algorithms. Researchers can test hypotheses and predictions using synthetic data before applying them to real- world scenarios. Synthetic data can also be used to augment real datasets to provide larger or more balanced datasets for Artificial Intelligence (Al) and machine learning (ML) systems.

[0007] Where privacy requirements prevent the use of an empirically derived dataset, anonymisation and / or sanitization approaches can produce sanitized data that aspires to be anonymous by selectively deleting outlying data or by generalising sensitive information in the dataset. However, the process of anonymising the data can change the characteristics of the dataset, and, in particular, reduce the diversity that was present in the empirically derived dataset.

[0008] Synthetic data can serve as a substitute, providing privacy protection. For instance, healthcare researchers can create synthetic patient records to develop and test medical algorithms without exposing sensitive patient information.

[0009] The process of generating synthetic data may involve simulating data points that resemble the characteristics of the original dataset without compromising sensitive information. Synthetic data may be generated using statistical methods or machine learning methods, including deep-learning and neural-network approaches.

[0010] Existing methods of synthetic data generation may not produce data that has characteristics that reflect the characteristics of actual data. Furthermore, existing methods of synthetic data generation may not scale well to high data dimensionality and may not provide privacy for the actual data from which the synthetic data was derived.

[0011] Accordingly, there is a desire to provide a method of generating synthetic data that ameliorates one or more of these difficulties, or other difficulties, of the prior art, or at least provides a useful alternative.

[0012] Any discussion of documents, acts, materials, devices, articles or the like which has been included in the present specification is solely for the purpose of providing a context for the present invention. It is not to be taken as an admission that any or all of these matters form part of the prior art base or were common general knowledge in the field relevant to the present invention as it existed before the priority date of each claim of this application.Summary

[0013] In accordance with an aspect of the present disclosure, there is provided a computer- implemented method for generating a synthetic dataset (320) from a set of input datasets (302), each input dataset comprising a plurality of data items, the method comprising: applying a plurality of logical expressions (520) to at least one data item in each input dataset to derive a plurality of logical signatures (530) that collectively characterise the value of each of the at least one data item in each input dataset; comparing the logical signatures (530) to derive at least one logical constraint (760) from the logical signatures; and determining, by applying the at least one logical constraint to a logic solving engine (140), the synthetic dataset (920), wherein the synthetic dataset (920) satisfies the at least one logical constraint (760).

[0014] In some embodiments, at least one logical constraint comprises a two-clause constraint.

[0015] In some embodiments, deriving the at least one logical constraint comprises: associating a first constraint variable with a first logical signature of the plurality of logical signatures; associating a second constraint variable with a second logical signature of the plurality of logical signatures; and defining the logical constraint as a Boolean expression comprising the first constraint variable, the second constraint variable and a Boolean operator.

[0016] In some embodiments, each data item comprises a value of a set of discrete values.

[0017] In some embodiments, the plurality of logical signatures for each data item comprises a first logical signature that indicates whether the data item positioned at a first position is equal to a specified value of the set of specified values.

[0018] In some embodiments, the plurality of logical signatures comprises: for each position, a set of logical signatures comprising a logical signature for each specified value of the set of specified values.

[0019] In some embodiments, each logical signature comprises an array of Boolean values, each Boolean value indicating whether a data item positioned in a specific array position of a respective dataset equals a specified value of the set of specified values.

[0020] In some embodiments, applying a plurality of logical expressions to the data items in each dataset to derive a first logical signature of the plurality of logical signatures comprises: for each data item of the dataset: for each specified value of the set of specified values: in response to the data item, positioned at the first position in the dataset, equaling the specified value, set a corresponding Boolean value of the logical signature to represent true; and in response to the data item, positioned at the first position in the dataset, not equaling the specified value, set a corresponding Boolean value of the logical signature to represent false.

[0021] In some embodiments, applying a plurality of logical expressions to the data items in each dataset to derive a second logical signature of the plurality of logical signatures comprises: for each data item of the dataset: for each specified value of the set of specified values: in response to the data item, positioned at the first position in the dataset, equaling the specified value, set a corresponding Boolean value of the logical signature to represent false; and in response to the data item, positioned at the first position in the dataset, not equaling the specified value, set a corresponding Boolean value of the logical signature to represent true.

[0022] In some embodiments, the method further comprises: identifying duplicate logical signatures in the plurality of logical signatures; and determining a subset of logical signatures, wherein the subset of logical signatures does not include the duplicate logical signatures.

[0023] In some embodiments, comparing logical signatures comprises: comparing a first logical signature of the plurality of logical signatures with a second logical signature of the plurality of logical signatures.

[0024] In some embodiments, deriving a logical constraint comprises, in response to comparing the first logical signature with the second logical signature: determining that a logical condition holds between logical signs of the first signature and logical signs of the second signature; and defining the logical constraint in accordance with the logical condition.

[0025] In some embodiments, the first logical signature and the second logical signature define a plurality of pairs of logical signs, and wherein comparing the first logical signature with the second logical signature comprises: for each pair of logical signs, determine whether both logical signs of the pair of logical signs are false.

[0026] In some embodiments, the method further comprises, in response to determining, for each pair of logical signs, that both logical signs of the pair are not false, deriving a logical constraint comprising the Boolean OR operator applied to a constraint variable associated with the first logical signature and a constraint variable associated with the second logical signature.

[0027] In some embodiments, comparing logical signatures comprises determining logical conditions that hold across the dataset.

[0028] In some embodiments, comparing logical signatures comprises comparing each logical signature of the plurality of logical signatures with each other logical signature of the plurality of logical signatures.

[0029] In some embodiments, the at least one logical constraint defines at least one logical condition that holds across the dataset.

[0030] In some embodiments, determining the synthetic dataset comprises: obtaining, from the logical solving engine, a synthetic logical sign for each constraint variable of a set of constraint variables; and for each array position of the synthetic dataset determining a synthetic value for the synthetic data item in the array position of the synthetic dataset.

[0031] In some embodiments, the method further comprises determining a level of privacy of the synthetic dataset, wherein the level of privacy comprises a level of determinability of a membership of a first input dataset in the set of input datasets.

[0032] In some embodiments, determining a level of privacy comprises: determining, by applying a logic solving engine, a set of potential input datasets; and determining themembership of one or more input datasets of the set of input datasets in the set of potential input datasets.

[0033] In some embodiments, the set of input datasets comprises a synthetic dataset that was previously generated by the logic solving engine.

[0034] In some embodiments, deriving the at least one logical constraint from the logical signatures comprises applying a randomisation to the logical signatures.

[0035] In some embodiments, the set of input datasets comprises a randomly generated input dataset.

[0036] A machine-readable storage medium stores instructions which, when executed by one or more processors, cause the one or more processors to perform the above method.

[0037] A system comprises one or more processors and memory comprising computer executable instructions, which when executed by the one or more processors, cause the system to perform the above method.

[0038] A machine-readable storage medium stores a synthetic dataset determined from a set of input datasets (302), each input dataset comprising a plurality of data items, wherein the synthetic dataset is determined by a method comprising: applying a plurality of logical expressions to the data items in each dataset to derive a plurality of logical signatures characterising the value of each data item; comparing the logical signatures to determine at least one logical constraint based on the logical signatures; and determining the synthetic dataset by applying the at least one logical constraint to a logical solving engine, wherein the synthetic dataset satisfies each of the at least one logical constraints.

[0039] There is provided a system for determining a synthetic dataset from a set of input datasets, each input dataset comprising a plurality of data items, the system comprising one or more processors, individually or in combination, configured to: apply a plurality of logical expressions to the data items in each dataset to derive a plurality of logical signatures characterising the value of each data item; compare the logical signatures to determine at least one logical constraint based on the logical signatures; and determine the synthetic dataset by applying the at least one logical constraint to a logical solving engine, wherein the synthetic dataset satisfies each of the at least one logical constraints.

[0040] There is provided a system for determining a synthetic dataset from a set of input datasets, each input dataset comprising a plurality of data items, the system comprising meansto: apply a plurality of logical expressions to the data items in each dataset to derive a plurality of logical signatures characterising the value of each data item; compare the logical signatures to determine at least one logical constraint based on the logical signatures; and determine the synthetic dataset by applying the at least one logical constraint to a logical solving engine, wherein the synthetic dataset satisfies each of the at least one logical constraints.

[0041] There is provided a computer- implemented method for determining a Boolean representation of an array of data items, each data item comprising a respective value of a set of specified values, the method comprising: for each data item of the array of data items: for each specified value of the set of specified values: in response to the data item equaling the specified value, set a corresponding Boolean value of the Boolean representation to represent true; and in response to the data item not equaling the specified value, set a corresponding Boolean value of the Boolean representation to represent false.

[0042] In some embodiments, the method further comprises for each data item of the array of data items: for each specified value of the set of specified values: in response to the data item equaling the specified value, set a corresponding Boolean value of the Boolean representation to represent false; and in response to the data item not equaling the specified value, set a corresponding Boolean value of the Boolean representation to represent true.

[0043] It is to be understood that optional features provided in relation to the method are equally optional features for the other aspects provided above.Brief Description of Drawings

[0044] The embodiments of the disclosure will now be described with reference to the accompanying drawings, in which:Figure 1 illustrates a system for generating synthetic data, in accordance with an embodiment;Figure 2 is a flowchart illustrating a method, as performed by the synthetic data generator, to generate synthetic data, in accordance with an embodiment;Figure 3 illustrates a set of input datasets, in accordance with an embodiment;Figure 4 illustrates a set of logical signatures generated by the pre-processor by applying logical expressions to the data items located in Location 1 of the input datasets illustrated in Figure 3, in accordance with an embodiment;Figure 5 illustrates logical signatures generated by the pre-processor, by applying logical expressions to each of the data items of the input datasets, in accordance with an embodiment;Figure 6 illustrates a set of unique logical signatures determined from the set of logical signatures illustrated in Figure 5, in accordance with an embodiment;Figure 7 illustrates the generation of indistinguishability constraints by the constraint engine, in accordance with an embodiment;Figure 8 illustrates the logical constraints input into the logic solving engine, and the synthetic values output by the logic solving engine for each of the constraint variables, in accordance with an embodiment;Figure 9 illustrates the determination, by the post processor, of the values of the synthetic data items based on the synthetic logic signs, in accordance with an embodiment;Figure 10 illustrates a set of logical signs associated with Person 0 and a set of logical signs associated with Person 1, in accordance with an embodiment;Figure 11 illustrates the comparison of constraint variables, associated with the synthetic logical signs, with the logical signs derived from the input dataset associated with Person 0, and with the logical signs derived from the input dataset associated with Person 1, in accordance with an embodiment;Figure 12 is a graph 1200 illustrating the runtime 1202 for an embodiment of the synthetic data generator 102 configured to generate synthetic genomic data, compared to runtimes of alternative methods of generating synthetic genomic data;Figure 13 is a graph 1300 illustrating square error in linkage disequilibrium reproduction between loci in original dataset against that reflected in the synthetic data for the AGBL4 gene, in accordance with an embodiment;Figure 14 illustrates a system for evaluating the privacy of input datasets, in accordance with an embodiment;Figure 15 is a flowchart illustrating a method, as performed by the privacy engine, to evaluate the privacy of input datasets used to generate synthetic data, in accordance with an embodiment;Figure 16 illustrates a scenario in which a hypothetical attacker seeks to determine the input datasets used by a synthetic data generator to generate a synthetic dataset, in accordance with an embodiment;Figure 17 illustrates a set of candidate input datasets, in accordance with an embodiment;Figure 18 illustrates a set of logical signatures generated by the constraint engine, by applying logical expressions to the data items of the set of candidate input datasets, in accordance with an embodiment;Figure 19 illustrates the derivation of indistinguishability constraints by the constraint engine, in accordance with an embodiment;Figure 20 illustrate combinations of constraint variables, as output by the logic solving engine, according to an embodiment;Figure 21 is a graph illustrating a posteriori probability that a candidate input dataset is present in the actual set of datasets used to generate the synthetic dataset, in accordance with an embodiment;Figure 22 illustrates the addition of randomisation to a logical constraint, in accordance with an embodiment;Figure 23 illustrates randomised 8hresholding, in accordance with an embodiment;Figure 24 illustrates the generation of a logical constraint based on a consideration of a partition in a logical signature, in accordance with an embodiment;Figure 25 illustrates a process where the function to determine the addition of a respective 2-clause considers a range of pairs in the logical signatures compared, in accordance with an embodiment;Figure 26 illustrates an example in which a pairwise OR-ing schema is equivalent to an expanded multipart repeated partitioning reflective, in accordance with an embodiment;Figure 27 illustrates an application of Figure 26, in accordance with an embodiment;Figure 28 illustrates another application of Figure 26, in accordance with an embodiment;Figure 29 illustrates a PCA graph of the Genomator, in accordance with an embodiment;Figure 30 illustrates a PCA graph of a Markov Chain Generation (MARK) implementation which is configured to generate synthetic genomic data, in accordance with an embodiment;Figure 31 illustrates a PCA graph of a Restricted Boltzmann Machine (RBM) implementation which is configured to generate synthetic genomic data, in accordance with an embodiment;Figure 32 illustrates a PCA graph of a Generative Adversarial Network (GAN) implementation which is configured to generate synthetic genomic data, in accordance with an embodiment;Figure 33 is a scatter plot illustrating square error in linkage disequilibrium reproduction between loci in original dataset against that reflected in the synthetic data for the AGBL4 gene, in accordance with an embodiment;Figure 34 is a scatter plot illustrating square error in linkage disequilibrium reproduction between loci in original dataset against that reflected in the synthetic data for the FHIT gene, in accordance with an embodiment;Figure 35 is a scatter plot illustrating square error in linkage disequilibrium reproduction between loci in original dataset against that reflected in the synthetic data for the CCSER1 gene, in accordance with an embodiment;Figure 36 is a scatter plot illustrating square error in linkage disequilibrium reproduction between loci in original dataset against that reflected in the synthetic data for the RBFOX1 gene, in accordance with an embodiment;Figure 37 comprises a table 3700 of results of an SNP inference experiment for different implementations of a synthetic data generator, in accordance with an embodiment;Figure 38 is a scatter plot illustrating the average number of times that a ‘private’ combination of SNPs appeared in the output dataset was computed, as well as the number of times ‘fictions’ SNP combinations were generated, in accordance with an embodiment; andFigure 39 is a graph illustrating the trade-off between privacy and accuracy for Genomator, in accordance with an embodiment;Figure 40 shows an example pseudocode implementing some of the methods disclosed herein, which may be referred to as Genomator, in accordance with an embodiment;Figure 40 shows an example pseudocode implementing some of the methods disclosed herein, which may be referred to as reverse Genomator, in accordance with an embodiment; andFigure 41 is a flowchart of the methods disclosed herein, in accordance with an embodiment.Description of Embodiments

[0045] Provided herein is a method and system that employs constraint satisfaction techniques for the deductive generation of synthetic data. Methods and systems provided herein employ a logic solving engine to generate synthetic data based on logical constraints that are derived from characteristics inherent within a set of input datasets.

[0046] A logic solving engine is a software system that can perform automated reasoning on a given set of logical constraints. The logical constraints form a satisfiability problem that may be solved by the logic solving engine.

[0047] A satisfiability problem (SAT problem) is a problem of determining if there exists an assignment of values to some constraint variables that makes a given Boolean expression true. For example, if the expression is “x OR y”, then the satisfiability problem is to find out if there are values of x and y (either TRUE or FALSE) that make the expression true.

[0048] A logic solving engine may comprise logic programming languages, such as Prolog, Answer Set Programming (ASP) and Datalog. These languages allow writing programs, databases or knowledge bases as a set of clauses in the form of A Bl, . . Bn, which are read as logical implications. Logic programming languages may employ a problem-solving strategy called backward reasoning, which reduces goals to subgoals until they are either proven or disproven.

[0049] A logic solving engine may comprise a rule engine, such as Drools, Jess and “C” Language Integrated Production System (CLIPS). Rule engines represent conditional logic as discrete rules, which can be applied to data or facts to infer new information or actions.

[0050] A logic solving engine may comprise a satisfiability solver such as Boolean satisfiability solver (e.g., a SAT solver) or satisfiability modulo theories (SMT) solver. These satisfiability solvers are algorithms that can determine if there exists an assignment of values to some constraint variables that makes a given logical expression true.

[0051] Many satisfiability solvers are single-threaded solving algorithms embodying a systematic backtracking search procedure. Backtracking SAT solvers aim to satisfy one or more Boolean constraint expressions by iteratively choosing and assigning constraint variable values to be True / False - from a preliminary state of their values being undefined or ‘free’. The selection of these values may lead the solver into a state where it cannot choose an additional constraint variable value either way without violating the Boolean constraint expression, at this point the SAT solver encounters a ‘contradiction’, arising from the variable values it has assigned so far, and in this case, it backtracks its most recent variable value assignment and then makes a different decision in the way it assigns that variable.

[0052] If both the assignment of a variable and its opposite assignment, leads to a contradiction, then the solver backtracks yet again, and assigns the opposite value to the next most recent variable assignment, and so on. The SAT solver keeps on doing this exploration and backtracking systematic search until it finds a solution to the problem, or it has to backtrack to the very beginning of its search indicating that there is no solution to the problem at all.

[0053] The systematic nature of backtracking search means that, given enough time, the SAT solver will either find a solution, or it will demonstrate that there are no solutions - that the problem is unsatisfiable or ‘UNSAT’.

[0054] In practice, if a satisfiability problem contains many constraint variables and / or is so difficult to solve that it may be computationally expensive or infeasible for a computer to solve the problem or demonstrate that there is no solution. Accordingly, it may be desirable to define a satisfiability problem in terms of few constraint variables.Indistinguishability

[0055] Indistinguishability of synthetic data is the property that synthetic data cannot be distinguished from real input data by an observer but is not actually identical to the real input data. Indistinguishability is desirable for synthetic data, as it implies that the synthetic data preserves the statistical characteristics and patterns of the real data, and thus can be used for various purposes, such as data analysis, machine learning, privacy protection, and more.

[0056] Indistinguishability of synthetic data can be measured by various methods, such as statistical tests, machine learning models, or human evaluations. Data A may be considered to be distinguishable from a set of data B if there is a set of logical conditions that holds for all elements of dataset B but does not hold for A.

[0057] Indistinguishability may be balanced with diversity, as too much similarity may lead to overfitting or bias in the synthetic data. Therefore, indistinguishability of synthetic data is a relative and subjective concept and may be evaluated according to the specific use case and application of synthetic data.Computational challenges

[0058] One of the main challenges of developing a logic solving engine is to ensure its correctness, efficiency, and scalability. Correctness means that the engine can produce valid and consistent answers for any valid input. Efficiency means that the engine can solve problems within reasonable time and space limits. Scalability means that the engine can handle large and complex inputs without compromising its performance.

[0059] Provided herein is a method of defining logical constraints, such that the synthetic dataset produced by the logical solving engine is different from the input datasets of the set of input datasets for a set distance, and indistinguishable from the input datasets of the set of input datasets under all pairwise combinations over a class of specified logical conditions. This deductive approach is very efficient, and facilitates a range of features, configurations and augmentations which could be used to enhance the privacy of the output.Figure 1 - System architecture

[0060] Figure 1 illustrates a system for generating synthetic data, in accordance with an embodiment. The system 100 comprises one or more processors. A processor may comprise one or more microprocessors, microcontrollers or controlling circuitry. The one or more processors of the system may be, in combination or individually, configured to execute program code stored within the memory to control the functionality of the system.

[0061] The system 100 comprises a synthetic data generator 102 and a controller 104 configured to control the synthetic data generator. The synthetic data generator comprises a preprocessor, configured to receive and pre-process a set of input datasets 106. The synthetic data generator further comprises a constraint engine 130 and a logic solving engine (LSE) 140. The constraint engine is configured to determine one or more logical constraints to be input in a logic solving engine.

[0062] The logic solving engine is configured to determine a synthetic dataset that satisfies the one or more logical constraints. In one embodiment, the logic solving engine 140 is supported by the PySAT library (Alexey Ignatiev, Antonio Morgado, and Joao Marques-Silva. PySAT: A Python toolkit for prototyping with SAT oracles, SAT, pages 428-437, 2018.)

[0063] In some embodiments, the synthetic data generator further comprises a post processor 150, configured to convert and / or post-process the output of the logic solving engine for use by a user.

[0064] The memory 160 may comprise one or more volatile or non-volatile memory types. For example, the memory may comprise one or more of random-access memory (RAM), readonly memory (ROM), electrically erasable programmable read-only memory (EEPROM) or flash memory. The memory is configured to store, in part or in full, program code (also referred to as machine readable code or software) accessible by the processor(s) of the synthetic data generator 102. The program code may comprise executable code modules. The executable code modules, when executed by the processor(s) cause the system 100 to perform the functionality according to the described embodiments.Figure 2 - Generating synthetic data

[0065] Figure 2 is a flowchart illustrating a method 200, as performed by the synthetic data generator 102, to generate a synthetic dataset, in accordance with an embodiment. The operations of method 200 are described in relation to Figures 3 to 9, which illustrate an example embodiment of the present technology, in which the synthetic data generator is configured to generate synthetic genomic data.Genomic data

[0066] Genomic data refers to the DNA data of organisms. In the field of bioinformatics, which combines biology and computer science, genomic data is collected, stored, and processed. Genomic data may encompass information about the structure, function, evolution, mapping, and editing of DNA and genomes. By analyzing genomic data, researchers, clinicians and other practitioners uncover how differences in DNA impact human health and disease, as well as environmental and agricultural applications.

[0067] In an embodiment described herein, the problem of generating appropriate synthetic genomic information is rendered as a SAT problem, and consequent genomic information is then generated by passing that SAT problem to a logic solving engine.Figure 3 - Input dataset

[0068] Figure 3 illustrates a set of input datasets, in accordance with an embodiment. The set of input datasets comprises empirically derived genomic datasets associated with 5 different people. For clarity of explanation, the genomic dataset associated with each person, as shown in Figure 3, is quite small, comprising only 5 nucleotides, and only 5 people are represented inthe set of input datasets. It will be understood that, in other embodiments, the number of nucleotides, and the number of people represented, may be considerably numerous.

[0069] The set of input datasets 302 comprises five datasets, of which dataset 304 is one. Each dataset is associated with a person, and represents genomic data associated with that person. Each dataset comprises a set of five data items, wherein each data item is positioned at a particular location within the dataset.

[0070] For example, dataset 304 comprises: data item C located in Location 1; data item A located in Location 2; data item T located in Location 3; data item G located in Location 4; and another data item G located in Location 5. Furthermore, data item 306 is located in Location 5 of the dataset associated with Person 0, and data item 312 is located in Location 1 of the dataset associated with Person 0.

[0071] Also illustrated in Figure 3, is a representation 320 of the synthetic dataset that is to be generated by the synthetic data generator 102. The synthetic dataset to be generated is represented as empty dataset 320 comprising unknown values. Each data item of the synthetic dataset 320 will comprise a value from the set of specified values A, T, C and G.Specified values

[0072] Each data item in the set of input datasets 302 has a value from a set of specified values associated with the dataset. In the example of Figure 3, the dataset comprises a set of nucleotides. Each data item represents one nucleotide of the four basic building blocks of nucleic acids, namely adenine (A), cytosine (C), guanine (G) and thymine (T). Accordingly, the set of specified values associated with the set of input datasets 302 comprises a set of four specified values, namely: A; C; G and T.

[0073] The value of the data items in the input dataset will depend on the type of data represented by the input datasets. For example, for numeric input data, the set of values may comprise numbers between a maximum and minimum integer. In an embodiment in which the input data comprises RGB (Red, Green, Blue) data representing images, the data values may comprise ‘R’, ‘G’ and ‘B’. In an embodiment in which the input dataset comprises binary strings, the data values may comprise ‘ L and ‘O’.

[0074] In embodiments, if the data items in the input datasets are not values of a set of specified values, then the pre-processor 120 may, in operation 202, be configured to adjust (e.g., reconfigure, translate, convert) the data items such that each data item in the input datasets has a value that is defined in terms of a set of specified values.Datasets

[0075] The constraint engine is configured to consider and compare the value of data items that are located in the same position within respective different datasets. Accordingly, it is desirable that the data format in which the data items are stored is such that the synthetic data generator 102 can determine the position of each data item. That is, data items that are positioned in the same location in different datasets may be determined as being positioned in the same location.

[0076] As referred to herein, a dataset comprises a set of data items, wherein the position of the data items may be determinable by the way in which the data items are stored. A dataset is not intended to be limited to any particular data structure, storage structure, or data format. A dataset may be implemented as a list, a key / value pair, a table column, a table row, an array memory structure, a comma separated string, or any other format in which the order of the data items in the dataset is determinable.Figure 4 - Logical signatures

[0077] In operation 204, the pre-processor 120 is configured to derive a set of logical signatures by applying logical expressions to the data items of the input datasets. A logical expression is a query regarding the value of a data item. The result of applying a logical expression is a binary logical sign (e.g., True (T) or False (F) which, characterises the value of the data item to which the logical expression was applied.

[0078] Figure 4 illustrates a set of logical signatures 430 generated by the pre-processor 120, by applying logical expressions 420 to the data items located in Location 1 of the input datasets 302 illustrated in Figure 3, in accordance with an embodiment.

[0079] For each data item located in Location 1, a logical expression is applied to the data item, to produce a logical sign. The logical sign is a binary representation of whether the logical expression is True (T) or False (F) in relation to the data item. In this example, each logical sign is True (T) or False (F); however, in other embodiments, an alternative binary representation may be used.

[0080] Each logical expression poses a True / False question regarding the value of the data item. If the data item comprises a value of a set of specified values, the logical expression may pose the question enquiring whether the data item is equal to a particular value of the set of specified values. In another example, a logical expression may enquire as to whether a data item is: within a certain range; below a maximum value; above a minimum value; is equal toone of a set of values; or another question that produces a logical sign that can be represented as a True or a False response. Accordingly, the logical sign determined in response to applying a logical expression to a data item characterises the value of the data item.

[0081] With reference to the example illustrated in Figure 4, logical expression 402 enquires whether the data item 312 is a value ‘A’. As data item 312 is a value ‘A’, the logical sign 404 determined by the pre-processor 120 indicates True (T).

[0082] The data item, in Location 1 of the dataset associated with Person 2, has a value of ‘C’, therefore in response to applying the logical expression 402 to that data item, the pre-processor 120 determines a logical sign 406 which indicates False (F).

[0083] The pre-processor is configured to apply the logical expression 402 to each of the five data items located in Location 1, to produce a set of five logical signs. The set of five logical signs is referred to as a logical signature 408. The logical signature 408 is associated with the logical expression 402.

[0084] The pre-processor is configured to apply a logical expression for each specified value (e.g. ‘A’, ‘C’, ‘G’, ‘T’) of the set of specified values associated with the input datasets 302. This set of four logical expressions is referenced as 422 in Figure 4. In response to applying the set of logical expressions 422 to the data items in Location 1 of the input datasets 302, the preprocessor generates four logical signatures 432.

[0085] In some embodiments, the pre-processor may be further configured to apply a second set of logical expressions 424 to the data items in Location 1 of the input datasets. The second set of logical expressions may each be an inverse logical expression to a respective one of the set of logical expressions 422. For example, the logical expression ‘Are you an A?’ is inverse to the logical expression ‘Are you not an A?’.

[0086] In response to applying the set of inverse logical expressions 424 to the data items in Location 1 of the input datasets, the pre-processor generates a set of four logical signatures 434. Each of the four logical signatures 434 is inverse to a respective one of the logical signatures 432.

[0087] By applying a set of inverse logical expressions 424 to the data items as well as the set of logical expressions 422, a set of inverse logical signatures 434 is produced. Advantageously, the inverse logical signatures may be compared with the logical signatures 432 to identify relationships (e.g., logical conditions) between the data items, wherein the relationship isdefined in terms of one of the data items NOT being a particular value. Accordingly, additional inter-data relationships may be identified by the constraint engine 130.Figure 5 - Set of logical signatures

[0088] The pre-processor 120 is configured to iteratively perform operation 204 to determine a logical signature for each logical expression for each data item in the set of input datasets 302.

[0089] Figure 5 illustrates logical signatures 530 generated by the pre-processor 120, by applying logical expressions 520 to each of the data items of the set of input datasets 302, in accordance with an embodiment. Each logical signature comprises five logical signs, one sign for each data item of the five datasets in the set of input datasets 302.Synthetic logical signs

[0090] Also illustrated in Figure 5, is a representation of synthetic logical signs that are to be generated by the synthetic data generator 102. The synthetic logical signs to be generated is represented unknown values 540. Each synthetic logical sign will comprise a True (T) or False (F) value.Constraint variables

[0091] In operation 208, the constraint engine 130 is configured to assign a constraint variable to each of the logical signatures 430. In Figure 4, constraint variables associated with the logical signatures 430 (and therefore also associated with the logical expressions 420) are referenced by reference numeral 440.

[0092] Constraint variables represent logical signs that are output by the logic solving engine 140. The constraint variables are used by the post-processor 150 to determine the value of synthetic data items (320).

[0093] Each constraint variable is a Boolean variable (e.g., True or False) that is representative of the result of applying a logical expression to a synthetic data item. For example, the constraint variable 442 (x2) is representative of the result of applying the logical expression ‘Are you a C?’ to the synthetic data item located at Eocation 1 of the synthetic dataset. Accordingly, if the logic solving engine 140 determines that the value of constraint variable x2is True, then the expression ‘Are you a C?’ is True. Therefore, the post processor 150 can determine that the synthetic data item located in Eocation 1 of the synthetic dataset is a C.

[0094] Similarly, the constraint variable 444 (->x6) is representative of the result of applying the logical expression ‘Are you an A?’ to the synthetic data item located at Location 1 of thesynthetic dataset. Accordingly, if the logic solving engine 140 determines that the value of constraint variable x6is False, then the value of -ix6is True (where the symbol -> indicates NOT), and the expression ‘Are you an A?’ is True. Therefore, the post processor 150 can determine that the synthetic data item located in Location 1 of the synthetic dataset is an A.Duplicate signatures

[0095] In some embodiments, the constraint engine 130 may be configured to identify duplicated logical signatures and assign constraint variables such that duplicated logical signatures are assigned the same constraint variable.

[0096] In the set of logical signatures 530 generated by the pre-processor 120 in operation 204, one or more logical signatures may be identical to another logical signature. For example, logical signature 504 is identical to logical signature 508, in that both signatures comprise the logical signs F, T, F, F, F.

[0097] Logical signature 504 corresponds to the logical expression ‘Are you a G?’ as posed to data items in Location 2 of the dataset in the set of input datasets 302. Logical signature 508 corresponds to the logical expression ‘Are you an A?’ as posed to data items in Location 4 of the datasets 302.

[0098] Accordingly, for each instance in which the data item in Location 2 of a dataset is a ‘G’, the data item in Location 4 of the same dataset is an ‘A’. Conversely, for each instance in which the data item in Location 2 of a dataset is not a ‘G’, the data item in Location 4 of the same dataset is not an ‘A’. This constitutes a relationship between the values of the data items in Location 2 and the data items in Location 4, wherein the relationship is defined in terms of whether the data item in Location 2 is a ‘G’ and whether the data item in Location 4 is an ‘A’.

[0099] To ensure that the synthetic data is indistinguishable from the datasets in the set of input datasets, the synthetic data generator 105 can therefore infer that the same relationship should hold for the synthetic data generated by the synthetic data generator. In other words, if the data item in Location 2 of the synthetic dataset is a ‘G’, then the data item in Location 4 of the synthetic dataset should be an ‘A’, and conversely, if the data item in Location 2 of the synthetic dataset is not a ‘G’, then the data item in Location 4 of the synthetic dataset should not be an ‘A’.

[0100] If the expression ‘Are you a G?’ is applied to the data item in Location 2 of the synthetic dataset the resulting logical sign (e.g. T or F) will be the same as the logical sign resulting from applying the expression ‘Are you an A?’ to the data item in Location 4 of thesynthetic dataset. Accordingly, in some embodiments, the constraint engine applies the same constraint variable (in this example X4) to both logical signature 504 and logical signature 508.

[0101] Advantageously, consolidating the constraint variables associated with a first and a second identical logical signature such that the logical signatures are associated with a common constraint variable implicitly defines an implicit indistinguishability constraint that forces the logical sign generated by the logic solving engine 130 for the first logical signature to be the same as the logical sign generated by the logic solving engine for the second logical signature. This reduction of the number of constraint variables reduces the computational burden of the logic solving engine 140.

[0102] Alternatively, the constraint engine 130 may be configured to associated different constraint variables (e.g., xi0and x2o) to each of logical signature 504 and logical signature 508. To communicate the relationship between the values of the data item in Location 2 and the data item in Location 4 to the logic solving engine, the constraint engine may be configured to define a logical constraint such that the constraint variable associated with the logical signature 504 must be the same as the constraint variable associated with the logical signature 508 (e.g., -i Xio XOR x20).Inverse signatures

[0103] In some embodiments, the constraint engine may be configured to identify inverse logical signatures and assign inverse constraint variables to the inverse logical signatures.

[0104] In the set of logical signatures 530 generated by the pre-processor 120 in operation 204, one or more logical signatures may be inverse to another logical signature. For example, logical signature 502 (F, F, T, F, F) is inverse to logical signature 506 (T, T, F, T, T) as each logical sign within the logical signature 502 is the Boolean opposite of the corresponding logical sign within the logical signature 506.

[0105] Logical signature 502 corresponds to the logical expression ‘Are you a C?’ as posed to data items in Location 1 of the datasets in the set of input datasets 302. Logical signature 506 corresponds to the logical expression ‘Are you NOT an A?’ as posed to data items in Location 2 of the datasets 302.

[0106] Accordingly, for each instance in which the data item in Location 1 of a dataset is a ‘C’, the data item in Location 2 of the same dataset is an ‘A’. Conversely, for each instance in which the data item in Location 1 of a dataset is not a ‘C’, the data item in Location 2 of the same dataset is not an ‘A’. This constitutes a relationship between the values of the data itemsin Location 1 and the data items in Location 2, wherein the relationship is defined in terms of whether the data item in Location 1 is a ‘C’ and whether the data item in Location 2 is an ‘A’.

[0107] To ensure that the synthetic data is indistinguishable from the input dataset, the synthetic data generator 105 can therefore infer that the same relationship should hold for the synthetic data generated by the synthetic data generator. In other words, if the data item in Location 1 of the synthetic dataset is a ‘C’, then the data item in Location 2 of the synthetic dataset should be an ‘A’, and conversely, if the data item in Location 1 of the synthetic dataset is not a ‘C’, then the data item in Location 2 of the synthetic dataset should not be an ‘A’.

[0108] If the expression ‘Are you a C?’ is applied to the data item in Location 1 of the synthetic dataset the resulting logical sign (e.g. T or F) will be inverse to the logical sign resulting from applying the expression ‘Are you NOT an A?’ to the data item in Location 2 of the synthetic dataset. Accordingly, in some embodiments, the constraint engine applies a constraint variable (in this example x ) to logical signature 502 and its inverse (in this example to -1x2) to logical signature 506.

[0109] Advantageously, consolidating the constraint variables associated with a first and a second identical logical signature such that the logical signatures are associated with a common constraint variable and its inverse, defines an implicit indistinguishability constraint that forces the logical sign generated by the logic solving engine 130 for the first logical signature to be inverse to the logical sign generated by the logic solving engine for the second logical signature. This reduction of the number of constraint variables reduces the computational burden of the logic solving engine 140.

[0110] Alternatively, the constraint engine 130 may be configured to associated different constraint variables (e.g., xi0and x2o) to each of logical signature 502 and logical signature 506. To communicate the relationship between the values of the data item in Location 1 and the data item in Location 2 to the logic solving engine, the constraint engine may be configured to define a logical constraint such that the constraint variable associated with the logical signature 502 must be inverse to the constraint variable associated with the logical signature 506 (e.g., X10 XOR x20).Figure 6 - Reflected unique signatures

[0111] Figure 6 illustrates a set of unique logical signatures 630 determined from the set of logical signatures 530, in accordance with an embodiment. In this embodiment, the constraint engine has generated a subset of the set of logical signatures 530 which contains only the uniquelogical signatures. Furthermore, the constraint engine has arranged the unique logical signatures such that there comprises a set of unique signatures 650 and a set of reflected signatures 660 that are each inverse to a signature of the set of unique signatures 650. For example, logical signature 655 (F T F F T) is inverse to logical signature 665 (T F T T F).

[0112] Each unique logical signature is associated with a unique constraint variable (e.g., Xi to x8) or an inverse of a constraint variable (e.g., -1X1 to ->x8). Reference numeral 690 refers to the synthetic logic signs (T, F) to be output from the logic solving engine 140.

[0113] Advantageously, the reduction from genomic information 302 into a set of unique signatures 630 may significantly reduce the volume of data that needs to be processed by the synthetic data generator. The number of unique signatures can be much less than the number of logical expressions, as many of the logical expressions may share the same signature.

[0114] The efficiency of this reduction is particularly desirable for genomic information because the number of logical expressions (4 positive logical expressions and 4 negative logical expressions for each data location in a dataset) over the span of a genome can be quite large, and millions of variants, and the possible pairs of these logical expressions, considered by the constraint engine (see below) when determining the indistinguishability constraints, is even larger yet still.Logical constraints

[0115] In operation 210, the constraint engine 130 is configured to form, using the constraint variables, logical constraints 125 that are input into the logic solving engine 140. A logical constraint is a Boolean expression comprising constraint variables and Boolean operators (e.g., OR, AND, NOT, XOR). The constraint engine determines the logical constraints from the logical signatures. Accordingly, the logical constraints are derived from the input data and define characteristics that are held by the input data.

[0116] The logical constraints are input into the logic solving engine, which seeks to determine Boolean values for each constraint variable, such that the logical constraints evaluate to True. In other embodiments, the synthetic data generator may be configured to determine logical constraints that are intended to evaluate to False, and the synthetic data generator may be configured to determine Boolean values for each constraint variable, such that the logical constraints evaluate as False.

[0117] In one embodiment, the output 135 of the logic solving engine comprises Boolean values (e.g., T or F) for each of the constraint variables, wherein the values of the constraint variables are such that each logical constraint evaluates as True.

[0118] In embodiments, different types of logical constraints maybe determined by the constraint engine, to force the logic solving engine to determine synthetic data with different desirable characteristics.Sensibility constraints

[0119] In an embodiment in which the pre-processor 120 is configured to apply a negative logical expression (e.g., ‘Are you NOT an A?’) to a data item as well as apply a positive logical expression (e.g., ‘Are you an A?’) to the data item, the constraint variable associated with the positive logical expression (e.g., 444) should be inverse to the constraint variable associated with the negative logical expression (e.g., 446). This relationship is referred to as a sensibility constraint.

[0120] In embodiments, the constraint engine is configured to assign a constraint variable (e.g., X6) and its inverse (e.g., -1X6) to represent this inverse relationship. Accordingly, a set of synthetic logical signs (e.g., a set of distinct values for each constraint variable) provided by the logic solving engine will implicitly satisfy the inverse relationship between each constraint variable and its inverse. Advantageously, the constraint engine 130 does not need to input additional logical constraints (e.g., Boolean expressions) into the logic solving engine to ensure sensibility constraints are satisfied.

[0121] Alternatively, in some embodiments, the constraint engine 130 may be configured to associated different constraint variables (e.g., xi0and x2o) to each of 444 and 446, and to communicate the inverse relationship between these constraint variables, the constraint engine may input a Boolean expression into the logic solving engine, wherein xi0XOR x2o is True.

[0122] Advantageously, by utilising a single constraint variable and its inverse, the number of constraint variables considered by the constraint engine and the logic solving engine is reduced. The reduction of the number of constraint variables can reduce the computational burden of the constraint engine and / or the logic solving engine 140.Reverse readability constraints

[0123] In some embodiments, the constraint engine is configured to determine reverse readability constraints to ensure that the logic solving engine (140) produces output (135) whichis uniquely interpretable as a synthetic dataset (320) of the set of specified data values. For example, a synthetic data item cannot be not equal to any of the specified values.

[0124] The set of logical signatures 432 are determined in response to applying the positive logical expressions 422 to the data items located in Location 1 of each dataset. Similarly, the set of logical signatures 434 are determined in response to applying the negative logical expressions 424 to the data items located in Location 1 of each dataset. The logical signatures in set 432 are associated with, respectively, constraint variables -ix6, Xi, x4and x2. The logical signatures in set 434 are each associated with a respective inverse constraint variable of the constraint variables -ix6, xi ;x4and x2. In particular, the logical signatures in the set of logical signatures 434 are associated with constraint variables x6, -1X1, ->x4and -ix2.

[0125] For each data item, the value of the data item is one of four specified values (A, C, G and T). Accordingly, for each data item, a logical sign determined in response to applying one of the logical expressions 422 to the data item will be True (T). In other words, for each data item, it will be True that the data item is either an A, or a C, or a G, or a T.

[0126] For the synthetic dataset, the value of the synthetic data item in Location 1 will be either an A, or a C, or a G or a T. Accordingly, the logical constraint 450 (->x6OR x4OR x4OR x2) will be True.

[0127] Logical constraint 450 comprises a reverse readability constraint, which constrains the synthetic data value at Location 1 to be at least one (NOTE: alternative wording = “not none”, perhaps linking with language in

[0093] ) of the set specified values (A, C, G or T).

[0128] The constraint engine 130 is configured to determine a reverse readability constraint for each location in the datasets of the set of input datasets 302. Referring to Figure 5, logical constraints 560 comprise reverse readability constraints.Indistinguishability constraints

[0129] In some embodiments, the synthetic data generator 102 is configured to generate synthetic data that is indistinguishable from the input dataset by also satisfying all inter-data relationships that exist in the input dataset that can be deduced by any pairwise comparison of logical signatures (520). As identified in

[0057] the comparison of logical signatures can be used to identify inter-data relationships. In this context, as identified in input datasets (302) there is no dataset that features both a ‘G’ at location 3 and a ‘G’ at location 4, this relationship is manifest in the pair of signatures: FTTFT in response to logical expression “Are you not a G”at location 3, and signature TTFTF in response to logical expression “Are you not a G” at location 4 - these signatures are present in (530) and identified also as (720) and (730). For the synthetic dataset to also satisfy this specific relationship would be for it to also not, have a ‘G’ at location 3 and a ‘G’ at location 4.

[0130] Consequently If, when considering a pair of logical signatures, there are no positions in the signatures which feature logical signs that are both False, then the synthetic logical signs should be True for one (or both) of the associated logical expressions. In some embodiments, the constraint engine is configured to compare all pairs of logical signatures and where there is no such a False / False conditions, a constraint is added between the associated constraint variables so that one (or both) of the associated synthetic logical signs are True. - these are indistinguishability constraints, and are manifest as a 2-clause constraint., wherein ‘for a pair of logical expressions associated with a dataset location, there is no pair of logical signs where both logical signs are False’ holds, set an indistinguishability constraint that prohibits the synthetic logical signs resulting from both of those logical expressions being False.

[0131] In another embodiment, the constraints engine may be configured to compare all pairs of logical signatures and where a True / True condition, wherein ‘for a pair of logical expressions associated with a dataset location, there is no pair of logical signs where both logical signs are True’ holds, set an indistinguishability constraint that prohibits the synthetic logical signs resulting from both of those logical expressions being True.

[0132] In another embodiment, the constraints engine may be configured to compare all pairs of logical signatures and where a True / False condition, wherein ‘for a pair of logical expressions associated with a dataset location, the logical signs associated with the first one of the logical expressions is True, and the logical signs associated with the second logical expression is False’ holds, set an indistinguishability constraint that sets the synthetic logical sign resulting from the first logical expression to be True, and the synthetic logical sign resulting from the second logical expression to be False. (NOTE: continuing the trend means we need to add a False / True paragraph as well)

[0133] Advantageously, in embodiments, it may be unnecessary to consider each of these mechanisms a False / False condition, a True / True condition, a True / False condition and a False / True condition, because each unique logical signature appears with its inverse in the set of unique signatures 630. Therefore, comparing the pairs of logical signatures against these extra conditions would be duplicating work.2-clause constraints

[0134] These types of indistinguishability constraint is a two-clause logical constraint. Two- clause logical constraints constrain any solutions such that one or the other, of the constraint variables corresponding to the logical signs in response to those logical expressions must be True.

[0135] In other embodiments, the synthetic data generator (102) may be configured to generate synthetic data that is indistinguishable from the input dataset relationships by also satisfying all inter-data relationships that exist in the input dataset that can be deduced by any3-wise (or higher order) comparison of logical signatures (520), comparing these triplets (or higher order) of logical signatures and then consequently generating 3 -clause (or higher order) indistinguishability constraints. However, this may not be desirable in all circumstances as a SAT problem composed of 3-clauses (or higher order) problem can potentially be nonpolynomial (NP) hard.

[0136] Additionally, the number of triple constraints is likely to be much larger than the number of pairs and handling them all could potentially be computationally prohibitive. Extending consideration to triples would result in a significantly strengthened indistinguishability constraint and more constrained (therefore potentially harder) problem for the SAT solver to solve.

[0137] In some embodiments, the 2-clause logical constraints are dynamically generated when needed (and forgotten when not) throughout the solving process in the logic solving engine 140. This facilitates scalability as only those constraints which are needed by the logic solving engine 140to arrive at a solution are generated, and the computer memory 160 usage is minimised as the potential number of generated 2-clauses can be very large.Figure 7 - Example indistinguishability constraints

[0138] Figure 7 illustrates the derivation of indistinguishability constraints by the constraint engine 130, in accordance with an embodiment. In this example, the indistinguishability constraints are generated from the set of unique logical signatures 630 determined from the set of logical signatures 530, in accordance with an embodiment. As shown in Figure 6, each unique logical signature 630 is associated with a unique constraint variable 640 (e.g., xl to x8 and -ixl to ->x8).

[0139] In operation 210, the constraint engine 130 is configured to consider each pairwise combination of unique signatures to determine whether there exists a condition between thepair of logical signatures which reflects a relationship across all of the datasets in the set of input datasets 302. In response to determining a relationship by inspecting the pair of logical signatures, the constraint engine derives an indistinguishability constraint 760, to be input into the logic solving engine.

[0140] For clarity of illustration, Figure 7 illustrates only a subset of the pair combinations of unique signatures that may be considered by the constraint engine.

[0141] In the example illustrated in Figure 7, each pair of logical signatures is comprised of five pairs of logical signs. Reference numeral 710 illustrates a pair of logical signs for the pairing of unique logical signature indexed 6 (reference numeral 720) and unique logical signature indexed 13 (reference numeral 730).

[0142] Considering the pairs of logical signs in the logical signatures 704, the constraint engine determines that there is a False / False condition that holds across all of the logical signs . Whereas, considering the pair of logical signs in the logical signatures 708 (associated with constraint variables x5 and ->x4,) the constraint engine can determine that, for each pair of logical signs, at least the logical sign from logical signature 4 (x5) is True, or the logical sign from logical signature 12 (->x4) is True (or both). Therefore, the constraint engine derives the indistinguishability logical constraint x5 OR -ix4.

[0143] In embodiments, the constraints engine is configured to consider all pairings of logical signatures 630 to derive indistinguishability constraints. Reference numeral 770 illustrates a plurality of indistinguishability constraints that are derived by the constraint engine from comparing pairs of logical signatures.Diversity constraints

[0144] In some embodiments, the constraint engine 130 is configured to determine diversity constraints, to be input into the logic solving engine. Diversity constraints are subject to customisation depending on what the input data represents. Diversity constraints ensure that the output synthetic data does not exactly replicate any of the real input datasets but satisfies at least some number (X) of logical expressions differently than all the members of the real input dataset. To encode these constraints, the constraint engine considers each entry of the input data in turn and adds a diversity constraint specifying that the synthetic data should satisfy at least the number (X) logical expressions differently than each input dataset. The number X may be configurable, based on a desired level of diversity in the synthetic data. The number X may bevaried (e.g. randomised within a range) to provide diversity variation across the input datasets, or across iterations of method 200. The number X may define a range or a plurality of numbers.

[0145] In one embodiment, the constraint engine 130 is configured to derive a diversity constraint to be satisfied by the LSE, wherein the diversity constraint is defined in terms of the diversity constraint variables derived from the constraint variables 550. The constraint variables 550 each represent a value of a synthetic logical sign 540.

[0146] Figures 10 and 11 illustrate the derivation of an example diversity constraint. In particular, Figure 10 illustrates a set of logical signs 1002 associated with Person 0 and a set of logical signs 1004 associated with Person 1, in accordance with an embodiment. The logical signs were determined by the synthetic data generator by applying the logical expressions 520 to the data items in the input datasets associated with Person 0 and Person 1, respectively. Each logical sign is associated with one of the constraint variables 550, which each represent a value of a synthetic logical sign.

[0147] Figure 11 illustrates the comparison of constraint variables, associated with the synthetic logical signs, with the logical signs derived from the input dataset associated with Person 0, and with the logical signs derived from the input dataset associated with Person 1, in accordance with an embodiment. Synthetic logical sign 1102 is indicated by constraint variable -ixi. The logical sign derived from the data item in Location 4 under logical query “are you not a ‘C’” 1 of the input dataset associated with Person 0 is indicated by reference numeral 1104.

[0148] In the example illustrated in Figures 10 and 11, the diversity constraint defines that for at least X of the synthetic logical signs (1102) to be different from the logical signs of Person 0(1104), then at least X of the diversity constraint variables 1116 should be equal to the Boolean value True. The diversity constraint variables are derived directly from the constraint variables 550 and the logical signs 1002. The diversity constraint variables each represent a situation in which the synthetic logical sign derived by the LSE 140 differs from the corresponding logical sign associated with an input dataset. Accordingly, in response to the logical sign (e.g. 1104) being True, the diversity constraint variable 1106 is equal to the inverse of the associated constraint variable 1102. As the constraint variable 1102 is -1X1, then the diversity constraint variable 1106 is equal to Xi. In response to the logical sign being False, the diversity constraint variable is equal to the constraint variable. The constraint engine can ensure that the synthetic data is diverse from all the input data by adding one diversity constraint per input dataset.

[0149] In some embodiments, the method 200 is performed iteratively, and the constraint engine is configured to add diversity constraints at each iteration to ensure that subsequently generated synthetic data maintains diversity from the synthetic data generated in previous iterations. In this way it is possible to generate a spread of synthetic data each maintaining distances from each other.

[0150] For clarity, only a subset of the constraint variables 550, logical signs 1002 and constraint variables 1116 are illustrated in Figure 11. Additionally, for clarity, only the derivation of the diversity constraint for the input datasets associated with Person 0 and Person 1 are illustrated in Figure 11. In embodiments, the constraint engine is configured to derive the diversity constraint based on a consideration of all the constraint variables, logical signs, and input datasets.Other logical constraints

[0151] In some embodiments, additional logical constraints may be input into the logic solving engine 140 to tailor the properties of the output synthetic data. For example, in an embodiment in which the synthetic data comprises genomic data, additional logical constraints may be defined to generate synthetic genome data that reproduces a particular ethnicity, and this synthetic data can be used to augment existing genome datasets to provide ethnic balance. In some application, stronger or different kinds of diversity constraints may be desired such as to ensure that the synthetic data that is generated is different from the input datasets by different measures, or alternatively to ensure the synthetic data is not different to a specific input dataset by some measure. In some embodiments, the logic solving engine 140 is also configured to be biased in the solutions which it resolves, which satisfy its input constraints, to tailor the probable properties of the output synthetic data.Figure 8 - Logic solving engine inputs

[0152] In operation 212, the logic solving engine 140 is configured to receive the logical constraints from the constraint engine 130 and determine a value for each of the constraint variables, wherein the values of the constraint variables satisfy the logical constraints.

[0153] Figure 8 illustrates the logical constraints input into the logic solving engine, and the synthetic values output by the logic solving engine for each of the constraint variables, in accordance with an embodiment.

[0154] The logical constraints comprise the reverse readability constraints 560, the indistinguishability constraints 760 (illustrated in part only), and any other constraints 860 as determined by the constraint engine 130.

[0155] The synthetic logic signs 890 comprise a Boolean value for each of the constraint variables Xi to x8. The value of the inverse constraint variables -1X1 to -ix8can be determined by inversing the synthetic values for constraint variables Xi to x8.

[0156] Each of the logical constraints 560, 760 and 860, are satisfied (e.g. can be evaluated as True) when the synthetic logic signs 890 are applied in place of their respective constraint variables.Two-clause constraints

[0157]

[0158] A clause is an example of a logical constraint in a SAT problem. A clause is composed of literals, where the constraint is satisfied if any one (or many) of its literal expressions are TRUE. A literal is an expression of a constraint variable or its negation. If a literal expression is TRUE, it is referred to as being ‘set’. If a literal expression is FALSE, it is referred to as being ‘resolved’. A clause is thus satisfied if any of its literals are ‘set’. A SAT problem in ‘Conjunctive Normal Form’ (CNF) is entirely composed of a set of clauses, and such a problem is solved if all its clauses are satisfied, which occurs if relevant literals are set such that all the clauses are satisfied. Contrastingly such a SAT problem is not solved, if there is any clause that is unsatisfied, specifically that there is a clause where all of its literals are resolved.

[0159] Two-clause constraints are clauses composed of exactly two literals and these interact productively with an innovation in backtracking search SAT solving algorithms called ‘unit propagation’, where during the process of assigning variable values, it may happen that all-but- one of the literals in a clause are resolved, in this context the only way the clause can be satisfied (without backtracking on previous assignments) is by setting the remaining literal. Since backtracking search algorithms explore the space in which all the clauses could be satisfied, the SAT solving algorithm can therefore immediately proceed to set the remaining literal to satisfy the clause. In this context, for a clause of size N, if N - 1 literals are resolved then the remaining variable will be set.

[0160] This process of setting the last remaining literals to satisfy clauses - called unit propagation - can then be instrumental in triggering other unit propagations in other clauses, that can rapidly determine the values of many of the variables in the problem. This effect isparticularly notable when there are many clauses of size 2, where if any literal of a two-clause is resolved then it immediately triggers a unit propagation which sets the value of the remaining variable. In this way clauses of size 2 are quite powerful as they readily lead to unit propagations. Particularly clauses of size 2 are sometimes called implications, since in propositional logic the implication A => B (where A and B are literals) is equivalent to the constraint NOT-A OR B and vice versa.

[0161] A satisfiability problem composed of entirely two-clauses is referred to as a 2SAT problem and may be solvable in polynomial time by backtracking search processes.

[0162] Advantageously, the majority of the logical constraints that are input into the logic solving engine are indistinguishability constraints 760, which are two-clause constraints. This means that the logical constraints define an almost 2SAT problem, which can be solved very efficiently by backtracking logic solving engines. Due to this efficiency, the synthetic data generator 102 can readily scale to larger problem sizes, for example on the scale of genome data.Figure 9 - Synthetic data output

[0163] Figure 9 illustrates the determination, by the post processor 150, of the values of the synthetic data items based on the synthetic logic signs 890, in accordance with an embodiment.

[0164] The post processor determines the value of the synthetic data item located at Location 1 of the synthetic dataset by determining which of the constraint variables associated with the positive logical expressions for Location 1 is True. Due to the sensibility constraints, the post processor need not consider the constraint variables associated with the negative logical expressions to determine the value of each synthetic data item; however, consideration of the constraint variables associated with the negative logical expressions can indicate whether an error has occurred.

[0165] Applications

[0166] Embodiments of the synthetic data generator 102 may be applied to generate synthetic data other than genomic data. The method 200 applied by the synthetic data generator and the set of logical expressions which generates logical signatures upon which a logic solving engine processes solutions, are agnostic to what the input data represents. Accordingly, the syntheticdata generator can generate synthetic data based on any input data that is arranged (or has been translated) as datasets, wherein each data item comprises a value of a set of specified values.

[0167] In an embodiment, the synthetic data generator may be applied to generate synthetic digital objects such as videos and / or images, by encoding key features of such objects into binary representations using explicit or implicit dimensionality reduction techniques, such as an neural network autoencoder with a quantized middle layer.

[0168] In an embodiment, the synthetic data generator may be applied to generate synthetic digital objects such as videos and / or images, by encoding key features of such objects into binary representations using an explicit or implicit dimensionality reduction technique.

[0169] In embodiments, the synthetic data generator may be applied to power system applications, to generate private synthetic data representative of household power consumption and generation patterns. Continuous power data may be thresholded into specified representations as datasets with data items of the set of specified values. Synthetic power data may be shared with networks or interested parties for network modelling and control (e.g., demand response, load shedding, network infrastructure planning and placement).

[0170] In embodiments, the synthetic data generator may be applied to supply chain modelling. Supply chain relationships may be represented as a graph of connections, which can be represent as adjacency matrix, to be flattened. The adjacency matrices may be represented as a dataset of specified values, from which the synthetic data generator can generate synthetic data that is representative of confidential supply chain business relationships. The synthetic data may be considered to be private and may be shared with government agencies (for resource allocation) without violating confidentiality of specific business agreements.

[0171] In embodiments, the synthetic data generator may be applied to survey data modelling applications. The synthetic data generator may be configured to generate synthetic survey data based on binarised survey data. The synthetic data generator may be configured to avoid producing synthetic data that comprises unlikely or unfeasible pairs of binary traits (e.g., driving infringements associated with infants). The synthetic survey data may be considered to be private and can be shared with other parties without violating the privacy of the actual survey respondents.

[0172] In embodiments, the synthetic data generator may be applied to creation of synthetic faces. Attributes of a subjects’ faces (such as feature sizes, orientations and spacings) may be defined in terms of a compact binary representations (by neural network, PCA clustering,eigenface decomposition, etc.) that are presented as datasets of specified values. The synthetic data generator may be configured to generate synthetic data which defines a synthetic face based on attributes of the input subjects’ faces.Privacy considerations

[0173] A level of privacy afforded by a synthetic dataset, as it pertains to protecting the particular input dataset used to generate the synthetic data, may comprise an indication of a level of determinability of that particular input dataset and / or its membership in the context of a range of other input datasets and their members that may have been used to generate the synthetic data. A level of determinability of the membership composing the input datasets used to generate the synthetic data amoung a superset of potential members and the combinations of possible input datasets used to generate synthetic data, (and therefore a determination of the privacy of the synthetic data) may be determined by a privacy engine, as described in relation to Figures 14 to 21.

[0174] An application of the synthetic data generator is to the production of private synthetic data, which is facsimile synthetic data that possesses the statistical properties of real data, but which cannot be used to reasonably infer private information about any specific dataset input into the synthetic data engine. For example, private synthetic data may comprise synthetic genomic data that possesses the statistical properties of real people’s genomic information, but which cannot be used to reasonably infer private information about any specific person’s genomic information.

[0175] A method of adding greater degree of privacy to the generation of synthetic data is by the addition of different kinds of noise in the generation process. Higher levels of noise obscure signal information from which private inferences about the input data can be drawn.

[0176] Noise can be added in a range of different ways, in the context of an embodiment, such as by one or more of: randomly relaxing and / or adding constraints into the logic solving engine (140); adding randomised alterations or mutations to the input dataset prior to processing (106); including entirely randomised genomes into the dataset; and including synthetic genomes into the input dataset.

[0177] Noise-adding processes generally introduce more privacy into the generating process, as any resulting features may then be less informative about the original input dataset. However, too much noise also has the potential to destroy the fidelity of the synthetic data to accuratelyrepresent the features of the real data. However, some amount of noise can also potentially enhance the fidelity of the synthetic data in its representation of the features of the real data.

[0178] In some embodiments, the generation of indistinguishability constraints (illustrated in Figure 7) can be randomised to achieve greater privacy. A strengthening count may be defined such that the strengthening count modifies the generation of indistinguishability constraints

[0179] By employing these methods, the output from synthetic data generator can be potentially more private, as less may be inferred about the input data because the synthetic data was generated from randomised constraints.

[0180] In some embodiments, the privacy of the synthetic output data may be enhanced by feeding the synthetic data output from former iterations of method 200 into the input dataset of future runs. This reprocessing of synthetic data may compound the randomness inherent in generation of further synthetic data and may afford privacy by making reverse inferences about the input real data from witnessed synthetic data output, quite difficult.

[0181] Furthermore, the indistinguishability constraints are witnessed to naturally attenuate the presence of rare variants and pairwise combinations of variants. This reprocessing is witnessed to propagate the stochastic attenuation of pairs, into the attenuation of rare triples and potentially quadruples of variants.

[0182] This process of attenuating rare combinations of variants may serve to enhance the privacy of the synthetic data result, as the presence of rare information in the witnessed synthetic output is seen as a principle means of linking input data to with real people.Membership inference

[0183] Given a synthetic dataset that has been generated by a synthetic data generator, an entity or person may be interested in knowing whether a particular input data set was present in the set of input datasets used to generate the synthetic dataset.

[0184] Membership inference is the determination of whether a target dataset (or data item) is present in a larger set of datasets (or data items). Membership inference may comprise a determination of certainty regarding the membership of the target dataset in the set of input datasets or may merely comprise an indication of likelihood (or probability) of the target dataset being present in the set of input datasets.

[0185] For privacy reasons, it may be desirable to obscure the fact that a particular input dataset (e.g. a target input dataset) was present in the set of input datasets used by a syntheticdata generator to generate a synthetic dataset. Accordingly, it may be desirable to be able to prevent the determination of certainty that a target input dataset is present in a set of input datasets.

[0186] Because a synthetic data generator 102 utilises a deductive process (e.g. SAT solving) it is possible to consider the constraints and techniques applied by the synthetic data generator to generate a synthetic dataset and deduce backwards to obtain information regarding the set of input datasets that may have been used by the synthetic data generator to generate that synthetic dataset. Particularly it may be possible to derive answers to questions about the possible sets of input data sets which could (and could not) have given rise to witnessed synthetic datasets that are output from the synthetic data generator 130. From a consideration of the possible sets of input datasets, a determination may be made as to the effective privacy provided by the synthetic data generator.

[0187] Provided herein is a privacy engine that is configured to conduct an analysis of a synthetic dataset that was derived from a set of input datasets and provide an indication of whether one could infer the membership (e.g. presence) of any particular input dataset in that set of input datasets. The set of input datasets from which the synthetic dataset was generated is referred to as the actual set of input datasets.Figure 14 Privacy engine block diagram

[0188] Figure 14 illustrates a system for evaluating the privacy of input datasets, in accordance with an embodiment. The system 1400 comprises one or more processors. A processor may comprise one or more microprocessors, microcontrollers or controlling circuitry. The one or more processors of the system may be, in combination or individually, configured to execute program code stored within the memory to control the functionality of the system.

[0189] The system 1400 comprises a privacy engine 1402 and a controller 1404 configured to control the privacy engine.

[0190] The privacy engine further comprises a constraint engine 1430 and a logic solving engine (LSE) 1440. The privacy engine 1402 is configured to use a logic solving engine 1440 to deductively generate a plurality of potential sets of input datasets, which could have given rise to the synthetic dataset 1420.

[0191] The constraint engine is configured to define one or more logical constraints 1435 to be input in a logic solving engine. The constraint engine determines the one or more logical constraints based on input information including: a synthetic dataset 1420 which was generatedby a synthetic data generator, based on a set of input datasets; one or more known logical constraints 1422 which were known to have been applied by the synthetic data generator to generate the synthetic dataset 1420; and a set of candidate input datasets 1424, wherein each input dataset in the set of candidate datasets were or were not used by the synthetic data generator to generate the synthetic dataset 1420.

[0192] The logical constraints are defined in terms of one or more constraint variables. The logic solving engine is configured to determine values for the constraint variables that satisfy the one or more logical constraints.

[0193] In some embodiments, the privacy engine further comprises a post processor 1450, configured to convert and / or post-process the output 1445 of the logic solving engine for use by the privacy evaluation module 1460. In some embodiments, privacy evaluation module takes as input a target input dataset 1470.

[0194] The memory 1480 may comprise one or more volatile or non-volatile memory types. For example, the memory may comprise one or more of random-access memory (RAM), readonly memory (ROM), electrically erasable programmable read-only memory (EEPROM) or flash memory. The memory is configured to store, in part or in full, program code (also referred to as machine readable code or software) accessible by the processor(s) of the privacy engine 1402. The program code may comprise executable code modules. The executable code modules, when executed by the processor(s) cause the system 1400 to perform the functionality according to the described embodiments.Figure 15 - Evaluating privacy of input datasets

[0195] Figure 15 is a flowchart illustrating a method 1600, as performed by the privacy engine 1402, to evaluate the privacy of input datasets used to generate synthetic data, in accordance with an embodiment. The operations of method 1600 are described in relation to Figures 14 and 16 to 21, which illustrate an example embodiment of the present technology.

[0196] In some embodiments, a synthetic data generator 102 may comprise a privacy engine 1402 and may perform the operations of method 1600 in response to the generation of a synthetic dataset.Figure 16 - Hypothetical attacker

[0197] In privacy analysis, it can be useful to refer to a hypothetical attacker, and what such an attacker could infer under various assumptions about their knowledge of the process anddata to achieve a goal. Figure 16 illustrates a scenario in which a hypothetical attacker 1602 seeks to determine the input datasets used by a synthetic data generator 102 to generate a synthetic dataset, in accordance with an embodiment.

[0198] One minimal goal of a hypothetical attacker is to infer if a specific input dataset was part of the actual set of input datasets that was used by a synthetic data generator to generate a synthetic dataset. This may be referred to as a Membership Inference Attack (MIA) and is a more minimal criterion than other privacy attack models (such as Attribute Inference, and Reconstruction Attack models where the attacker is attempting to infer more specific information about the target individual than just if any of such information from that target individual vas evidenced at all.

[0199] In embodiments, the privacy engine is configured to provide an analysis of the potential sets of input datasets that could have been used to generate a particular synthetic dataset. In embodiments, the privacy engine is configured to provide this analysis with some knowledge regarding the generation of the synthetic dataset. This knowledge may comprise: a set of candidate input datasets comprising one or more input datasets from the actual set of input datasets; constraints applied by the synthetic data generator (e.g. 102) used to generate the synthetic dataset; and / or the number of input datasets in the actual set of input datasets.

[0200] In some scenarios an attacker may have full knowledge about synthetic data generator 102 and may have full knowledge of the actual set of input datasets 106, and the outputted synthetic dataset 1420, and a particular target input dataset (1470) in the set of candidate input datasets (1424), and only seeks the minimal information about whether the target input dataset is in the actual set of input datasets or not.

[0201] Accordingly, the preponderance of the presence of the target input dataset in the set of candidate datasets by the privacy engine 1430, can act as a proxy for the confidence that the target dataset was part of the actual input dataset used to derive the synthetic dataset.

[0202] In some embodiments, the set of potential input datasets comprises some or all of the actual input datasets used by a synthetic data generator to generate the synthetic dataset. In some embodiments, the potential input dataset comprises randomly generated input datasets. In some embodiments, the potential input dataset comprises input datasets that were not used by a synthetic data generator to generate the synthetic dataset.

[0203] The process applied by the privacy engine provides a proxy indication of the relevant privacy of the datasets in the set of actual datasets, as if there is no consensus in thecombinations of the input datasets that could have given rise the synthetic dataset, then the privacy of the input datasets may not be regarded to be violated by the synthetic dataset.

[0204] The privacy engine produces one or more combinations of potential input datasets which could have given rise to the synthetic data, and from these potential input datasets, the privacy engine analyses the number of times that each of the potential input datasets occurs among these combinations. The degree that a input dataset is present in these combinations of potential input datasets gives a proxy indication of the confidence by which a hypothetical attacker could have about the input dataset being a part of the actual set of input datasets from which the synthetic dataset was generated.

[0205] For instance, if the privacy engine determines that there is an input dataset that occurs in all of the combinations of potential input datasets that could have given rise to the synthetic data, then the synthetic dataset is potentially highly revealing of that particular input dataset’s presence in the actual set of input datasets. Conversely, if there is an input dataset that never occurs in any combinations of potential sets of input datasets inputs that could have given rise to the synthetic dataset, then that is strongly indicative that the input dataset is not part of the actual set of input datasets.

[0206] Between these cases, if all the input datasets occur with approximately equal degree of among these combinations of potential input datasets, then this is indicative that there are many different ways in which the synthetic dataset could have been generated with and without any specific input dataset.

[0207] If a hypothetical attacker (even having information regarding the input dataset and the settings utilized by the synthetic data generator) cannot confidently identify which input datasets were actually used, then the input datasets are considered to be private.

[0208] Advantageously, the combination of synthetic data generator 102 with the privacy engine 1402, allow for both the production of synthetic data and analysis of the privacy afforded by use of it.Privacy engine

[0209] The privacy engine constructs a logic satisfiability (SAT) problem to solve for one or many possible sets of input datasets (referred to as set X ) including the actual set of input datasets wherein the actual set of input datasets was used by a synthetic data generator to generate the synthetic dataset 1420, and possible sets of input datasets are those compatible with the synthetic data generator having generated synthetic data 1420. The privacy engine doesthis by associating and solving for binary variables x ,... ,xNwhich indicate the presence of each input dataset as part of that set X . In this context, where x{is True, then input dataset #1 is included in the set X and reversely if False, input dataset #1 is not included in the set X . Similarly, if x2is True input dataset #2 is part of set X and reversely if False, input dataset #2 is not part of set X , and so on.

[0210] To determine the values those variables x{xN, the privacy engine constructs a satisfiability problem which is only satisfiable by the variables where they indicate a set X of potential input datasets 1445 as a subset of the Candidate Input datasets 1424 which would not result in any constraint of 1435 being unsatisfied, and which could have been input into the synthetic data generator to yield synthetic dataset 1420. In this way, the set X (as indicated by the variables j,...,^ ) are each considered, and all the combinations of values for those variables x^- . ^x,^ which would yield constraints in the synthetic data generator that would have prohibited the synthetic data generator from generating the synthetic dataset 1420 are excluded, by SAT constraints 1435 over variables. Once these SAT constraints over variables are determined, a Logic Solving Engine 1440 (in embodiments, a SAT solver) can then be employed to determine the set X , and / or run multiple times to sample solutions such as sets X, which identify sets of input datasets which are compatible with the generated synthetic dataset 1420.

[0211] So, privacy engine 1402 considers each of the constraints 125 that synthetic data generator 102 would generate from input datasets indicated by variables over the set of candidate input datasets 1424 X -.^x^ , and whether those resulting constraints would be violated by the synthetic dataset 1420.

[0212] In one embodiment, the constraints 1422 comprise a reverse readability constraint, which defines that exactly one variation is present in each loci in the synthetic dataset should be subject to immediate inspection. If this condition fails, then synthetic data generator could not yield the synthetic dataset irrespective of any values xx,...,xN.

[0213] In one embodiment, the constraints 1422 comprise a diversity constraint, which defines that the synthetic dataset should be a distance Y away from any datasets included in the set X , eliminates from consideration all input datasets within a distance Y of the synthetic dataset, these input datasets can be immediately excluded from further analysis with their x;values set to False (or they can also be excluded as a prior step), leaving a remaining set of potential input datasets with their xtvalues undetermined.

[0214] In one embodiment, the constraints 1422 comprise an indistinguishability constraint which defines that for all pairs of elementary logical conditions, that those pairs of conditions that apply (either-or-both one and the other) to all input datasets indicated by positive variables xt, . .. , xNshould also apply (either-or-both) to the synthetic dataset.

[0215] In one embodiment, the constraints 1422 comprise an contrapositive constraint, which defines that, for all pairs of elementary logical conditions, if neither are satisfied by the synthetic dataset, then there must exist input datasets indicated by positive variableswhere neither are satisfied also.Figure 17 - Candidate input datasets

[0216] Figure 17 illustrates a set of candidate input datasets 1702, in accordance with an embodiment. The set of candidate input datasets may comprise candidate input datasets 1424, as illustrated in Figure 14. The set of candidate input datasets comprises nine candidate input datasets, each candidate input dataset corresponding to one of nine people (Person 0 - Person 8). Each input dataset also corresponds to a constraint variable 1710 xi to X9. For example, input dataset 1704 corresponds to Person 3 and corresponds to constraint variable X4. Each input dataset comprises five data items, structured in an ordered list corresponding to Location 1 to Location 5.

[0217] Figure 17 also illustrates the synthetic dataset 1420, which comprises five data items. The synthetic dataset was generated by a synthetic data generator, such as synthetic data generator 102.

[0218] In contrast to the embodiment illustrated in Figure 6, in which the constraint variables 640 correspond to synthetic signs associated with unique logical signatures, for the privacy engine 1402, the constraint variables 1710 correspond to individual input datasets.Figure 18 - Logical signatures

[0219] In operation 1504, the privacy engine 1402 converts all input data items (from the candidate input datasets 1424) and synthetic data items (from the synthetic dataset 1420) into logical signatures under the range of elementary logical conditions. The signatures that privacy engine calculates, are over all the potential input datasets (that is all input datasets with variablesx ,...,xN) and the synthetic dataset 1420, that is, each i th vector position in those signatures are associated with the i th input dataset.

[0220] In embodiments, after this transformation, the input data items may no longer be required by the privacy engine, as the logical signatures are sufficient for all further processing.

[0221] Figure 18 illustrates a set of logical signatures 1820 generated by the constraint engine 1430, by applying logical expressions 1810 to the data items of the set of candidate input datasets 1702 and synthetic dataset 1712, in accordance with an embodiment.

[0222] The constraint engine 1430 is configured to apply logical expressions 1810 to each data item of each dataset of the candidate input datasets 1702, and to each data item of the synthetic dataset 1420 to determine a plurality of logical signatures 1820. For example, the constraint engine 1430 is configured to apply logical expression 1812 (Are you NOT a A?) to the data items 1712 in Location 1 of the set of datasets 1702 as well as the synthetic dataset 1420, to generate logical signature 1822.

[0223] In some embodiments, the constraint engine 1430 is further configured to remove duplicate signatures of the logical signatures 1820 to produce a set of unique signatures 1910, as illustrated in Figure 19.Logical constraints for privacy engine

[0224] In operation 1506, the privacy engine uses the logical signatures to construct all the logical constraints on constraint variables x ,...,xN, which determine the set X . Most notably, where the costraits 1422 comprise the indistinguishability constraint is over all pairs of elementary logical conditions, these logical conditions will have a signature, from set 1910, , and so the privacy engine considers all pairs of signatures in turn 1960. For each pair of signatures there will be a pair of synthetic signs from the synthetic dataset (such as 1950, 1940) , and the constraint engine 1430 of the privacy engine 1402 processes this information by applying the contrapositive constraint to the indistinguishability constraint.

[0225] Specifically, if the logical signs to both logical signatures is False (and thus the synthetic dataset does not satisfy both of the elementary logical conditions), then at least one of the variables x{, . .. , xNmust be true where the signatures at the respective vector positions are False&False (such as 1930), indicating the presence at least one input dataset where both of those elementary logical conditions are unsatisfied. Scanning over the pairs of logical signatures, the constraint engine considers if the synthetic signs to both are false, and if theyare, it checks over the positions in the logical signatures that are both false, and where they are, it generates a constraint that at least-one of the associated variables x ,... ,xNmust be true.

[0226] In some embodiments, operation 1506 may generates a lot of constraints over the variables x ,...,xN, and many of these will be logically redundant, and these logically redundant constraints may be eliminated prior to inputting the constraints into the logic solving engine 1440. The logically redundant constraints are identified by subsumption, particularly if there are two constraints, one having only a subset of the literals that the other has, then the larger is redundant. For instance, a clause x OR x2OR x3is a redundant constraint if there is another constraint x2OR x3, because if x2and / or x3is satisfied, then x OR x2OR x3is necessarily also satisfied. Eliminating all of these redundant logical constraints allows the logic solving engine to process solutions more effectively.

[0227] Once the redundant logical constraints are removed, in operation 1508 the logic solving engine 1440 is run to determine the solutions that satisfy the constraints 1435, identifying the values of variables x ...,xN, which determines the set X , which identifies a potential set of input dataset by which a synthetic data generator could have generated the synthetic dataset 1420.

[0228] In embodiments, in operation 1508 the logic solving engine 1440 may be run multiple times on the same constraints 1435, identifying a multiple sets of potential input datasets, and from this information, the preponderance of specific input datasets in those sets can be counted.

[0229] The satisfiability problem that privacy engine constructs and solves, can be augmented with additional constraints 1422 that reflect the knowledge of a hypothetical attacker, for instance, if the attacker knows how many input datasets were present in the actual set of input datasets, the satisfiability problem van be augmented with the constraint that exactly that many variables x ,... ,xNare True.

[0230] In some embodiments, privacy engine is configured to generate the set(s) of potential input datasets 1445 under the assumption that the synthetic data generator 102 that generated the synthetic dataset did not apply privacy augmentations (such as the addition of noise, or the re-addition of rare mutations, as detailed above).

[0231] In other embodiments, the privacy engine can potentially be modified to appropriately encompass these considerations. For instance, the stochastic strengthening of theindistinguishability constraint (such as illustrated in Figure 23) could link directly with a modification of contrapositive indistinguishability (as depicted in Figure 19).Figure 19 Logical constraints example

[0232] In one embodiment, based on the premise that the synthetic dataset 1420 was generated by a logical solving engine that was configured to satisfy one or more 2-clause indistinguishability constraints (such as those described in relation to Figure 7), the constraint engine 1430 is configured to determine logical constraints (herein referred to as indistinguishability constraints) that define a 2-clause indistinguishability relationship between the dataset and the candidate input datasets.

[0233] Figure 19 illustrates the derivation of indistinguishability constraints comprising 1435 by the constraint engine 1430, in accordance with an embodiment. In this example, the indistinguishability constraints are generated from the set of unique logical signatures 1910 determined from the set of logical signatures 1820, in accordance with an embodiment. As shown in Figure 19, each logical sign in the logical signatures 1910 is associated with a unique constraint variable 1902 (e.g., Xi to x9). For example, logical sign 1920 (representing False), from unique logical signature 4 (1910) is associated with constraint variable Xi.

[0234] In operation 1506, the constraint engine 1430 is configured to consider each pair combination of unique signatures 1910 to determine whether there exists a (2-clause indistinguishability) relationship between the pairs of logical signs in the pairs of logical signatures, that corresponds to a relationship between a pair of logical signs associated with the synthetic dataset. In response to determining a relationship, the constraint engine derives an indistinguishability constraint, defined in terms of the constraint variables 1902, to be input into the logic solving engine 1440.

[0235] In the example illustrated in Figure 19, each pair of logical signatures is comprised of nine pairs of logical signs, derived from the candidate input datasets 1702, and a pair of logical signs derived from the synthetic dataset 1420. Reference numeral 1930 illustrates a pair of logical signs from logical signature 0 and logical signature 2 for the input dataset associated with Person 0 (xi). Reference numeral 1940 illustrates a pair of logical signs from logical signature 0 and logical signature 2 for the synthetic dataset.

[0236] The constraint engine considers whether a relationship that exists for a pair of logical signatures derived from the synthetic dataset (synthetic logical signs) holds across any other pair of logical signatures derived from a candidate input dataset.

[0237] For example, the logical signs 1940 are both False. Accordingly, the constraint engine 1430 considers whether any other pair of logical signs from logical signature 0 and logical signature 2 also embody this False:False relationship. In the example illustrated in Figure 19, the logical signs associated with the input datasets associated with Person 0 (xi), Person 1 (X2), Person 3 (x4), Person 4 (xs), Person 5 (xe), Person 7 (xs) and Person 8 (xg) also embody this False:False relationship. Accordingly, the constraint engine may assume that for the synthetic dataset to be indistinguishable from the set of input datasets considered by the synthetic data generator 102 to generate the synthetic dataset, the set of input datasets comprises at least one of the candidate input datasets xi, X2, X4, xs, X6, xs or X9. Therefore, the constraint engine derives the logical constraint 1912.

[0238] Similarly, the constraint engine derives logical constraint 1906, which states that the set of input datasets must comprise candidate input datasets xi, X2, X4, xs, X6, or X9, and logical constraint 1908, which states that the set of input datasets must comprise candidate input datasets X3 or xs.

[0239] For the pairing of logical signatures 4 and 5, there is no False:False relationship between the pair of logical signs 1950 associated with the synthetic dataset. Accordingly, the constraints engine does not derive a logical constraint from logical signatures 4 and 5. Similarly, for the pairing of logical signatures 5 and 28, there is a True:True relationship, rather than a False:False relationship, between the pair of logical signs associated with the synthetic dataset. Accordingly, the constraints engine does not derive a logical constraint from logical signatures 5 and 28.

[0240] In some embodiments, the constraint engine 1430 may be configured to derive logical constraints based on True:True relationships, or True:False relationships between the logical signs associated with the synthetic dataset, rather than based on False:False relationships. However, as each logical expression 1810 also has an inverse logical expression (e.g. Are you a G? and Are you NOT a G?) a logical signature 1820 is produced for each scenario and its inverse. Therefore, the comparison of each pairing 1960 of logical signatures includes a comparison between each of the signatures and also their inverse. Accordingly, it may be sufficient for the constraint engine to be configured to consider only the False:False relationships.

[0241] For clarity of illustration, Figure 19 illustrates only a subset of the pair combinations of unique signatures that may be considered by the constraint engine; however, in someembodiments the constraint engine is configured to consider all pairs of combinations of the unique signatures 1910 to derive logical constraints.

[0242] In some embodiments, the constraint engine is configured to combine and logically reduce the logical constraints (e.g. 1906, 1908 and 1912). For example, a constraint specifying ‘A or B’ makes constraint ‘A OR B OR C’ redundant by subsumption.

[0243] In some embodiments, the constraint engine may be provided with a known constraint 1422 which indicates the number (N) of input datasets in the actual set of input datasets. Accordingly, the constraint engine may define a constraint such that ‘exactly N of variables xi, X2, X3, X4, X5, X6, X7, xs or X9 are True.

[0244] The indistinguishability constraints illustrated in Figure 19 are two-clause logical constraints. In other embodiments, the constraint engine may be configured to consider triplets (or higher order) of logical signatures and then consequently generating 3 -clause (or higher order) indistinguishability constraints.Figure 20 - Potential input datasets

[0245] Figure 20 illustrates an exhaustive list of combinations of solutions of constraint variables xl ...x9 (where those variables are True, otherwise False), as output by the logic solving engine 1440, according to an embodiment. Each combination of constraint variables satisfies the logical constraints 1435 input into the logic solving engine. Each combination of constraint variables indicates a set of candidate input datasets which may have been present in the actual set of input datasets used to derive the synthetic dataset 1420. For example, in accordance with combination 2002, the logic solving engine considers that it satisfies the logical constraints 1435 that the input dataset comprises candidate input datasets X2 AND X3 AND X4 AND xs AND X9.

[0246] As the constraint variables X2, X3, X4, and xs occur as evaluated to be True in all the solutions from the logic solving engine 140, the candidate input datasets associated with these constraint variables are, by logical certainty, a part of the actual set of input datasets used to generate the synthetic dataset 1420 (under assumption that the Candidate input datasets 1424 subsumes the actual set of input datasets that were used to generate the synthetic dataset and identifies all possible inputs to be considered). These candidate input datasets are the input datasets associated with Person 1, Person 2, Person 3 and Person 4. Accordingly, these input datasets may be considered to be privacy violated. A privacy evaluation for the synthetic data, in this situation, may indicate no privacy (e.g. a 0% level of privacy) for these input datasets.Figure 21 - Privacy evaluation

[0247] Figure 21 is a graph illustrating a privacy evaluation in the form of a posteriori probability that a candidate input dataset is present in the actual set of datasets used to generate the synthetic dataset 1420, in accordance with an embodiment. The mathematics used to generate such Figure 21 is encoded as Theorem 1.

[0248] With the assumption that all the potential sets of input datasets 1445 are about equally likely to generate the synthetic dataset, then: xi, X6, X7, xs, X9 are then equally likely to be present in the actual set of input datasets. In one embodiment, xi, X2, X3, X4, xs was actual set of input datasets.Theorems

[0249] Theorem 1 In the combinations Ckof inputs provided by Privacy engine that are compatible with event O of the witnessing of output o (e.g. synthetic data) from a synthetic data generator, the ratio Z of the counts of that feature the target individual i over those that do not, is a proxy indicator for the confidence an attacker should have for the inference I that individual i was part of the dataset that yielded o , as:

[0250] Proof. If O is the proposition that output o is observed to given by the synthetic data generator, I is the proposition that the target individual i (e.g. target input dataset) is part of a combination of input datasets that was used by synthetic data generator to generate o , We consider that for any combination C of input datasets, that f C) is the set of possible outputs from the synthetic data generator. We consider sets of all possible input Ckwhich are compatible with O , ie. V, o e / ( ) .

[0251] We can consider Bayes Theorem:

[0252] This inner term can be expanded over the set of possible input sets Ck, as:k _ (viaBayesTheoremagain)^(OIGW IGMG)

[0253] If each combination Ckis equally likely to have been selected, and if the logic solving engine is unbiased (or it is desirable to assert vagueness assumptions about its biasing) then nooutput is more likely than any other, and: p(O | Ck) = and hence:

[0254] From this expression, it can be seen that if the Ratio of Average Inverse Image Sizes R « 1 then then the Ratio of Counts Z is the only relevant factor information the confidence that an attacker should have about any individual input dataset i having being used to generate witnessed output o .

[0255] From this theorem, it can be seen that the use of counts as a proxy indicator of the confidence that a hypothetical attacker should have about proposition I is well founded, particularly, when there is approximately equal number of solutions from the synthetic data generator with and without the presence of the target individual, over all of the eventualities of combinations Ck.

[0256] This is something which could potentially be expected to be true over most ordinary input datasets, relative to the dataset. Particularly, the degree to which the presence of a specific input dataset would influence the number of solutions that the synthetic data generator would generate may be expected to be principally in proportion to the extent to which it would break the pre-conditions of the indistinguishability constraint in the context of other input datasets, which would be loosely in proportion to how different the input dataset was in the context of the others.

[0257] In some embodiments, privacy analysis could be done specifically in the context of a set of candidate input datasets in which an individual target dataset is compared by the privacy engine to similar input datasets, and thus if the privacy engine 1402 reveals private membership of target individuals in the set of inputs that were actually used to generate the witnessedsythetic dataset, it might be said to do so, only as far as it does so above those of similar input datasets to them. Thus potentially privacy engine’s analysis should be done in the context of an input dataset where there are a range similar input datasets. And in this context, one may expect the assumption R « 1 to be even more appropriate.

[0258] In some embodiments, the calculus in Theorem 1, is under assumption that the synthetic data generator’ s logic solving engine is unbiased (or should be treated as if it were) - which may not be true in all embodiments. Real world logic solving engines can feature biases - and even strong biases. It is worth noting that in the absence of any information about the biasing of the logic solving engine used (and in the absence of justifiable vagueness assumptions) privacy engine’s counts may represent the most information an attacker could gain.

[0259] For some embodiment, there are several avenues which may yield enhancement of the information provided by privacy engine. Firstly, actually counting the number of likely solutions that the synthetic data generator would give to the combinations of possible inputs that privacy engine yields - that is directly calculating | f Ck) | terms. Counting the number of solutions that the synthetic data generator may give under a particular combination of inputs, can be done directly (by multiple iterations of the logic solving engine), or accelerated with some simplifications. Particularly as the bulk of the logical constraints used by the synthetic data generator are 2-clauses, the problem of counting the number of satisfying constraints to a 2S AT problem, known as the #2S AT problem, is known to be on the more tractable side of NP- hard. Additionally, there are dedicated SAT model counting algorithms, and also approximate SAT model counting algorithms. Additionally, not necessarily all of the | f Ck) | terms may need to be calculated as statistical averaging techniques (such as stratified by specific members in Ck) may give statistical estimate on R .

[0260] Additionally, approximating counts by training on cues may be applied. For any particular combination Ckthe resulting number of possible outputs from the synthetic data generator |f(Ck)| may be indicated by various cues from the constraint that it constructs, particularly in the context of SAT constraints, particularly the clause-to-variable ratio, median variable neighbourhood size, the average implication chain branching factor and depth, as well as the number of forced and unbounded variables. Machine learning approaches may be employed to approximate model counts on these bases and potentially could be used to approximate the counts | F (C. ) | , and / or directly for the ratio R . Additionally, the syntheticdata generator specific problem cues could be utilised - such as statistics over mean overlap of the signatures in the problem.

[0261] Additionally, approximate reformulation and SAT counting may be applied. Particularly, a Jensen’s inequality-like approximation (or something more sophisticated) may be imagined where:

[0262] Jensen’s Inequality is: i(E[X]) < E[ i(X)] for convex function (j) .We consider i(E[X]) « E[ i(X)] application to numerator and denominator with (zi(x) = 1 / x .

[0263] In this context, both the numerator and denominator of the Ratio of Models M can be directly computed as two #SAT problems, directly amenable to approximate #SAT solving routines, and the inverse Ratio of Counts 1 / Z is already known by privacy engine.

[0264] The inventors note that this kind of analysis, that is, inferring the privacy afforded by a synthetic data generation technique, by deductively generating the range of potential input datasets with and without specific individuals and algorithmically resolving the likelihood that these combinations of potential input datasets would yield the witnessed output (e.g. synthetic dataset) of the algorithm, is possible because of the synthetic data generator’s deductive approach, such as embodied in Genomator by employing SAT solvers.

[0265] In some situations, the assumptions of the knowledge about the input dataset used in privacy engine’s analysis may be beyond the degree of knowledge that a real- world privacy attacker poses about the input dataset or the target input dataset. In this way privacy engine’s analysis may be seen as a proxy for a worst-case privacy attack susceptibility. Modifications and / or simplifications to privacy engine could be made to account for weaker and more realistic information and / or different attack models that reflect the information available to, and goals of, such an attacker.Genomator

[0266] Improvements, variations and various operational configurations for the generation of synthetic data by a synthetic data generator are described herein with reference to anembodiment of a synthetic data generator referred to as Genomator, which is configured to generate synthetic genomic data.

[0267] Genomator is configured to derive synthetic genomic data where any pair of logical expressions that is satisfied by the input set of genomic datasets, is also satisfied by the output synthetic data. Accordingly, the principle of ‘indistinguishability’ is applied by the Genomator.

[0268] The output from Genomator is a processed result of the SAT solver, and the SAT solver is configured to produce an output that satisfies the constraints given to it. The constraints input into the SAT solver set out a domain of possible solutions which can be output from the SAT solver. If the constraints are too restrictive then there may be no possible solution to the SAT problem, and no possible output from the SAT solver can be given. Conversely, if the constraints into the SAT solver are too loose, there may be many possible solutions which the SAT solver may yield.

[0269] Different SAT solvers have different characteristics with regard to their tendencies to output specific solutions more likely than others, For practical purposes many SAT solvers may be sufficiently random in this output behavior, however other SAT solvers are designed to be more-uniform in their likelihood of generating possible solutions. Insofar as the output that the SAT solver resolves is uniformly random (among the solutions compatible with the constraints input into it), so too is the output from Genomator uniformly random.

[0270] Additionally, in the context of Genomator, additional constraints can be added, notably the diversity constraint, which is a constraint which may be input into the SAT solver to ensure that the synthetic data output from Genomator will be sufficiently different from each of the input datasets by some specified measure.

[0271] The combination of these two factors, a) randomness from the SAT solver, and b) diversity of the output from each of the input datasets, may be sufficient in some circumstances for the synthetic data to be considered to be sufficiently ‘private’ with regard to the input datasets from which it was generated. However, the privacy of generated synthetic data is not necessarily easily clarified or ensured for all applications.

[0272] The synthetic data that is output from Genomator, although somewhat random and disparate from the input datasets from which it was derived, may not be absolutely private, in the sense that information about input genomic dataset is not revealed by a witness of the synthetic data output. As genomic information is often regarded as being sensitive data, it is there is motivation to consider concepts of privacy and the degree of privacy afforded byGenomator. Then, consequently, there are measures which can be taken to evaluate and enhance the privacy of the synthetic data that is output by Genomator.Randomisation strategies

[0273] There are several different methods for adding randomisation into the generation of synthetic data by Genomator. In one method, the SAT solver (e.g. logic solving engine) of Genomator may be configured to have inherent randomness. In another method, randomisation and noise may be added to the output from Genomator. In another method, randomisation may be added to the input into Genomator. In another method, randomisation may be added into the constraints which Genomator produces for its SAT solver. It is noteworthy, that these methods can be combined in various ways to complement their effects.Randomness from the SAT solver

[0274] Logical constraints defined in terms of constraint variables are input into a logic solving engine, such as a SAT solver, to generate solutions to those constraint variables so that the logical constraints are satisfied. However different solutions from the SAT solver may be possible, and insofar as there are different solutions which a SAT solver could give, different SAT solver software may have different biases between the likelihoods of giving these possible solutions. Different SAT solvers may have different biases, and these biases can influence the likely output from Genomator.

[0275] In principle, if the SAT solver software used to generate the synthetic data was known to a hypothetical attacker, then the resulting biases evident in the output from Genomator could be used to infer information about the inputs into Genomator. Accordingly, these biases could make Genomator more privacy violating for the input datasets.

[0276] While there are many different logic solving engines available, logic solving engines may be designed to be more uniform, and unbiased, in the solutions that they generate. These more-uniform-like logic solving engines produce solutions within the provided constraints more uniformly, and in so doing raise the randomness of the output.

[0277] The more random the output from the logic solving engine, the less the output is likely to be indicative of the specific constraints input into the logic solving engine, and the less can be inferred about those constraints, and thus to the input which generated them. Thus, the more uniform the output from the logic solving engine, the more likely a specific output is plausiblycompatible with a larger constellation of possible constraints from possible inputs, and potentially the more private Genomator’s synthetic output data can be considered to be.Decreasing constrainedness

[0278] An additional means of making the output of Genomator more varied, is to decrease the constrainedness of the constraints, thereby increasing the span of possible solutions that the logic solving engine could yield for each any actual input, and therefore the likely overlap between outputs that those inputs could yield.

[0279] A method of decreasing constrainedness is to increase the number of genomes input into Genomator, particularly this has the effect of making the generation of the ‘indistinguishability’ clause-2 constraints more rare. The instantiation of an ‘indistinguishability’ constraint depends on pairings of logical signatures, and if there is no ‘FF’ between the pairs of the possible pairings, then the size-2 clause is added to the SAT solver. Thus, the more (and more varied) the genomes that are input into Genomator, the more likely that there will be an ‘FF’ in such a pairing, and thus the 2-clause constraint will not be added, making the SAT problem less constrained.

[0280] In this way, the choice of SAT solver and the number and variability of the genomes input into Genomator can be controlled to make the output from Genomator more (or less) potentially private.Adding randomisation to output

[0281] In some embodiments, the output from Genomator can have randomisation (or other processing) added to it, to make the synthetic data more private, such that the determination of which input datasets were used by the logic solving engine to generate the synthetic data is more difficult, or less feasible. Adding a very large amount of noise to the output from Genomator or doing processing to reduce the number features present in Genomator’s output, may have the effect of reducing the information that the output presents to a potential hypothetical attacker.

[0282] Classical techniques of producing private data, such as differentially private data, proceed by adding sufficient noise to the output of a mechanism (this is the basis of the Laplace and Gaussian mechanisms for achieving differential privacy). These techniques achieve differential privacy by adding a sufficient amount of noise to overcome the discernible output difference that a single person’s data in the input could produce. These techniques tend have difficulty with higher dimensional data, as copious amounts of noise often needs to be addedwhen the output data, to which the noise is added, is sufficiently high dimensional, as there is an increasing number of ways that an individual’s data could affect the output.

[0283] In light of these difficulties, a technique may be applied that determines the character of the noise needed to be added to the output of the mechanism by empirical observations of such an output under various conditions. The runtime efficiency of Genomator may generate sufficient empirical data to play well with such techniques.

[0284] There are different potential ways of adding randomisation to the output from Genomator. Particularly other techniques for achieving privacy for tabular data, naturally can be applied to the aligned data that Genomator produces. A data sanitization approach may be applied, which deletes specific and / or sensitive cells in tabular data, either to delete or selectively share single nucleotide polymorphisms (SNP) information. These approaches to anonymise genomic data are compatible with being appended to Genomator’ s output.

[0285] Tabular data methods can be applied, including identifying cellular data that can be deleted; generalised; selectively shuffled; and / or deleted and then imputed.

[0286] These methods of randomising the output of Genomator can be naturally and directly be appended to Genomator with-or-without other modifications. Additionally, these methods of adding privacy to specific data can also be applied to the input into Genomator.Adding randomisation to the input

[0287] A means of adding randomisation to Genomator is to randomise the input datasets. Particularly, it is possible to add a degree of randomness to input datasets by adding noise, deleting, shuffling, etc. Insofar as these techniques provide privacy, then that privacy propagates through Genomator’ s algorithm as well, to create a private output.

[0288] However, the input into Genomator can facilitate randomisation which might be less effective on the output, such as the appending of individual genome data which is completely randomly generated. This appending of purely random individual data might well be obvious in the context of the output, but appending random data the input adds an extra degree of randomisation through the Genomator algorithm itself. As it is already considered, increasing the number of genomes input into Genomator serves to loosen the SAT constraints that Genomator uses, and so appending a random input dataset in the set of input datasets that are input into Genomator has the net effect of randomly loosening the SAT constraints which Genomator instantiates, and this effect can be compounded with multiple randomly generated genomes appended.

[0289] Rather than purely random genome data being appended, it is also possible to consider other kinds of genome data being appended to the input into Genomator. For instance, other genome data generation techniques (or Genomator itself) could be used to generate synthetic data which could be appended to the input into Genomator itself.

[0290] Additionally, doing multiple rounds of feeding synthetic data into Genomator, is naturally expected to compound the processing required to conduct specific inferences back to particular input data. In this way, there is the potential of providing a degree of privacy through assumptions about an attacker’s computational capacity, in addition to the privacy provided through randomisation.

[0291] Randomising the input and output from Genomator is two primary sites of potential randomisation which can serve to add privacy to Genomator’ s algorithm, however randomisation may be considered inside the context of Genomator’ s algorithm too.Adding randomisation to the constraints

[0292] In some embodiments, in the process of generating logical constraints in Genomator, there is a pairwise consideration of all logical signatures. In these pairwise considerations there is condition which serves to indicate whether a corresponding constraint will be added to the SAT solver, and this condition is subject to potential alteration and randomisation. In Genomator’s algorithm, the indistinguishability constraint considers whether there is a ‘FF’ pair (reference numerals A02 and A04), and if there is not, then the respective logical constraint is added. The generation of indistinguishability constraints is described in relation to Figure 7.

[0293] Figure 22 illustrates the addition of randomisation (e.g., the application of a random element) to a logical constraint, in accordance with an embodiment. In particular, Figure 22 illustrates the application of a function , which Genomator uses to determine a 2-clause indistinguishability constraint, to two logical signatures 2210 (assigned constraint variable xi) and 2220 (assigned constraint variable X2). In the situation where there is no ‘FF’ across the pair of logical signatures 2210 and 2220, the function (. ) returns boolean value ’TRUE’ 2230. The Boolean value ‘TRUE’ is associated with the logical constraint xi OR X2, indicated by reference numeral 2230. In the situation where there is at least one ‘FF’ (e.g. 2202 or 2204), the function returns a Boolean value of ‘FALSE’ .

[0294] This process of generating a logical constraint can be randomised, particularly having the result of randomly strengthening the likelihood that constraints are added. These processes may make the resulting constraints unsatisfiable. Accordingly, it may be beneficial to inputsufficient numbers of input datasets into Genomator to loosen the constraints to the determination of a solution feasible for the logic solving engine.Randomised thresholding

[0295] In embodiments described above, an indistinguishability constraint is added if there is no pair of ‘FF’ logical signs across a pair of logical signatures, however this threshold can be modified, and randomly so. A constraint could be added if there is less than N ‘FF’s across a pair of logical signatures and that this threshold number N can be randomly generated and applied with each pair of signatures. This consideration is illustrated in Figure 23 which illustrates the case for N=1 where the constraint is added if there is 0 or 1 ‘FF’s and not added if there is 2,3,4, etc ‘FF’s, in this situation the number N may be generated with each pair considered and the number of ‘FF’ pairs is calculated. If the number of ‘FF’ s is less than N then the constraint is added, otherwise not.

[0296] This alteration (particularly with larger N values) makes the constraints that are input into the SAT solver to be more constraining, as it is more likely that the respective 2-clause will then be added - but randomly so. Notably, and in-practice, this randomisation of the constraints makes it less likely that any specific path of inference is possible from the output from Genomator back to specific qualities of the input.Conjunction of parts

[0297] The concept of random strengthening of constraints can be implemented in other ways as well. Genomator can handle additional SAT constraints, and conceptually it is possible to conduct multiple runs of Genomator to generate different sets of SAT constraints, and to simply collect all the constraints and solve them together, to determine a synthetic output that is compatible with those multiple Genomator runs. In this way, the resulting SAT solutions from the collection of these sets of constraints would logically be within the intersection of the possible SAT solutions from each.

[0298] Particularly, if these multiple Genomator runs were on entirely disjoint input datasets, then the output of this conjoining constraints would be compatible with output of a single Genomator run in which any specific individual input datasets was not present. This process reflects the idea of k-deniability, particularly that it should be possible for any individual to deny (with some plausibility) that their data was a part of the dataset that gave rise to the witnessed output. In this context, a valid means of adding privacy by means of randomising theconstraints, is by randomisation over which-and-how-many input datasets are considered with their resulting constraints conjoined.

[0299] In practice, an efficient means of achieving this outcome is to pass all datasets into Genomator at once, and consider a partitioning within the signatures that Genomator computes.

[0300] Figure 24 illustrates the generation of a logical constraint based on a consideration of a partition in a logical signature, in accordance with an embodiment. As illustrated in Figure 24, a partition in the signatures compared may be considered, and a function determining whether the respective 2-clause is added if a required condition holds for either partition.

[0301] Figure 24 illustrates a single partition (indicated by line 2430) in the logical signature 2410 that partitions the signature into two parts. Similarly, logical signature 2420 is partitioned by line 2440. In the context illustrated, the function considers each of the parts of the compared logical signatures 2410 and 2420 and if there is no ‘FF’ in either part (as separated by partitions 2430 and 2240, respectively), the function returns ‘TRUE’ and the respective 2-clause added. Conversely, if there is an ‘FF’ pair in both part then the respective 2-clauses are not added.

[0302] This dynamic of partitioning the signatures and adding the constraint if a required condition holds for any parts, has the effect of adding constraints insofar as they hold for any part of the partition. In this way, the constraints generated is the intersection of those that would be added if only any one part of the partition were considered, thus the output from the SAT solver consistent with what would be output from any partition of inputs alone.

[0303] It is notable that if the ordering of the inputs is randomised (and therefore the partitioning into multiple parts is randomised) that there can be a large number of ways that this separation can occur. Additionally, if the number of partitions and their size, is also randomised, this potentially introduces a great degree of randomisation into Genomator’ s process.Combinatorial intersection of Genomator constraints

[0304] It is possible to construct, within Genomator, a configuration where the output from Genomator would be analogous to the intersection of possible outputs from other possible Genomator runs. This ability corresponds to an intersection of solution spaces, which is naturally more constrained than any of those spaces individually. Taking this approach to a much larger extreme, it is possible to consider the intersection of multiple combinations of intersections.

[0305] Figure 25 illustrates a process where the function to determine the addition of a respective 2-clause considers a range of pairs in the logical signatures compared, in accordance with an embodiment. Each of the sequential logical pairs of Boolean values (shown in boxes in Figure 25) in logical signatures input into the function are OR’ed together. After this OR’ing process the reduced vectors are produced and compared, and thereafter if there is no ‘FF’ in this resulting vector then the output is TRUE, and the respective 2-clause added.

[0306] Notably, this process of taking pairwise OR’ings of Boolean values prior to the normal Genomator operation of considering if there is an ‘FF’ pair, makes ‘T’ values much more likely, and thus much more likely that the corresponding 2-clauses will be added within Genomator - thus making Genomator’ s SAT solver much more constrained.

[0307] It can be considered that this operation could be considered as if it were effectively the same as a very large, repeated partitioning of the genomic information as similar to that described in relation to Figure 24.

[0308] Figure 26 illustrates an example in which a pairwise OR-ing schema is equivalent to an expanded multipart repeated partitioning reflective, in accordance with an embodiment. In particular, Figure 26 illustrates that the ORing process, even just for the first two elements (shown in faint dotted boxing) can be expanded as a repeated partitioning. (In this figure, variables xi ... xs and yi ... y 5 stand for arbitrary values of True / False).

[0309] If there is an ‘FF’ after ORing process, then there must be an ‘FF’ across in all of the partitions which it could equivalently be divided into, and this is illustrated in Figure 27.

[0310] Figure 27 illustrates an application of Figure 26, in accordance with an embodiment. In particular, Figure 27 illustrates an application of Figure 26 in which xi, yi, X2, and y? = F, in this case because xi OR X2 is False, and yi OR y? is False, there is necessarily an ‘FF’ in every equivalent parition, and the respective 2-clause constraint is not added.

[0311] If there is not an ‘FF’ after the ORing process, then it does not necessarily hold that there isn’t an ‘FF’ among the partitions, and this is illustrated in Figure 28. Figure 28 illustrates another application of Figure 26, in accordance with an embodiment. In particular, Figure 28 illustrates an application of Figure 26 in which xi, yi and y = F and x = T, in this case because xi OR X2 is True, and yi OR y? is False, there is not necessarily an ‘FF’ in every equivalent parition, and the respective 2-clause constraint is potentially added.

[0312] In this way, it can be considered, that this ORing process (especially over all sequential pairs - such as per Figure 25) stands effectively equivalent to the intersection to acombinatoric ally large number permutation partitions and therefore is a very strong constriction of constraints into the SAT solver within Genomator.

[0313] Conversely, there is a very large number of different ways in which a dataset can be grouped into pairs to be OR’ed together. If this pairing process is randomised, then that is potentially great degree of randomisation of constraints within Genomator. Additionally, for even stronger constraints, instead of pairs we could also consider triples to be ORed together, or some mixture of singles and / or doubles and / or triples - and this would further enhance the degree of constraint randomisation within Genomator’ s process.Diversity and Reverse Genomator

[0314] Provided herein are various methods via which randomisation can be added to Genomator. These different methods can be mixed and matched and extended in a combination of different ways. However, to evaluate the actual effective privacy gained by their application, it can be desirable to evaluate with an implementation of a privacy engine 1402. An example implementation of a privacy engine is described herein as ‘Reverse Genomator’.

[0315] Particularly, just adding randomisation may not necessarily result in actually more private data, as the complexity of these various interactions and influences is not simple to model, and therefore the degree to which they could facilitate a hypothetical attacker making inferences about the input data may not be obvious.

[0316] Reverse Genomator informs the user what combinations of a set of genomic input could have yielded the witnessed output from Genomator, and particularly if all combinations of input feature a specific genomic input, then that input’ s presence is almost certainly logically assured. However, if Genomator’ s algorithm is modified, then for accurate inferences, Reverse Genomator should also be modified in response. For instance, in the situation in which Genomator is configured to add privacy by randomised thresholding, this technique may be implemented as a programmatic option within Genomator software, and also a similar modification to handle this is an option within Reverse Genomator’ s software.

[0317] There are a wide range of different ways in which Genomator can be randomised, and these ways can be combined as needed and their combined effect on the privacy of the output data can be assessed by such tools as Reverse Genomator. These randomisations can be applied to the input, the output, or in the constraint construction process of Genomator itself. These randomisations have their own effects (which may (or may not be) desirable in their own right) and can loosen or tighten the constraints input into the SAT solver as part of Genomator’ salgorithm. Care must be taken when using Genomator to select appropriate modifications to provide the required utility and privacy for an application. It is noted that the Reverse Genomator may be implemented with any suitable type of logic solving engine, not just SAT solvers.Experimental results

[0318] Methods and systems provided herein may be contrasted with other advanced techniques that use machine learning and deep learning methods to generate synthetic data.

[0319] Experimental results of an embodiment of the synthetic data generator (referred to as ‘Genomator’) which is configured to generate synthetic genomic data, show that the synthetic data generator can generate high-quality synthetic genomes, with little privacy loss and little computational burden. The embodiment is capable of retaining single variant frequencies as well as linkage-disequilibrium between long-range variants. At the same time, it retains the inter- and intra- population structure while avoiding privacy leakage from the training data. On top of that, the embodiment is able to perform all the computations using a single CPU with limited RAM while the comparison methods require much more compute time and graphical processing units (GPUs).Runtime performance

[0320] Figure 12 is a graph 1200 illustrating the runtime 1202 for an embodiment of the synthetic data generator 102 configured to generate synthetic genomic data, compared to runtimes of alternative methods of generating synthetic genomic data. In particular, graph 1200 illustrates the runtime of the four methods, including training or processing the real data, on increasing Variant Call Format (VCF) data file sizes up to the full 11 million SNPs of the full human genome (excluding sex chromosomes) from the 1000 Genome Project. These VCF files were of 400 human individuals, with increasing genome segments across chromosomes 1-22 with MAF > 0.01 filtering.

[0321] The x-axis of graph 1200 comprises logarithmic scale representing the number of single nucleotide polymorphisms (SNPs) generated in the synthetic genomic data. The y-axis of graph 1200 comprises a logarithmic scale representing the computational runtime to generate one synthetic genome, in minutes. Plot 1202 represents the runtime data for an embodiment (entitled Genomator) of the synthetic data generator 102. Plot 1204 represents the runtime data for a Generative Adversarial Network (GAN) implementation which is configured to generate synthetic genomic data. Plot 1206 represents the runtime data for a Restricted BoltzmannMachine (RBM) implementation which is configured to generate synthetic genomic data. Plot 1208 represents the runtime data for a Markov Chain Generation (MARK) implementation which is configured to generate synthetic genomic data. All methods were run with 80 GB of ram, with GPU Nvidia Tesla P100-SXM2-16GB graphics card and two days of compute time.

[0322] Points 1220 on the plots for the GAN implementation, the RBM implementation and the MARK implementation indicate points at which the respective implementation failed to completed processing. For example, the RBM implementation and the GAN implementation ran out of graphics card VRAM above 102k SNPs, and Markov method ran out of RAM above 1638k SNPs. Genomator was the only method that was able to scale to the largest file size, producing synthetic sets of all human chromosomes 1-22. Genomator took 8 hours to process.

[0323] Advantageously, the runtimes for the Genomator implementation are consistently and significantly lower than the runtimes for the alternative implementations, in many cases by an order of magnitude.Accuracy performance

[0324] An embodiment of the synthetic data generator 102, configured to generate synthetic genomic data, was compared to three alternative implementations for generating synthetic genomic data to compare the ability of the four implementations to produce synthetic data that faithfully captures the higher-order complexities of genomic data. A principal component analysis (PCA) was run, for each of the four implementations, on 805 chosen SNPs of the 1000 Genomes Project to visualise the well-known “V” shape of the underlying population structure.

[0325] The PCAs of the synthetic data generated by the four implementations are illustrated in Figures 29 to 32. Figure 29 illustrates a PCA graph of the Genomator, in accordance with an embodiment. Figure 30 illustrates a PCA graph of a Markov Chain Generation (MARK) implementation which is configured to generate synthetic genomic data, in accordance with an embodiment. Figure 31 illustrates a PCA graph of a Restricted Boltzmann Machine (RBM) implementation which is configured to generate synthetic genomic data, in accordance with an embodiment. Figure 32 illustrates a PCA graph of a Generative Adversarial Network (GAN) implementation which is configured to generate synthetic genomic data, in accordance with an embodiment. In each of Figures 29 to 32, the PCA of the generated synthetic genome data is shown in Blue, against the two greatest principle components of the underlying 805 SNP dataset shown in Red.

[0326] As shown in Figures 29 and 30, the Genomator and Markov Chain implementations reproduce the population structure accurately (sliced Wasserstein Error of 3%). In contrast, both the RBM implementation and the GAN implementation produce synthetic data that are shifted from the real data, with a sliced Wasserstein error of 4% and 22% respectively. This means that the synthetic data produced by the RBM implementation and the GAN implementation contains genotypes at a different frequency than they occur in the real dataset.

[0327] In addition, the high error of the GAN implementation indicates that the network did not fully converge, likely suffering from a partial mode-collapse where one “arm” of the PCA V-shape is not reproduced.

[0328] The alternative implementations’ abilities to replicate local interactions were also examined, as these are of particular importance to medical and research applications. The linkage disequilibrium (LD) structure in 1000 synthetically generated genomes were compared against the observed structure across the largest four genes that all four methods can cope with (RBFOX1, FHIT, AGBE4, CCSER1). For AGBE4, Genomator captures both short distance ED (central boxes) as well as long distance LD (horizontal and vertical lines). In contrast, Markov Chains only reproduce short distance LD, while RBM and GAN capture both short- and long-range interactions but with lower accuracy.

[0329] As illustrated in Figures 13 and 33 to 36, the ability of each implementation to replicate local interactions was quantified for genes RBFOX1, FHIT, AGBL4 and CCSER1 by calculating the distance between the real and reproduced LD over increasing distances. In particular, Figure 13 is a graph 1300 illustrating square error in linkage disequilibrium reproduction between loci in original dataset against that reflected in the synthetic data for the AGBL4 gene, in accordance with an embodiment. The x-axis of graph 1300 represents the distance between loci (kb), and the -axis of graph 1300 represents the average square error.

[0330] Plot 1302 represents the average square error per various distances between loci for an embodiment (entitled Genomator) of the synthetic data generator 102. Plot 1304 represents the average square error per various distances between loci for a Generative Adversarial Network (GAN) implementation which is configured to generate synthetic genomic data. Plot 1306 represents the average square error per various distances between loci for a Restricted Boltzmann Machine (RBM) implementation which is configured to generate synthetic genomic data. Plot 1308 represents the average square error per various distances between loci for a Markov Chain Generation (MARK) implementation which is configured to generate synthetic genomic data.

[0331] Figure 33 is a scatter plot illustrating square error in linkage disequilibrium reproduction between loci in original dataset against that reflected in the synthetic data for the AGBL4 gene, in accordance with an embodiment. Figure 33 is a scatter plot of Figure 13. Figure 34 is a scatter plot illustrating square error in linkage disequilibrium reproduction between loci in original dataset against that reflected in the synthetic data for the FHIT gene, in accordance with an embodiment. Figure 35 is a scatter plot illustrating square error in linkage disequilibrium reproduction between loci in original dataset against that reflected in the synthetic data for the CCSER1 gene, in accordance with an embodiment. Figure 36 is a scatter plot illustrating square error in linkage disequilibrium reproduction between loci in original dataset against that reflected in the synthetic data for the RBFOX1 gene, in accordance with an embodiment.

[0332] For each of Figures 33 to 36, data points shown in blue 3302 correspond to the Genomator implementation, data points shown in red 3304 correspond to the GAN implementation, data points shown in pink 3306 correspond to the RBM implementation and data points shown in green 3308 correspond to the MARKOV implementation.

[0333] Advantageously, the average square error for the Genomator implementation is consistently and significantly lower than the average square error for the alternative implementations. In particular, for the Genomator has an average square error of 0.0003 over all genes compared to Markov Chain of 0.0011, RBM of 0.0021 and GAN of 0.0038.Privacy performance

[0334] Alongside accuracy and efficiency, a key evaluation criterion is the ability of a synthetic data generator to produce synthetic data that is similar but not identical the input datasets, and / or similar but not identical to other synthetic data generated by the synthetic data generator. This ability comprises a potential indication of privacy performance.

[0335] In the context of generating synthetic genomic data, privacy performance may be determined by simulating an attribute inference attack, where an attacker knows a subset of a target’s genome and uses the nearest neighbour in the synthetic cohort to fill in the rest.

[0336] To experimentally quantify this vulnerability, the accuracy of such an attacker on the datasets generated by the four implementations of a synthetic data generator for the AGBL4 gene can be calculated. As different datasets have varying population compositions, it is desirable to normalize attack- success by the baseline in genetic variability in that dataset. To do this, the 1000 genomes cohort may be split into two similar subsets and the attackeffectiveness may be calculated on synthetic data generated from the subset containing the target individual ("in-data”) and compare it to the cohort without.

[0337] Figure 37 comprises a table 3700 of results of an SNP inference experiment for different implementations of a synthetic data generator, in accordance with an embodiment. Scores represent the likelihood of incorrect SNP inference.

[0338] Table 3700 reports the average performance over 1000 replicates. The Markov Chains implementation produces synthetic data that represents the real data the closest (in-data distance = 0.05) compared to Genomator (0.09), RBM (0.13) and GAN (0.34). However, the Markov Chains implementation has a much larger out-data distance (0.30), indicating large privacy violation as the synthetic data closely resembles the data it was generated from.

[0339] Conversely, Genomator has the smallest Out-distance (0.27) indicating that Genomator has better learned the general trend of the data without representing any specific individual. While this produces data that is marginally less accurate (0.09 vs 0.05) it reduces the chance of revealing content specific to an individual. This indicates that there may be a natural trade-off between re -producing the data perfectly and preserving privacy.

[0340] The accuracy-privacy relationship may be further considered by quantifying how often ‘private’ SNP combinations leak into the synthetic data output. These are sets of SNPs that are only seen in one individual in the dataset and hence replicating them might serve to expose uniquely identifiable features of that individual and facilitate identification. This may be compared to the rate at which ‘fictions’ SNP combinations (that are not seen in any individual) are created to quantify the balance between accuracy and privacy.

[0341] Figure 38 is a scatter plot illustrating the average number of times that a ‘private’ combination of SNPs appeared in the output dataset was computed, as well as the number of times ‘fictions’ SNP combinations were generated, in accordance with an embodiment.

[0342] In particular, Figure 38 illustrates the results of an experiment in which 1000 synthetic versions of the AGBL4 gene were generated using the 4 implementations of a synthetic data generator (Genomator, GAN, RBM and Markov). Each plot point illustrates the average number of times that a ‘private’ combination of SNPs appeared in the output dataset was computed, as well as the number of times ‘fictions’ SNP combinations were generated. Plot points 3802 represents the results produced by the Genomator implementation. A fictitious quadruplet is one that is not featured in the input dataset (hence reflective of noise in themechanism) and a private quadruplet is one that is featured in exactly one individual in the input dataset. Each of the mechanisms were run with a series of parameters.

[0343] Plot points 3804 represents the results produced by the GAN implementation. Plot points 3806 represents the results produced by the RBM implementation. Plot points 3808 represents the results produced by the Markov implementation.

[0344] The ideal implementation produces results in the bottom left corner by having fewer privacy-revealing combinations, while also not making up unseen combinations that are potentially not viable in humans.

[0345] The graph of Figure 38 illustrates that Genomator is the only method operating in the ideal quadrant. While GANs also refrain from exposing private combinations, they create fictional SNP combination at a higher rate than Genomator. In contrast, synthetic data created by Markov Chains and RBMs use private as well as fictional SNPs at a high rate.

[0346] It is noted that the tools perform remarkable consistent, despite different parameters explored, i.e. for Genomator (cluster sizes of 10,20,30,40,50), Markov Chain (window sizes of 20,40,60,80,100,120,140,160,180,200), RBM (hidden layer sizes of 300,500,700 with learning rates of 0.005 and 0.01 to 1000 epochs), and GANs (trained to 300 epochs with densely packed generate / discriminator neural layers of 500x500;300xl00, 600x600;400x200,700x700; 500x300)..Customisation for accuracy -privacy trade-off

[0347] For implementations of a synthetic data generator, there is a trade-off being achieving privacy and achieving accuracy. Advantageously, implementations of synthetic data generator described herein (e.g. Genomator) are able to customize this balance to the application at hand. A means of measuring the privacy provided is with a Privacy Engine (eg. Reverse Genomator) where embodiments of the Reverse Genomator identify, for any synthetic genome, the space of all possible subsets of input data that could have been used to generate it, given the full information about Genomator (with its parameters) and the dataset from which Genomator’ s input is selected from. Individuals who appear in all these subsets are deduced to have been used as input and are at risk of privacy exposure.

[0348] Allowing to tailor privacy, Randomized Thresholding (see clause 365) was used an the parameter Z is introduced. Specifically, the parameter Z reflects the randomisation in the threshold in the ‘indistinguishability’ constraints N.

[0349] To test this approach experimentally, 400 SNPs from 400 samples of the AGBL4 gene were randomly selected, synthetic data was created with Genomator and then Reverse Genomator was used to reconstruct 1000 combinations of possible input datasets. In this analysis, ff an individual is part of all reconstructions, it is defined as privacy exposed.

[0350] Figure 39 is a graph illustrating the trade-off between privacy and accuracy for Genomator, in accordance with an embodiment. In particular, Figure 39 illustrates the experimental likelihood that a synthetic output from Genomator could be used by Reverse Genomator to logically identify an individual in the input dataset, across Z parameter values for Genomator. Figure 39 also shows the Wasserstein distance between synthetic data generated by Genomator for those parameters and the real dataset. From Figure 39 it can be seen that increasing Z increases the privacy, i.e. lowers the risk of being identified. The sliced- Wasserstein distance between the real and synthetic dataset is overlaid, visualizing the tradeoff between accuracy and privacy.

[0351] It will be appreciated by persons skilled in the art that numerous variations and / or modifications may be made to the above-described embodiments, without departing from the broad general scope of the present disclosure. Furthermore, it will be appreciated by persons skilled in the art that embodiments disclosed herein can be combined with one or more other embodiment disclosed herein, without departing from the broad general scope of the present disclosure. The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive.

[0352] It will be appreciated by persons skilled in the art that any suitable distribution of functionality between different functional units may be used without detracting from the invention. For example, functionality illustrated to be performed by separate computing devices may be performed by the same computing device. Likewise, functionality illustrated to be performed by a single computing device may be distributed amongst several computing devices. Hence, references to specific functional units are only to be seen as references to suitable means for providing the described functionality, rather than indicative of a strict logical or physical structure or organization.

[0353] It will be appreciated by persons skilled in the art that, for processes and methods disclosed herein, the operations performed in the processes and methods may be implemented in differing order. Furthermore, the outlined steps and operations are only provided as examples, and some of the steps and operations can be optional, combined into fewer steps andoperations, or expanded into additional steps and operations without detracting from the essence of the disclosed embodiments.

[0354] References herein to software or executable instructions are to be understood as referring to executable instructions stored in volatile or non-volatile memory. The memory can include any data storage device that can store data which can thereafter be read by a processor. Examples of memory include read-only memory (ROM), random-access memory (RAM), magnetic tape, optical data storage device, flash storage devices, or any other suitable storage devices.

[0355] Throughout this specification the word ‘comprise’, or variations such as ‘comprises’ or ‘comprising’, will be understood to imply the inclusion of a stated element, integer or step, or group of elements, integers or steps, but not the exclusion of any other element, integer or step, or group of elements, integers or steps.

[0356] As used herein, any reference to “one embodiment” or “an embodiment” means that a particular element, feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. The appearances of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment. Similarly, use of “a” or “an” preceding an element or component is done merely for convenience. This description should be understood to mean that one or more of the element or component is present unless it is obvious that it is meant otherwise.

[0357] Unless expressly stated to the contrary, use of the terms “random” and “randomisation” and “stochasticity” includes “pseudo-random” and “pseudo-randomisation”, respectively. Unless expressly stated to the contrary, uses of the terms ‘all pairs’ refers inclusively to pairs of distinct as well as non-distinct (i.e. repeated) elements.

[0358] Unless expressly stated to the contrary, “or” refers to an inclusive or and not to an exclusive or. For example, a condition A or B is satisfied by any one of the following: A is true (or present) and B is false (or not present), A is false (or not present) and B is true (or present), and both A and B are true (or present).

[0359] Unless expressly stated to the contrary, the description of an entity (e.g. any entity described herein, including a dataset or a logical signature) as a first entity or a second entity is used to distinguish one entity from another entity and does not constitute an implied or explicit ordering of the referenced entities.

[0360] This section describes additional aspects and features of synthetic data generation and privacy evaluation, presented without limitation as a series of paragraphs, some or all of which may be alphanumerically designated for clarity and efficiency. Each of these paragraphs can be combined with one or more other paragraphs, and / or with disclosure from elsewhere in this application, in any suitable manner. Some of the paragraphs below expressly refer to and further limit other paragraphs, providing without limitation examples of some suitable combinations.AO. A method of inferring a membership of a target input dataset in a set of input datasets, the set of input datasets used to generate a synthetic dataset, the method comprising: determining a set of candidate input datasets, the candidate input datasets comprising at least one of the input datasets of the set of input datasets; determining, based on the synthetic dataset and the set of candidate input datasets, at least one logical constraint; determining a set of potential input datasets by applying the at least one logical constraint to a logic solving engine; and determining, based on the set of potential input datasets, whether the target input dataset is a member of the set of input datasets.Al. The method of AO, wherein the at least one logical constraint is associated with a first candidate input dataset of the set of candidate input datasets, and wherein the at least one logical constraint indicates whether the first candidate input dataset is a member of the set of input datasets.A2. The method of AO, wherein determining whether the target input dataset is a member of the set of input datasets comprises determining an indication of probably of a target input dataset being a member of the set of input datasets.BO. A method of evaluating the privacy of a synthetic dataset generated from a set of input datasets, the method comprising: determining, based on the synthetic dataset and the set of input datasets, at least one logical constraint; determining one or more sets of potential input datasets by applying the at least one logical constraint to a logic solving engine; anddetermining, based on the one or more sets of potential input datasets, a level of privacy of the synthetic dataset.B 1. The method of BO, wherein the level of privacy of the synthetic dataset comprises an indication of probably of a target input dataset being a member of the set of input datasets.B2. The method of AO or BO, wherein the logic solving engine comprises a satisfiability solver.B3. The method of AO or BO, wherein each set of potential input datasets of the one or more sets of potential input datasets comprises a different combination of input datasets from the set of input datasets.B4. The method of AO or BO, wherein the at least one logical constraint comprises an indistinguishability constraint.B5. The method of AO or BO, wherein the at least one logical constraint comprises an indication of a number of input datasets in the set of input datasets.B6. The method of AO or BO, wherein the at least one logical constraint comprises a logical constraint applied by a logic solving engine for the generation of the synthetic dataset.B7. The method of BO, further comprising determining a level of privacy associated with a target input dataset by determining whether the one or more sets of potential input datasets comprises the target input dataset.B8. The method of BO, further comprising determining a level of privacy associated with a target input dataset by determining how many of the one or more sets of potential input datasets comprises the target input dataset.CO. A system for evaluating the privacy of a synthetic dataset generated from a set of input datasets, the system comprising: one or more processors; and memory comprising computer executable instructions, which when executed by the one or more processors, cause the system to perform a method described herein.DO. A system for determining a synthetic dataset from a set of input datasets, each input dataset comprising a plurality of data items, the system comprising one or more processors, individually or in combination, configured to:determine, based on the synthetic dataset and the set of input datasets, at least one logical constraint; determine one or more sets of potential input datasets by applying the at least one logical constraint to a logic solving engine; and determine, based on the one or more sets of potential input datasets, a level of privacy of the synthetic dataset.

Claims

CLAIMS:

1. A computer-implemented method for generating a synthetic dataset (320) from a set of input datasets (302), each input dataset comprising a plurality of data items, the method comprising: applying a plurality of logical expressions (520) to at least one data item in each input dataset to derive a plurality of logical signatures (530) that collectively characterise the value of each of the at least one data item in each input dataset; comparing the logical signatures (530) to derive at least one logical constraint (760) from the logical signatures; and determining, by applying the at least one logical constraint to a logic solving engine (140), the synthetic dataset (920), wherein the synthetic dataset (920) satisfies the at least one logical constraint (760).

2. The method of claim 1, wherein the at least one logical constraint comprises a two- clause constraint.

3. The method of any one of claims 1 to 2, wherein deriving the at least one logical constraint comprises: associating a first constraint variable with a first logical signature of the plurality of logical signatures; associating a second constraint variable with a second logical signature of the plurality of logical signatures; and defining the logical constraint as a Boolean expression comprising the first constraint variable, the second constraint variable and a Boolean operator.

4. The method of any one of claims 1 to 3, wherein each data item comprises a value of a set of discrete values.

5. The method of claim 4, wherein the plurality of logical signatures for each data item comprises a first logical signature that indicates whether the data item positioned at a first position is equal to a specified value of the set of specified values.

6. The method of claim 4, wherein the plurality of logical signatures comprises: for each position, a set of logical signatures comprising a logical signature for each specified value of the set of specified values.

7. The method of any one of claims 4 to 6, wherein each logical signature comprises an array of Boolean values, each Boolean value indicating whether a data item positioned in a specific array position of a respective dataset equals a specified value of the set of specified values.

8. The method of any one of claims 4 to 7, wherein applying a plurality of logical expressions to the data items in each dataset to derive a first logical signature of the plurality of logical signatures comprises: for each data item of the dataset: for each specified value of the set of specified values: in response to the data item, positioned at the first position in the dataset, equaling the specified value, set a corresponding Boolean value of the logical signature to represent true; and in response to the data item, positioned at the first position in the dataset, not equaling the specified value, set a corresponding Boolean value of the logical signature to represent false.

9. The method of claim 8, wherein applying a plurality of logical expressions to the data items in each dataset to derive a second logical signature of the plurality of logical signatures comprises: for each data item of the dataset: for each specified value of the set of specified values: in response to the data item, positioned at the first position in the dataset, equaling the specified value, set a corresponding Boolean value of the logical signature to represent false; and in response to the data item, positioned at the first position in the dataset, not equaling the specified value, set a corresponding Boolean value of the logical signature to represent true.

10. The method of any one of claims 1 to 9, further comprising: identifying duplicate logical signatures in the plurality of logical signatures; and determining a subset of logical signatures, wherein the subset of logical signatures does not include the duplicate logical signatures.

11. The method of any one of claims 1 to 10, wherein comparing logical signatures comprises: comparing a first logical signature of the plurality of logical signatures with a second logical signature of the plurality of logical signatures.

12. The method of claim 11, wherein deriving a logical constraint comprises, in response to comparing the first logical signature with the second logical signature: determining that a logical condition holds between logical signs of the first signature and logical signs of the second signature; and defining the logical constraint in accordance with the logical condition.

13. The method of claim 11, wherein the first logical signature and the second logical signature define a plurality of pairs of logical signs, and wherein comparing the first logical signature with the second logical signature comprises: for each pair of logical signs, determine whether both logical signs of the pair of logical signs are false.

14. The method of claim 13, further comprising, in response to determining, for each pair of logical signs, that both logical signs of the pair are not false, deriving a logical constraint comprising the Boolean OR operator applied to a constraint variable associated with the first logical signature and a constraint variable associated with the second logical signature.

15. The method of any one of claims 1 to 14, wherein comparing logical signatures comprises: determining logical conditions that hold across the dataset.

16. The method of any one of claims 1 to 15, wherein comparing logical signatures comprises: comparing each logical signature of the plurality of logical signatures with each other logical signature of the plurality of logical signatures.

17. The method of any one of claims 1 to 16, wherein the at least one logical constraint defines at least one logical condition that holds across the dataset.

18. The method of any one of claims 1 to 17, wherein determining the synthetic dataset comprises: obtaining, from the logical solving engine, a synthetic logical sign for each constraint variable of a set of constraint variables; and for each array position of the synthetic dataset: determining a synthetic value for the synthetic data item in the array position of the synthetic dataset.

19. The method of any one of claims 1 to 18, further comprising: determining a level of privacy of the synthetic dataset, wherein the level of privacy comprises a level of determinability of a membership of a first input dataset in the set of input datasets.

20. The method of claim 19, wherein determining a level of privacy comprises: determining, by applying a logic solving engine, a set of potential input datasets; and determining the membership of one or more input datasets of the set of input datasets in the set of potential input datasets.

21. The method of any one of claims 1 to 19, wherein the set of input datasets comprises a synthetic dataset that was previously generated by the logic solving engine.

22. The method of any one of claims 1 to 19, wherein deriving the at least one logical constraint from the logical signatures comprises applying a randomisation to the logical signatures.

23. The method of any one of claims 1 to 19, wherein the set of input datasets comprises a randomly generated input dataset.

24. A machine-readable storage medium, storing instructions which, when executed by one or more processors, cause the one or more processors to perform the method of any one of claims 1 to 23.

25. A system comprising: one or more processors; and memory comprising computer executable instructions, which when executed by the one or more processors, cause the system to perform the method of any one of claims 1 to 23.

26. A machine-readable storage medium storing a synthetic dataset determined from a set of input datasets (302), each input dataset comprising a plurality of data items, wherein the synthetic dataset is determined by a method comprising: applying a plurality of logical expressions to the data items in each dataset to derive a plurality of logical signatures characterising the value of each data item; comparing the logical signatures to determine at least one logical constraint based on the logical signatures; and determining the synthetic dataset by applying the at least one logical constraint to a logical solving engine, wherein the synthetic dataset satisfies each of the at least one logical constraints.

27. A system for determining a synthetic dataset from a set of input datasets, each input dataset comprising a plurality of data items, the system comprising one or more processors, individually or in combination, configured to: apply a plurality of logical expressions to the data items in each dataset to derive a plurality of logical signatures characterising the value of each data item; compare the logical signatures to determine at least one logical constraint based on the logical signatures; and determine the synthetic dataset by applying the at least one logical constraint to a logical solving engine, wherein the synthetic dataset satisfies each of the at least one logical constraints.

28. A system for determining a synthetic dataset from a set of input datasets, each input dataset comprising a plurality of data items, the system comprising means to:apply a plurality of logical expressions to the data items in each dataset to derive a plurality of logical signatures characterising the value of each data item; compare the logical signatures to determine at least one logical constraint based on the logical signatures; and determine the synthetic dataset by applying the at least one logical constraint to a logical solving engine, wherein the synthetic dataset satisfies each of the at least one logical constraints.

29. A computer-implemented method for determining a Boolean representation of an array of data items, each data item comprising a respective value of a set of specified values, the method comprising: for each data item of the array of data items: for each specified value of the set of specified values: in response to the data item equaling the specified value, set a corresponding Boolean value of the Boolean representation to represent true; and in response to the data item not equaling the specified value, set a corresponding Boolean value of the Boolean representation to represent false.

30. The method of claim 29, further comprising: for each data item of the array of data items: for each specified value of the set of specified values: in response to the data item equaling the specified value, set a corresponding Boolean value of the Boolean representation to represent false; and in response to the data item not equaling the specified value, set a corresponding Boolean value of the Boolean representation to represent true.

Images