Statistical measurements with active learning for privacy protection evaluation

Abstract

Embodiments of the present disclosure relate to statistical measurements with active learning for privacy protection assessment. Evaluation methods and systems for privacy protection are provided. The method includes determining a first sub-dataset of the first dataset based on a sampling rate, a tolerance, and a first threshold. The method further includes determining a second sub-dataset of the first dataset based on the sampling rate, the tolerance, and a second threshold. The method comprises the following steps: determining a first distance between a first sub-data set and a first threshold value, determining a second distance between a second sub-data set and a second threshold value, and when the first distance is smaller than or equal to the second distance, determining that the first sub-data set is a first sub-data set; a first intersection of the first sub-dataset and the second dataset is generated and a posteriori for an element of the first dataset is updated based on the first intersection, and a positive membership and a negative membership of the element of the first dataset in the second dataset are determined based on the posteriori for the element of the first dataset.

Description

Technical Field

[0001] Embodiments described herein generally relate to evaluating the effectiveness of privacy-preserving protocols or algorithms. More specifically, embodiments described herein relate to statistical measures with active learning for evaluating the effectiveness of privacy-preserving protocols or algorithms. Background Art

[0002] Private Set Intersection (PSI) is a secure two-party or multi-party protocol or algorithm by which statistics related to the intersection are calculated, and PSI has attracted great interest in the industry. The PSI algorithm or protocol allows two or more organizations to jointly calculate functions (e.g., counts, sums, etc.) on the intersection of their respective data sets without explicitly revealing them to the other party. In applications, the two parties may be unwilling or unable to reveal the underlying data to each other, but they may still want to calculate overall group level measurements. The two parties may want to do this while ensuring that the input data sets do not reveal anything other than these aggregate values ​​about individual users. It should be understood that a protocol or algorithm that discloses the size, count, or cardinality of the intersection of the two parties' data sets to one or both parties can be referred to as an intersection size disclosure protocol or algorithm. Summary of the invention

[0003] Features in the embodiments disclosed herein provide a privacy protection effectiveness assessment protocol, algorithm, method and / or system for intersection size revealing protocols. Features in the embodiments disclosed herein employ statistical measurements of active learning methods to enhance and improve privacy protection and / or privacy protection effectiveness assessment techniques or technical fields by, for example, confirming the roles of both positive and negative memberships in the membership inference process, expanding the scope of the data considered, improving efficiency while making negligible compromises on inference accuracy, maintaining effectiveness against counterattack privacy protection mechanisms, thereby providing a robust means to rigorously check the privacy guarantees provided by such mechanisms, providing a more comprehensive privacy assessment solution within a limited number of protocol runs, and providing an efficient quantitative assessment of the effectiveness of privacy protection in set intersection operations, etc.

[0004] It should be understood that some set membership reasoning attacks or evaluation mechanisms may treat the membership of each individual as a deterministic value (e.g., dichotomizing it into positive membership or negative membership with deterministic values). Such deterministic mechanisms may suffer from low reasoning efficiency and susceptibility to noise-introducing privacy-preserving mechanisms (such as differential privacy). The inefficiency lies in the rigid approach of the deterministic mechanism, which can be inferred when the mechanism has high certainty about the membership of an individual. For example, if the cardinality of an input data set to a privacy-preserving protocol is "N", and the protocol's (number of positive or negative memberships) return is "N" or 0, then such a mechanism can be inferred, which means that all individuals in the input data set are positive or negative members. Therefore, when randomness is incorporated into the results (e.g., set intersection, etc.), the "reasoning stop" rule of the deterministic mechanism may be invalid, effectively hindering the evaluation. Features in the embodiments disclosed herein provide a robust statistical evaluation algorithm that utilizes a Bayesian inference method to efficiently determine the membership of each individual and allows for more detailed reasoning beyond the binary constraints of existing mechanisms.

[0005] In an example embodiment, a method for evaluating privacy protection is provided. The method includes determining a first sub-dataset of a first data set based on a sampling rate, a tolerance, and a first threshold. The method also includes determining a second sub-dataset of the first data set based on the sampling rate, the tolerance, and the second threshold. The method also includes determining a first distance between the first sub-dataset and the first threshold, determining a second distance between the second sub-dataset and the second threshold, when the first distance is less than or equal to the second distance, generating a first intersection of the first sub-dataset and the second data set and updating a posteriori for an element of the first data set based on the first intersection, and determining positive membership and negative membership of an element of the first data set in the second data set based on the posteriori for the element of the first data set.

[0006] In another example embodiment, a privacy protection assessment system is provided. The system includes a processor and a memory, the memory being used to store a first data set, a sampling rate, a tolerance, a first threshold, and a second threshold. The processor is used to determine a first sub-data set of the first data set based on the sampling rate, the tolerance, and the first threshold. The processor is also used to determine a second sub-data set of the first data set based on the sampling rate, the tolerance, and the second threshold. The processor is also used to determine a first distance between the first sub-data set and the first threshold, determine a second distance between the second sub-data set and the second threshold, and when the first distance is less than or equal to the second distance, generate a first intersection of the first sub-data set and the second data set and update a posteriori for elements of the first data set based on the first intersection, and determine positive membership and negative membership of elements of the first data set in the second data set based on the posteriori for elements of the first data set.

[0007] In yet another example embodiment, a non-transitory computer-readable medium is provided on which computer-executable instructions are stored. The instructions, when executed, cause one or more processors to perform operations, the operations comprising determining a first sub-dataset of a first data set based on a sampling rate, a tolerance, and a first threshold. The operations also include determining a second sub-dataset of the first data set based on the sampling rate, the tolerance, and the second threshold. The operations also include determining a first distance between the first sub-dataset and the first threshold, determining a second distance between the second sub-dataset and the second threshold, when the first distance is less than or equal to the second distance, generating a first intersection of the first sub-dataset and the second data set and updating a posteriori for elements of the first data set based on the first intersection, and determining positive membership and negative membership of elements of the first data set in the second data set based on the posteriori for elements of the first data set. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The accompanying drawings illustrate various embodiments of embodiments of various other aspects of the system, method and the present disclosure. It will be appreciated by those skilled in the art that the element boundaries (e.g., boxes, box groups or other shapes) shown in the figures represent an example of boundaries. It may be that in some examples, an element may be designed as multiple elements, or multiple elements may be designed as one element. In some examples, an element shown as an internal component of an element may be implemented as an external component in another element, and vice versa. Non-restrictive and non-exhaustive descriptions are described with reference to the following drawings. The components in the figures are not necessarily drawn to scale, but are focused on the illustrated principle. In the following specific embodiments, the embodiments are described only as explanations, because various changes and modifications can become apparent to those skilled in the art from the following specific embodiments.

[0009] Figure 1 is a schematic diagram of an example privacy protection effectiveness assessment system arranged in accordance with at least some embodiments described herein.

[0010] Figure 2A is a diagram illustrating an example of a set intersection operation in accordance with at least some embodiments described herein.

[0011] Figure 2B is a schematic diagram illustrating an example of processing stages or steps of a privacy-preserving effectiveness evaluation algorithm according to at least some embodiments described herein.

[0012] Figure 3 is a flow chart illustrating an example process flow for evaluating the effectiveness of a privacy preservation protocol in accordance with at least some embodiments described herein.

[0013] Figure 4is a flow chart illustrating an example process flow for determining distances for a data set in accordance with at least some embodiments described herein.

[0014] Figure 5 is a flow chart illustrating an example process flow for updating a posteriori probabilities in accordance with at least some embodiments described herein.

[0015] Figure 6 is a schematic block diagram of an example computer system suitable for implementing an electronic device, arranged in accordance with at least some embodiments described herein. DETAILED DESCRIPTION

[0016] In the following detailed description, specific embodiments of the present disclosure are described herein with reference to the accompanying drawings, which form a part of the description. In this description and the accompanying drawings, unless the context otherwise dictates, the same reference numerals represent elements that can perform the same, similar or equivalent functions. In addition, unless otherwise specified, the description of each consecutive figure may refer to one or more features in the previous figures to provide a clearer context and a more substantial explanation of the current example embodiment. However, the example embodiments described in the specific embodiments, the drawings, and the claims are not intended to be limiting. Other embodiments may be utilized, and other changes may be made without departing from the spirit or scope of the subject matter presented herein. It will be readily understood that aspects of the present disclosure, as generally described herein and shown in the accompanying drawings, may be arranged, replaced, combined, separated and designed in a variety of different configurations, all of which are explicitly considered herein.

[0017] It should be understood that the disclosed embodiments are merely examples of the present disclosure, which may be embodied in various forms. Well-known functions or structures are not described in detail to avoid obscuring the present disclosure with unnecessary details. Therefore, the specific structural and functional details disclosed herein should not be interpreted as limiting, but should only serve as a basis for the claims and as a representative basis for teaching those skilled in the art to variously adopt the present disclosure with almost any appropriate detailed structure.

[0018] In addition, the present disclosure may be described in the form of functional block components and various processing steps. It should be understood that such functional blocks can be implemented by any number of hardware and / or software components configured to perform the specified functions.

[0019] The scope of the present disclosure should be determined by the appended claims and their legal equivalents, rather than by the examples given herein. For example, the steps recited in any method claim may be performed in any order and are not limited to the order given in the claims. In addition, no element is essential to the practice of the present disclosure unless expressly described as "critical" or "essential" herein.

[0020] As referred to herein, "data set" or "dataset" is a technical term and may refer to a collection of organized data that is electronically stored and accessed. In an example embodiment, a data set may refer to a database, a data table, a portion of a database or a data table, etc. It should be understood that a data set may correspond to one or more database tables, wherein each column of the database table represents a specific variable or field, and each row of the database table corresponds to a given record of the data set. The data set may list values ​​for each variable and / or for each record of the data set. It should also be understood that a data set may also or alternatively refer to a set of related data and the manner in which the related data is organized. In an example embodiment, each record of a data set may include (multiple) fields or (multiple) elements, such as one or more predefined or predetermined identifiers (e.g., membership identifiers, user identifiers, etc., such as a user's name, email address, phone number, a user's unique ID, etc.), and / or one or more attributes or features or values ​​associated with one or more identifiers. It should be understood that any user's (multiple) identifiers and / or user's data described in this document are permitted, licensed, and / or otherwise authorized by the user for use in the embodiments described herein and in appropriate legal equivalents as understood by those skilled in the art. It should also be understood that the size, count, or cardinality of a data set may refer to the number of records (or rows, elements, etc.) of the data set.

[0021] As referred to herein, "inner join" or "inner-join" is a technical term and may refer to an operation or function that involves combining records from data sets, particularly when there are matching values ​​in fields common to the data sets. For example, an inner join may be performed using a "department" data set and an "employee" data set to determine all employees in each department. It should be understood that an inner join may contain mutually related information from the two data sets in the resultant data set (i.e., the "intersection") of the inner join operation. On the other hand, an outer join may also contain information in its resultant data set that is unrelated to the other data set. A private inner join may refer to an inner join operation of two or more data sets that does not reveal data in the intersection of the two or more data sets.

[0022] As referred to herein, "set intersection" or "intersection" (of two data sets A and B) is a technical term in set theory and may refer to a data set that contains all elements or records in A that also belong to B, or equivalently, a data set that contains all elements or records in B that also belong to A. That is, the intersection of data sets A and B (denoted by A∩B) may refer to the set of all elements or records that are members of both data sets A and B. For example, x is an element or record of the intersection of data sets A and B if and only if x is both an element (or record) of A and an element (or record) of B. For example, the intersection of data sets A (e.g., {1, 2, 3}) and B (e.g., {2, 3, 4}), for example, is the data set {2, 3}. It should be understood that a "set intersection" operation or an "intersection" operation (e.g., an inner join operation, etc.) may refer to an operation, protocol, or algorithm for obtaining, determining, or identifying the intersection of two data sets.

[0023] As referred to herein, the "positive membership" of an element or record in (or relative to) a data set may refer to the state of an element or record as a member of a data set. That is, the positive membership of an element or record in (or relative to a data set) may indicate that the element or record belongs to the data set or is a member of the data set. As described herein, the "negative membership" of an element or record in (or relative to a data set) may refer to the state of an element or record not being a member of a data set. That is, the negative membership of an element or record in (or relative to a data set) may indicate that the element or record does not belong to the data set or is not a member of the data set. In an example where the intersection of data sets A (e.g., {1,2,3}) and B (e.g., {2,3,4}) is data set {2,3}, elements "1", "2" and "3" have positive membership in data set A or relative to data set A, and element "4" has negative membership in data set A or relative to data set A. Elements "2", "3" and "4" have positive membership in data set B or relative to data set B, and element "1" has negative membership in data set B or relative to data set B. Elements "2" and "3" of the intersection have positive membership in or with respect to both data set A and data set B.

[0024] As referred to herein, "leakage" or "privacy leakage" may refer to the disclosure or revelation of information that should be protected, for example, by a privacy protection protocol or algorithm. Membership leakage may refer to the disclosure or revelation of positive and / or negative membership of an element or record in or relative to a dataset. It should be understood that leakage may be quantified using appropriate mechanisms. Expected leakage may refer to a quantified leakage based on a probability distribution of leakage.

[0025] As referred to herein, "private set intersection" is a term of art and may refer to a secure multi-party computation cryptographic operation, algorithm, or function by which two or more parties holding corresponding data sets compare encrypted versions of those data sets to compute an intersection. It should be understood that for private set intersection, no party reveals data elements to the other party, except, for example, the aggregate data of the elements in the intersection.

[0026] As referred to herein, "MPC" or "multi-party computation" is a term of art and may refer to a field of cryptography whose goal is to create schemes that enable parties to jointly compute functions on the parties' joint inputs while maintaining the privacy of the respective inputs. It should be understood that, unlike traditional cryptographic tasks, in which cryptography can ensure the security and integrity of communications or storage when an adversary is outside the participants' systems (e.g., an eavesdropper on the sender and / or receiver), cryptography in MPC can protect the privacy of the participants relative to each other.

[0027] As referred to herein, "Bayesian inference" or "Bayesian probability" is a technical term in statistics and may refer to statistical inference in which Bayes' theorem may be used to update (i.e., Bayesian update) the probability for a hypothesis as more evidence or information becomes available. Bayesian updating may be used for dynamic analysis of data sequences.

[0028] As used herein, "prior", "prior belief", or "prior probability" is a technical term in statistics and may refer to a probability distribution that represents knowledge or uncertainty about a data object before or prior to observation. It is understood that a prior probability distribution for an uncertain quantity is a probability distribution assumed for it before certain evidence is considered. The unknown quantity may be a parameter or latent variable of a model rather than an observable variable. In Bayesian reasoning, Bayes' rule specifies how to update the prior with new information to obtain a posterior probability distribution, which is the conditional distribution of the uncertain quantity given new data.

[0029] As used herein, "posterior", "posterior belief" or "posterior probability" is a technical term in statistics and may refer to a conditional probability distribution representing a parameter that may occur after observing a data object. It should be understood that in Bayesian reasoning, the posterior probability may be a type of conditional probability obtained by updating the prior probability using information summarized by the likelihood method via the application of Bayes' rule. The posterior probability may contain everything that is to be known about the uncertain proposition (e.g., scientific hypotheses, parameter values, etc.) given the prior knowledge and the mathematical model describing the observations available at a particular time. Upon the arrival of new information, the current posterior probability may be used as the prior for the next round of Bayesian updates.

[0030] Figure 1is a schematic diagram of an example privacy protection effectiveness evaluation system 100 arranged in accordance with at least some embodiments described herein.

[0031] System 100 may include terminal devices 110, 120, 130 and 140, network 160 and server 150. It should be understood that Figure 1 Only example numbers of terminal devices, networks, and servers are shown. The embodiments described herein are not limited to the number of terminal devices, networks, and / or servers. That is, the number of terminal devices, networks, and / or servers described herein is provided only for description purposes and is not intended to be limiting.

[0032] According to at least some example embodiments, terminal devices 110, 120, 130, and 140 may be various electronic devices. Various electronic devices may include, but are not limited to, mobile devices such as smart phones, tablet computers, e-book readers, laptop computers, desktop computers, and / or any other suitable electronic devices.

[0033] According to at least some example embodiments, the network 160 may be a medium used to provide a communication link between the terminal devices 110, 120, 130, 140 and the server 150. The network 160 may be the Internet, a local area network (LAN), a wide area network (WAN), a local interconnect network (LIN), a cloud, etc. The network 160 may be implemented by various types of connections, such as a wired communication link, a wireless communication link, an optical fiber cable, etc.

[0034] According to at least some example embodiments, the server 150 may be a server for providing various services to users using one or more of the terminal devices 110, 120, 130, and 140. The server 150 may be implemented by a distributed server cluster including a plurality of instances of the server 150, or may be implemented by a single server 150.

[0035] The user can use one or more of the terminal devices 110, 120, 130, 140 to interact with the server 150 via the network 160. Various applications or their localized interfaces, such as social media applications, online shopping services, privacy protection effectiveness assessment applications, etc., can be installed on the terminal devices 110, 120, 130, 140.

[0036] It should be understood that the software application or service according to the embodiments described herein and / or according to the service provided by the service provider can be executed by the server 150 and / or the terminal devices 110, 120, 130, 140 (which may be referred to as user devices herein). Therefore, the apparatus for the software application and / or service can be arranged in the server 150 and / or in the terminal devices 110, 120, 130 and 140.

[0037] It should also be understood that when the service is not performed remotely, the system 100 may not include the network 160 but only include the terminal devices 110 , 120 , 130 , and 140 and / or the server 150 .

[0038] It should be further understood that the terminal devices 110, 120, 130 and 140 and / or the server 150 may each include one or more processors, memories and storage devices storing one or more programs. The terminal devices 110, 120, 130, 140 and / or the server 150 may also each include an Ethernet connector, a wireless fidelity receiver, etc. When one or more programs are executed by one or more processors, one or more processors may be enabled to perform the method (s) described in any embodiment described herein. In addition, it should be understood that a computer-readable non-volatile medium may be provided according to the embodiments described herein. The computer-readable medium stores a computer program. The computer program is used to perform the method (s) described in any embodiment described herein when executed by a processor.

[0039] Figure 2A is a diagram 200 illustrating an example of a set intersection operation in accordance with at least some embodiments described herein. Figure 2B is a schematic diagram 201 illustrating an example of processing stages or steps of a privacy-preserving effectiveness evaluation algorithm according to at least some embodiments described herein.

[0040] It should be understood that unless otherwise specified, the processing steps of Figures 200 and 201 disclosed herein may be performed by one or more processors (e.g., Figure 1 a processor of one or more terminal devices among the terminal devices 110, 120, 130 and 140, Figure 1 The processor of the server 150, Figure 6 605 and / or any other suitable processor).

[0041] like Figure 2A As shown, party A has data set X, and party B has data set Y. It should be understood that the format, content, and / or arrangement of the data sets described herein are for descriptive purposes only and are not intended to be limiting.

[0042] In an example embodiment, data set X includes eight elements or records {x1, x2, x3, x4, x5, c1, c2, c3}. Data set Y includes three or more elements or records {c1, c2, c3, y1, y2, y3, y4, y5...}. A set intersection operation (e.g., PSI, PSI based on MPC, etc.) of data set X and data set Y can be performed, and the resulting intersection of data set X and data set Y (data set C) includes three elements {c1, c2, c3}. It should be understood that PSI and / or PSI based on MPC algorithms or protocols can allow party A and party B to jointly calculate functions (e.g., counts, sums, etc.) on the intersection of data sets X and Y without explicitly revealing the intersection to either party. That is, the set intersection operation of data set X and data set Y may not reveal the intersection C to either party, but may disclose the size, count, or cardinality ("3" in this example) of the intersection C of data sets X and Y to one or both parties via, for example, an intersection size disclosure protocol or algorithm.

[0043] It should be understood that intersection operations or set intersection operations can refer to basic operations that can be performed on large data sets in situations such as MPC-based PSI operations, structured query language data set / database join operations, contact discovery, etc. It should also be understood that when an intersection operation requires input data sets from two different data owners and / or parties, the two parties may be required to compute the intersection of private data sets without disclosing the data and / or user membership within each data set. Many methods can protect user privacy by allowing two parties to jointly compute functions (e.g., counts, sums, averages, etc.) on the intersection of their data sets without revealing the underlying intersection data (e.g., retaining the set intersection in ciphertext). It should be understood that the cardinality of the intersection can be disclosed to one or both parties via, for example, an intersection size disclosure protocol.

[0044] It should also be understood that the intersection size revealing protocol may be subject to set membership inference attacks, that is, as a party to the intersection size revealing protocol, the attacker (or evaluator) may eventually learn the membership of any subset of individuals / users / records from one party by continuously calling the intersection size revealing protocol. Membership may refer to the state of whether an individual or user is in the data set of one party or the other party. It should also be understood that attacking or evaluating privacy protection protocols (such as the intersection size revealing protocol) can be used as a way to evaluate the effectiveness of such protocols. In practical applications, existing attacks may not be efficient enough because most data set owners may only allow a limited number of intersection size revealing protocols to be run on their data sets during fixed time intervals. Under a certain number (e.g., limited, predetermined, etc.) of intersection size revealing protocol calls, the membership leakage caused by existing attacks may be very small or zero, while the real privacy leakage caused by a potentially more powerful attacker may be more significant. The features in the embodiments disclosed herein take into account both positive membership leakage and negative membership information, and provide a more efficient privacy protection effect evaluation method within the number of times the intersection size revealing protocol is run.

[0045] It is further understood that the MPC-based PSI operation may include one or more of the following processes: Either party (Party A and / or Party B) may shuffle and encode (or encode and shuffle) its corresponding data set X and / or Y. It is understood that shuffle or shuffling is a technical term and may refer to an action or algorithm for randomly rearranging the order of, for example, records (elements, rows, etc.) of a data set. Encode or encoding is a technical term and may refer to an action or algorithm for, for example, encrypting records (elements, rows, etc.) or fields (attributes, columns, etc.) of a data set based on, for example, an encryption algorithm or protocol. The parties may exchange their corresponding shuffled and encoded data sets with each other, shuffling and encoding (or encoding and shuffling) the exchanged data sets. The parties may exchange their corresponding result data sets (encoded by Party A and then encoded by Party B, or encoded by Party B and then encoded by Party A). That is, both party A and party B have data set X (encoded by both party A and party B) and data set Y (encoded by both party A and party B), but may not know the contents of those data sets, because each party may have its own encryption key to encode the data sets. One or both parties may perform an intersection operation based on data set X (encoded by both party A and party B) and data set Y (encoded by both party A and party B) to generate or obtain an intersection. The contents of the intersection are unknown to either party (because each party may have its own encryption key to encode the data set, which may produce an unknown intersection), but the size, count or cardinality of the intersection may be known to one or both parties. It should be understood that the intersection may be used for further processing, such as generating confidential shares, collecting confidential shares and / or generating results by combining collected confidential shares, etc. The above-mentioned MPC-based PSI operation (returning or revealing the cardinality of the intersection and / or only revealing the cardinality of the intersection) may refer to a PSI-CA (private set intersection cardinality) operation.

[0046] review Figure 2A , a PSI operation (such as a PSI-CA operation) may be performed on dataset X and dataset Y, and the cardinality of the intersection (and only the cardinality of the intersection) C may be revealed to one or both parties. That is, after the PSI operation between dataset X and dataset Y, party A may know that "3" of the "8" elements or records in dataset X (without knowing which "3" elements or records) may have positive membership in or with respect to dataset Y, while "8-3=5" of the "8" elements or records in dataset X (without knowing which "5" elements or records) may have negative membership in or with respect to dataset Y.

[0047] Figure 2B Shown based on Figure 2A An example of a processing stage or step of an algorithm for evaluating the privacy-preserving effectiveness of the cardinality of the intersection disclosed in 201. Figure 2A As shown, it is known or revealed that (Party A's) data set X has 3 elements or records in data set Y or has positive membership relative to data set Y, and has 5 elements or records in data set Y or has negative membership relative to data set Y, but it is unknown which 3 members are positive members (i.e., the elements have positive membership in data set Y) and which 5 members are negative members (i.e., the elements have negative membership in data set Y). Figure 2B As shown, the upper threshold θ u (e.g., an upper stopping threshold for stopping the evaluation iteration / process), a lower threshold θ l (e.g., a lower stopping threshold for stopping the evaluation iteration / process), updated posterior probabilities of elements of a dataset (e.g., dataset X or a subset of dataset X, etc.), and sampling scenarios for PSI operations (between the selected sub-dataset and another input dataset Y as input to the next PSI operation). See . Figures 3 to 5 Description.

[0048] In the stage or step 210 of the privacy protection effect evaluation algorithm 201, based on Figure 2A The revealed cardinality of the intersection is to update the posterior probability of each element in the original dataset X from the initial value (i.e., the posterior probability before the PSI operation or the prior probability initialized with, for example, a random variable 0.5) to 3 / 8, where 3 is the number of positive members in the original dataset X (i.e., Figure 2A The revealed cardinality of the intersection is 3), and 8 is the number of all members of the original dataset X. For updating the posterior probability, see Figure 4 and Figure 5 It should be understood that τ (not shown) is a constraint (the intersection size reveals the maximum allowable number of invocations or the number of remaining invocations of a protocol (such as a PSI operation)), and each time the processor performs, calls, executes, runs, or implements a PSI operation on an input data set, the processor may obtain, receive, or generate the cardinality of the resulting intersection and reduce the constraint number τ by 1 (τ=τ-1).

[0049] In stage or step 210, in the original dataset X containing 8 elements, sub-datasets (in this example containing 3 elements) are generated, where the posterior probabilities of all sub-datasets of the dataset X are at the same distance from the threshold, see Figure 4 description) can be determined (see Figure 3 ), which can (along with the input data set Y) be used as input to the next PSI operation (see Figure 3As shown in stage or step 210, the next PSI operation (between the sub-dataset containing 3 elements and data set Y) can reveal the cardinality of the resulting intersection (in this example, "2"). Processing can proceed from stage 210 to stage 220.

[0050] In stage or step 220, the cardinality ("2") of the newly revealed result intersection in stage 210 can be used to update the posterior probability of the elements in the sub-dataset containing 3 elements, and the updated posterior probability can be 2 / |cardinality of the sub-dataset containing 3 elements|=2 / 3. For the remaining 5 elements (which are not members of the sub-dataset containing 3 elements, denoted as "sub-dataset containing 5 elements"), their posterior probabilities can be updated to (3-2) / |cardinality of the sub-dataset containing 5 elements|=1 / 5, where 3 is from Figure 2A is the cardinality of the intersection of the previous PSI operation of stage 210, and 2 is the cardinality of the intersection of the most recent PSI operation from stage 210. Figure 3 At this stage 220, no element in the dataset X is classified as a positive or negative member, and the next evaluation iteration continues. Of the two sub-datasets (e.g., the upper sub-dataset with 3 elements and the lower sub-dataset with 5 elements), the sub-dataset with the smaller distance (see Figure 3 ) can be selected (along with Y) as a candidate input for the next PSI operation. Assuming a sub-dataset with 3 elements is selected, by applying the sampling rate (see Figure 3 330 ), a portion of the sub-dataset containing 3 elements (2 elements in this example) is selected as the final input to the next PSI operation (together with Y). In this example, as shown in the example scenario of stage or step 220, another newly revealed cardinality (again "2" in this example) of the intersection of the PSI operation (between the final input data set containing 2 elements and data set Y) can be used to update the posterior probability of the final input data set containing 2 elements, as well as to update the posterior probability of the remaining elements in the sub-dataset containing 3 elements (1 element in this example, which is not a member of the final input data set containing 2 elements). Processing can proceed from stage 220 to stage 230.

[0051] At stage or step 230, based on the updated posterior probability (with a threshold θ u and / or θ l), 2 elements in the final input data set can be classified as positive members, and 1 remaining element can be classified as a negative member. The classified data sets Zpos (containing all identified positive members) and Zneg (containing all identified negative members) are updated so that Zpos contains 2 positive members and Zneg contains 1 negative member. See Figure 3 The next evaluation iteration continues. Similar to the processing in stage or step 210, in the sub-dataset containing 5 unidentified elements, the sub-sub-datasets (in this example containing 3 elements, where the distances of all sub-sub-datasets of the sub-dataset of 5 elements are the same) can be determined (see Figure 3 ), which can (along with the input data set Y) be used as input to the next PSI operation (see Figure 3 As shown in the sample example scenario in stage or step 230, the new PSI operation (between the sub-sub-dataset containing 3 elements and data set Y) can reveal the cardinality of the resulting intersection (in this case, "0"). Processing can proceed from stage 230 to stage 240.

[0052] At stage or step 240, the revealed cardinality ("0") may be used to update the posterior probabilities of the elements in the sub-sub-dataset. Figure 3 All 3 elements are classified as negative members at this stage 240, and the classified data sets Zpos and Zneg are updated (Zpos still contains 2 positive members, and Zneg now contains 4 negative members). See Figure 3 The next evaluation iteration continues. In the remaining data set containing the last 2 elements, a sub-data set (in this example containing 1 element, where the distances for all sub-data sets of the remaining data set are the same) can be determined (see Figure 3 ), which can (along with the input data set Y) be used as input to the next PSI operation (see Figure 3 As shown in the sample scenario in stage or step 240, the new PSI operation (between the sub-dataset containing 1 element and data set Y) can reveal the cardinality of the resulting intersection (in this example, "1"). Processing can proceed from stage 240 to stage 250.

[0053] At stage or step 250, the revealed cardinality ("1") can be used to update the posterior probabilities of the elements in the sub-dataset. Figure 3 In this stage 250, both elements are classified as 1 positive member (and now from Figure 2AAll 3 positive members of are identified) and 1 negative member (due to Figure 2A All 3 positive members of are identified, so the remaining 1 element must be a negative member), and the classified data sets Zpos and Zneg are updated (Zpos contains 3 positive members, and Zneg contains 5 negative members). See Figure 3 Description of block 370 of . Since all memberships (positive and negative memberships) of the elements in the original data set X are identified, the evaluation process ends.

[0054] Figure 3 is a flow chart illustrating an example process flow 300 for evaluating the effectiveness of a privacy protection protocol according to at least some embodiments described herein. It should be understood that the process flow 300 disclosed herein may include: Figure 2A and 2B Process flow 300 provides statistical measures or assessments to make or perform membership (positive or negative membership) inferences based on, for example, statistical reasoning.

[0055] It should be understood that the process flow 300 disclosed herein may be executed by one or more processors (e.g., Figure 1 a processor of one or more terminal devices 110, 120, 130 and 140, Figure 1 The processor of the server 150, Figure 6 605 and / or any other suitable processor), unless otherwise specified.

[0056] It should also be understood that the processing flow 300 may include one or more operations, actions, or functions, as shown in one or more of blocks 310, 320, 330, 340, 350, 360, 370, 380, and 390. These various operations, functions, or actions may correspond, for example, to software, program code, or program instructions executable by a processor that cause the functions to be instructed. Although illustrated as separate blocks, obvious modifications may be made, for example, two or more blocks may be reordered; additional blocks may be added; and various blocks may be divided into additional blocks, combined into fewer blocks, or eliminated depending on the desired implementation. It should be understood that prior to the processing flow 300, operations including initialization and the like may be performed. For example, system parameters and / or application parameters may be initialized.

[0057] In an example embodiment, Party A has a data set X (including multiple elements, rows, or records), and Party B has a data set Y (including multiple elements, rows, or records). It should be understood that the format, content, and / or arrangement of the data sets described herein are for descriptive purposes only and are not intended to be limiting. Party A is the evaluator of the effectiveness of the privacy protection protocol and / or the intersection size revealing protocol. That is, Party A can try various attack techniques or run algorithms disclosed herein to reveal as many memberships (positive memberships or negative memberships) of the elements of data set X in or relative to data set Y as possible to evaluate the effectiveness of the privacy protection protocol.

[0058] In an example embodiment, the size or count or cardinality of dataset X may be represented as |X|, and the cardinality of dataset Y may be represented as |Y|. It should be understood that "cardinality" is a mathematical term that may refer to a measure of the size of a set, meaning the number of elements in a set (or rows, records, etc. in a dataset). The cardinality of the intersection of dataset X and dataset Y may be represented as |X∩Y|. In the context of the intersection size revealing protocol, the party receiving the intersection size (cardinality of the intersection) obtains a measure of the similarity between its own dataset and the dataset of the other party. Given that the party can freely choose its own dataset for the protocol, the party may make a strategic evaluation of the other party's dataset based on its interests (e.g., evaluating the effectiveness of a privacy-preserving protocol, etc.). In an example embodiment, the PSI-SUM (private set intersection sum) operation or protocol may return a tuple containing the cardinality of the intersection and the sum of the elements in the intersection (|X∩Y|, ∑ xi∈(X∩Y) xi). Features in the embodiments disclosed herein may address the problem when an evaluator has a large number of target elements: whether the evaluator can use its ability to measure similarity to determine the membership (positive or negative membership) of the target element in the other party's dataset.

[0059] In an example embodiment, party A (evaluator, attacker, etc.) can initiate a privacy protection evaluation and can participate in running or executing the intersection size revealing protocol or algorithm as a party multiple times. During each protocol or algorithm execution, for the PSI-SUM operation generated by its input data set and party B's data set, the evaluator (e.g., party A, etc.) can select its input and obtain the intersection size (cardinality of the intersection) and / or the sum of the element values ​​in the intersection. The evaluator is allowed to repeatedly participate in the privacy protection protocol or algorithm call with the same party (e.g., party B, etc.) under the query budget (e.g., the predefined number of times the intersection size revealing protocol or algorithm is run or executed). Party A (evaluator) can have a set of target elements or records and aims to determine the membership status (e.g., positive or negative membership) of these target elements or records in party B's data set. Based on different applications, the evaluator may be interested in different membership information (e.g., positive membership or negative membership, etc.). Party A can adaptively design an evaluation strategy to maximize the expected return (e.g., the maximum number of positive membership or negative membership of elements in party B's data set). It is to be understood that the effectiveness of the evaluator depends on the ability to repeatedly call the protocol with party B. In actual scenarios, the number of times a protocol can be called (called, run, executed, etc.) may be limited due to constraints (e.g., a predetermined number or budget, etc.). Such constraints may limit the ability of the evaluator to perform an exhaustive search and reduce the accuracy of the inferred dataset membership. The features of the embodiments disclosed herein can provide an efficient algorithm (or method, protocol, etc.) for quantitatively evaluating membership leakage in such dataset intersection operations, and / or can maximize the number of revealed memberships (positive memberships or negative memberships) of elements of dataset X in dataset Y or relative to dataset Y as much as possible to evaluate the effectiveness of the privacy-preserving protocol within such constraints. The features of the embodiments disclosed herein can evaluate the effectiveness of the privacy-preserving protocol even when, for example, noise and / or uncertainty has been introduced into a dataset (e.g., dataset Y) or an intersection of datasets (e.g., X and Y), or when a dataset (e.g., dataset Y) or an intersection of datasets (e.g., X and Y) becomes dynamic.

[0060] review Figure 3 , the process flow 300 may start at block 310 .

[0061] At block 310 (Initialize), the processor may perform or conduct a number of initialization processes. In an example embodiment, the processor may receive, obtain, or provide a dataset X (e.g., for the evaluator, Party A) containing a target element or record. The target element (i.e., an element in dataset X, etc.) is an element for which the evaluator is attempting to identify membership (e.g., positive membership or negative membership in or relative to Party B's dataset Y). The processor may also receive, obtain, determine, or provide an upper threshold value (θu )), lower threshold (θ l ), sampling rate (r), tolerance factor or number (tol). The threshold, sampling rate, and tolerance factor will be described in detail below. The processor may also receive, obtain, or provide a constraint τ (i.e., the maximum allowable number or remaining number of PSI (PSI-CA, PSI-SUM, etc.) protocol calls or privacy-preserving protocol calls to be evaluated). Additionally, the processor may initialize the classified dataset Zpos (used to store the identified / classified positive members) to an empty dataset and the classified dataset Zneg (used to store the identified / classified negative members) to an empty dataset. The processor may also assign a probability distribution to each target element (e.g., user, record, etc.). It should be understood that before the PSI operation, the probability distribution of each element may be referred to as the prior or prior probability of such an element. After the PSI operation, the probability distribution of each element may be referred to as the posterior or posterior probability, which may be updated based on the prior probability and the result(s) of the PSI operation(s) performed. The processor may also initialize the prior probability (p[i], where i = 1 to N, and N is the cardinality of the dataset X, 0 <= p[i] <= 1) of each target element to a random number / variable (e.g., 0.5, etc.). For simplicity of description, the "posterior probability" (p[i], representing the likelihood that element "i" is a positive or negative member) may always be used to represent both the prior probability and the posterior probability, where the posterior probability p[i] is initialized to a random number / variable (e.g., 0.5, etc.). The process may proceed from block 310 to block 320.

[0062] At block 320 (constraint evaluation), the processor may execute a loop (e.g., a "while" loop) until either (1) the condition |Zpos + Zneg| < N is not satisfied or (2) the condition τ > 0 is not satisfied. That is, the loop may end either when the membership (positive membership or negative membership) of all elements in the dataset X is identified as positive membership or negative membership in or relative to the dataset Y (|Zpos + Zneg| = |N|), or when the constraint number limit is reached (i.e., each time a PSI operation is called, the number τ is decremented by 1 and τ is decremented to 0).

[0063] Within block 320, the first processing step is block 330. At block 330 (determine candidate sub-dataset), the processor may determine or generate a candidate sub-dataset, e.g., using active learning techniques. Active learning may refer to the process of actively identifying or determining a sub-dataset that may reveal the most memberships (e.g., maximum number of positive memberships and / or negative memberships) in the next PSI operation.

[0064] It should be understood that in the evaluation process / protocol / algorithm (of the effectiveness of the privacy protection protocol) described herein, the evaluator may treat the membership of each target element (e.g., individual) as a random variable (e.g., a binary random variable). It should be understood that "random variable" is a mathematical term and may refer to a measurable function from a probability measure space or a sample space to a measurable space. The distribution of a random variable may refer to a probability measure on a set of all possible values ​​of the random variable. The evaluation process described herein is based on updating the posterior probabilities on these random variables to maximize or minimize the random variables until the stopping criteria of the evaluation process are matched. The evaluation process described herein may make a guess for each element in a data set (e.g., in data set X), and this strategy may ensure that the guessing accuracy is equal to or higher than the expected threshold.

[0065] It should also be understood that in the statistical evaluation described herein, the membership of each target element is treated as a random variable (e.g., a binary random variable), which allows the evaluator to design a stopping criterion to end the guessing. Therefore, the evaluator's goal is to select a subset of the target dataset (e.g., dataset X and / or a sub-dataset of X) such that the evaluator can accurately infer the identity of the individual members. i Represented as a binary random variable, if the i-th element "xi" of the data set X is in the intersection (xi∈X∩Y), then L i =1 (otherwise L i =0). The membership of the target element can then be assigned to a random vector p, where p[i] represents the i-th element Pr(L i =1)(or Pr(L i =0)). The evaluator may determine a subset S of the dataset X as an input to the PSI operation (the other input to the PSI operation is the dataset Y), where “O” represents the cardinality of the intersection released or revealed from the PSI operation: O=|S∩Y|.

[0066] It should also be understood that the active learning technique (used by the processor to determine or generate a candidate sub-dataset as an input to the next PSI operation, where the other input to the PSI operation is the dataset Y) includes two stages: (1) a method for determining or generating a candidate sub-dataset by comparing each posterior probability (of an element in the candidate sub-dataset) to a threshold (upper threshold θ u and / or lower threshold θ l ), and (2) for constructing candidate sub-datasets based on the absolute distance between them (see Figure 4 Description of the PSI) to determine the stage of the input data set (to the PSI operation).

[0067] At block 330 (determine candidate sub-dataset), the processor may also minimize the posterior probability of the elements in the candidate sub-dataset and the threshold (upper threshold θ u and / or lower threshold θ l ) between the two sets of data. It should be understood that during the evaluation process, the evaluator may aim to select target elements (e.g., individuals) that may meet the stopping criteria of the evaluation process or iteration as the input dataset for the next PSI operation. This selection can be made from two different directions: (1) the posterior probability (i.e., the prior probability) before selecting the PSI operation is close to the upper threshold θ u The target element, and (2) the posterior probability (i.e., prior probability) before selecting the PSI operation is close to the lower threshold θ l It should also be understood that the upper threshold θ u and / or lower threshold θ l Can be used as a stopping criterion for an evaluation process or iteration, indicating a threshold for the likelihood or probability that an element is a positive or negative member.

[0068] For example, if θ u is set or predetermined to 0.99 (or other suitable number between 0.5 and 1), and the current or updated posterior probability of the target element is equal to or higher than the upper threshold θ u , then such a target element is considered a positive member (where the probability of a positive member being a positive member is assumed to be 1 or 100%). l is set or predetermined to 0.01 (or other suitable number between 0 and 0.5), and the current or updated posterior probability of the target element is equal to or lower than the lower threshold θ l , then such a target element is regarded as a negative member (where the probability of a negative member being a negative member is assumed to be 0). The upper threshold θ u and / or lower threshold θ l The determination and utilization of can provide an effective evaluation while having negligible impact on the inference accuracy. u The closer to 1 and / or the lower threshold θ l The closer to 0, the more accurate the membership inference result may be, and the less efficient the evaluation may be (e.g., more time and / or more calls to the PSI operation may be required to stop or end the evaluation process for membership inference). u The further away from 1 and / or the lower threshold θ l The further away from 0, the less accurate the result of membership inference may be and the more efficient the evaluation may be (eg, less time and / or fewer invocations of PSI operations may be required to stop or end the evaluation process for membership inference).

[0069] It can be understood that the absolute distance between the posterior probability of each target element in the candidate sub-dataset and the upper threshold can be expressed as It can be |p[i]-θ u |. The absolute distance between the posterior probability of each target element in the candidate sub-dataset and the lower threshold can be expressed as It can be |p[i]-θ l |.

[0070] For a given iteration (the "while" loop of block 320), the minimized distance (to the upper threshold and to the lower threshold) in the posterior vector "p" may be defined accordingly as and It should be understood that directly selecting the target element with the highest or lowest posterior probability (and / or prior probability), that is, selecting the element whose corresponding posterior probability has the smallest distance from the upper threshold (and / or lower threshold), may not introduce randomness into the output, and thus may cause an infinite loop (or a loop that does not cause useful results) in the evaluation process.

[0071] Therefore, a sampling rate (r) and a tolerance factor (tol) may be provided, obtained, determined, or generated by a processor to introduce randomness to the output, thereby improving the efficiency of the evaluation process. The sampling rate (r) may indicate a portion of the elements to be selected. In an exemplary embodiment, the sampling rate (r) may be a value between 0 and 1 (e.g., 0.5, 0.3, 0.8, etc.). If M elements meet certain conditions and are to be selected, the sampling rate of r indicates that random r×M elements of the M elements may be selected instead of selecting M elements. The tolerance factor (tol) may be a value between 0 and 0.5 (e.g., 0.2, 0.15, etc.). The tolerance factor (tol) may indicate the desired range between (1) the distance between the posterior probability of an element and a threshold (upper threshold or lower threshold) and (2) the minimized distance (to the upper threshold or to the lower threshold). For example, if the threshold tol is 0.05 and the minimized distance ( or ) is 0.1, then (or ) or less (which is 0.05+0.1=0.15 or less) can be selected, rather than only those with the minimized distance ( or ) is selected. In an example embodiment, the sampling rate (r) and / or the tolerance factor (tol) may be predetermined as a fixed value throughout the evaluation process. In an example embodiment, the sampling rate (r) and / or the tolerance factor (tol) may be dynamically changed during the evaluation process (e.g., each time before or after a PSI operation).

[0072] It will be appreciated that the sampling rate and the tolerance factor may together determine a grouping method for selecting elements of a candidate sub-dataset (of dataset X) as input (to the next PSI operation, the other input being dataset Y). The candidate sub-datasets for the next iteration (loop of block 320) include two sub-datasets: and 1 of them sample is an indicator function determined by the sampling result (i.e., the sampling rate). u is a candidate sub-dataset of dataset X, where (the distance between the posterior probability of an element and the upper threshold) and this distance is about the distance minimized Among all elements within the tolerance tol of (the distance to the upper threshold) (assuming that there are K elements in the dataset X that meet this condition), a portion (K×r) of all these elements (by multiplying by the sampling rate r) is randomly selected as the candidate sub-dataset S u part of S u The cardinality of is the nearest integer converted to K×r. Similarly, S l is a candidate sub-dataset of dataset X, where (the distance between the element posterior probability and the lower threshold), and this distance is about the distance minimized Among all the elements within the tolerance tol of (the distance from the lower threshold) (assuming that there are J elements in the dataset X that meet this condition), a portion (J×r) of all these elements (by multiplying by the sampling rate r) are randomly selected as candidate sub-datasets S l part of S l The cardinality of is the nearest integer converted to J×r.

[0073] It should also be understood that larger tolerance factor may cause more target elements that are not easy to meet the stopping criteria of evaluation process to be grouped, wherein the posterior probability of the stopping criteria indicating element that meets is equal to or higher than the upper threshold or is equal to or lower than the lower threshold. Larger tolerance factor allows to explore target element more widely to find those target elements that posterior probability may exceed threshold, and larger candidate size can increase the number of target elements that membership can be inferred in one iteration. Small sampling rate tends to reduce the reasoning scope quickly. The assessor can quickly infer the target element with a smaller part of positive or negative membership. When sampling rate is too small, this method may cause the overall reasoning efficiency of reduction, because the algorithm may be prone to depth-first search behavior. The selection of sampling rate and tolerance factor depends on specific evaluation target and constraint. By carefully tuning parameters (sampling rate and / or tolerance factor), the assessor can weigh the efficiency of evaluation process, the number of reasoning carried out in each iteration (loop of frame 320) and / or the trade-off between the overall accuracy of reasoning / evaluation process.

[0074] In summary, at block 330 (Determine Candidate Sub-Datasets), the processor may determine or generate candidate sub-datasets S of dataset X. u and / or S l From block 330 , processing may proceed to block 340 .

[0075] At block 340 (Determine Distance), the processor may determine the distance for the candidate sub-data set S u and / or S l (e.g., minimize the Manhattan distance). For more information, see Figure 4 From block 340 , processing may proceed to block 350 .

[0076] At block 350 (Perform PSI), the processor may select a candidate subset of data sets S u and / or S l The sub-dataset with the smaller distance determined at block 340 is selected between them. Assume S u With a ratio of S l If the distance is small, the processor can process the data set S u Perform, call, execute, run or implement PSI (PSI-CA, PSI-SUM, etc.) operations with dataset Y; obtain, receive or generate the cardinality CS of the result intersection u ; and reduce the number of constraints τ by 1 (τ = τ - 1). It should be understood that the cardinality of the intersection CS u It can indicate that the dataset S u How many positive members there are. The constraint number τ is the remaining constraints (eg, the remaining number of times the PSI operation can be called). Processing may proceed from block 350 to block 360.

[0077] At block 360 (update posterior), the processor may update the cardinality CS of the intersection of the results of block 350. u To update the selected candidate sub-dataset S u The posterior probabilities of all elements (in block 350) and the cardinality CS based on the intersection of the results of block 350 u To update all remaining elements in dataset X or its sub-dataset (dataset X or its sub-dataset is not sub-dataset S u The posterior probability of those elements that are members of . Figure 5 Processing may proceed from block 360 to block 370.

[0078] At block 370 (update sorted sets), the processor may execute a loop (e.g., a "for" loop) with a condition that all elements in data set X or a sub-data set of X have been processed. The condition may be "1<=i<=N", indicating that the integer i starts at 1 and goes up to the cardinality N of data set X or a sub-data set of X. Within the "for" loop, the processor may update the sorted data sets Zneg and Zpos by: if the i-th element (X) in data set X or a sub-data set of X is i ) is equal to or lower than the lower threshold θ l , then such elements are considered negative members and Zneg is updated to Zneg∪X i ; and if the posterior probability p[i] of the i-th element is equal to or higher than the upper threshold θ u , then such elements are considered positive members and Zpos is updated to Zpos∪X i Processing may proceed from block 370 to the beginning of block 320 until the loop of block 320 is complete. Processing may then proceed from either block 370 or 320 to block 380.

[0079] In block 380 (return classified set), the processor may return the classified data sets Zneg and Zpos as outputs of the privacy protection protocol evaluation algorithm. The processor may also evaluate the effectiveness of the privacy protection protocol (e.g., PSI protocol) by measuring the number of positive examples in the data set Zpos and / or the number of negative examples in the data set Zneg. Given the same constraints (e.g., the number of maximum allowable invocations of the privacy protection protocol to be evaluated), the greater the number in Zneg and / or Zpos, the less efficient the privacy protection protocol (to be evaluated) may be. Processing may proceed from block 380 to block 390.

[0080] At box 390 (update privacy protection protocol), the processor may improve the privacy protection protocol to be evaluated by at least one of the following: (1) adding (multiple) noises to data set Y, (2) reducing the maximum allowable number of calls (τ) of the privacy protection protocol to be evaluated, (3) adding (multiple) noises to the intersection result based on the evaluation of the privacy protection protocol (e.g., the PSI protocol) when the PSI operation is performed between the data set and the data set Y, etc.

[0081] Figure 4 is an example process flow 400 for determining distances for a data set according to at least some embodiments described herein (see Figure 3 340 of the flowchart).

[0082] It should be understood that, unless otherwise specified, the process flow 400 disclosed herein may be executed by one or more processors (e.g., Figure 1 a processor of one or more terminal devices among the terminal devices 110, 120, 130 and 140, Figure 1 The processor of the server 150, Figure 6 605 and / or any other suitable processor).

[0083] It should also be understood that the processing flow 400 may include one or more operations, actions, or functions, as shown by one or more of the blocks 410 and 420. These various operations, functions, or actions may correspond, for example, to software, program code, or program instructions executable by a processor that cause the functions to be performed. Although illustrated as separate blocks, obvious modifications may be made, for example, two or more blocks may be reordered; additional blocks may be added; and various blocks may be divided into additional blocks, combined into fewer blocks, or eliminated depending on the desired implementation. It should be understood that prior to the processing flow 400, operations including initialization and the like may be performed. For example, system parameters and / or application parameters may be initialized. The processing flow 400 may begin at block 410.

[0084] In block 410 (determining the u The processor can determine the distance of the data set S u It should be understood that the data set S u Can be Figure 3 The candidate sub-dataset S determined at block 330 u The distance can be the dataset S u (The posterior probability of each element in) and the upper threshold θ uManhattan distance is a technical term and can refer to the distance between two points measured along a rectangular axis, or a distance measurement measured by summing the distance between x and y coordinates. In an example embodiment, the data set S u (The posterior probability of each element in) and the upper threshold θ u The Manhattan distance D u can be expressed as: in is the distance between the posterior probability of the i-th element and the upper threshold, and |S u | is the dataset S u From block 410 , processing may proceed to block 420 .

[0085] In block 420 (determining the l The processor can determine the distance of the data set S l It should be understood that the data set S l Can be Figure 3 The candidate sub-dataset S determined at block 330 l The distance can be the dataset S l (The posterior probability of each element in) and the lower threshold θ l In one example embodiment, the data set S l (The posterior probability of each element in) and the lower threshold θ l The Manhattan distance D l can be expressed as: in is the distance between the posterior probability of the i-th element and the lower threshold, and |S l | is the dataset S l The cardinality (or size, count, etc.) of .

[0086] It should be understood that as the evaluator continues to call PSI operations, the posterior probability of each target element either moves towards θ u Either towards θ l Advance. Figure 5 Description and Figure 3 Given a tolerance factor tol, before each invocation of a PSI operation, the evaluator may u and S l The processor can select as the input for the next PSI operation by comparing S u and θ u and S l and θ lThe evaluator may select a dataset (S) with a smaller distance toward the corresponding threshold (e.g., a smaller Manhattan distance toward the corresponding threshold). u and S l See Figure 3 Description of blocks 340 and 350 .

[0087] Figure 5 is a flow chart illustrating an example process flow 500 for updating a posteriori probabilities according to at least some embodiments described herein (see Figure 3 Description of box 360).

[0088] It should be understood that the process flow 500 disclosed herein may be executed by one or more processors (e.g., Figure 1 a processor of one or more terminal devices among the terminal devices 110, 120, 130 and 140, Figure 1 The processor of the server 150, Figure 6 605 and / or any other suitable processor), unless otherwise specified.

[0089] It should also be understood that the processing flow 500 may include one or more operations, actions, or functions, as shown by one or more blocks 510 and 520. These various operations, functions, or actions may correspond, for example, to software, program code, or program instructions executable by a processor that cause these functions to be performed. Although shown as separate blocks, obvious modifications may be made, for example, two or more blocks may be reordered; additional blocks may be added; and various blocks may be divided into additional blocks, combined into fewer blocks, or eliminated depending on the desired implementation. It should be understood that prior to the processing flow 500, operations including initialization and the like may be performed. For example, system parameters and / or application parameters may be initialized. The processing flow 500 may begin at block 510.

[0090] In block 510 (update the t The processor can calculate the cardinality of the intersection based on the cardinality O of the intersection revealed in the most recent PSI operation. t (For example, the cardinality CS of the result intersection of block 350 u ) to update the dataset S t The posterior probability of all elements in (e.g., the selected candidate sub-dataset S at block 350) u ).

[0091] It should be understood that in each PSI operation (in data set S t and dataset Y), the evaluator can observe S t The intersection of O and Y tThe cardinality of the data set S that is the input to the PSI operation t The positive members contained in (in or relative to the dataset Y of party B). Based on this observation, the evaluator can update the prediction for the dataset S based on the maximum likelihood criterion (described in detail below). t For example, for a given pair of PSI input S t and output O t (the cardinality of the intersection of the results from the PSI operation), and the internal data set S t The updated posterior probability p[i] of each element in is O t / |S t |, where |S t | is the dataset S t For example, if the dataset S t The cardinality of the dataset S is 8 (i.e., t has 8 elements, rows, records, etc.), and O t (From S t The cardinality of the intersection of the result of the PSI operation between and Y is 3, then for the data set S t The posterior probability of each element in is updated to 3 / 8, which indicates that the dataset S t Each element in has a probability of 3 / 8 of being a positive member of data set Y (or a probability of 1-3 / 8=5 / 8 of being a negative member of data set Y).

[0092] It should be appreciated that the update to the posterior probability may reflect the observed positive members (i.e., t ) and the dataset S t The cardinality (which is S t The ratio of the input to the PSI operation between y and y. t Relative to the data set S t The higher the size, the more t The higher the posterior probability of each element in , the higher the posterior probability of each element in . Processing can proceed from block 510 to block 520.

[0093] In block 520 (update for non-S t The processor can calculate the cardinality of the intersection revealed in the most recent PSI operation based on the cardinality O of the intersection. t (For example, the cardinality CS of the result intersection of block 350 u ) to update the dataset X or S t The posterior probability of all remaining elements in the sub-dataset of dataset X (i.e., the posterior probability of all remaining elements in dataset X or the sub-dataset of dataset X that are not in dataset S t Those elements that are members of the dataset are denoted as “non-S t”, where X or a sub-dataset of X contains S t ). It should be understood that including S t All datasets of S can update the posterior probability of all its elements. t and non-S t It should also be understood that X or a subset of X (i.e., S t non-S t The cardinality of the PSI operation of Y is known (e.g., based on the PSI operation immediately preceding the most recent PSI operation) and is denoted as total . When observing the output O t After that, each element in the dataset non-S t The posterior probability of membership in (p[i]) is updated to (O total -O t ) / |non-S t |, where |non-S t | is the dataset non-S t That is, for non-S t This update is based on the known total positive count O total Subtract the observed positive count O t The remaining positive count after total -O t ). The posterior probability is calculated by dividing the remaining positive count by the non-S t See Figure 2B .

[0094] It should be understood that by updating the posterior probability based on the observed positive count (from the cardinality revealed in the most recent PSI operation) and the known total positive count (e.g., from the cardinality revealed in the PSI operation immediately before the most recent PSI operation), the evaluator can refine the posterior probability of the membership state of each element in the corresponding sub-dataset. The iterative posterior probability update process can enable the evaluator to incorporate new information and adjust the membership reasoning based on the observed results.

[0095] Regarding the stopping criteria of the evaluation process, it should be understood that in statistical evaluation, the evaluator can have the flexibility to enhance the efficiency of inference at the expense of some inference accuracy. This flexibility can be achieved by using a stopping criterion that involves setting an upper threshold θ u and the lower threshold θ l If the posterior probability of an element is greater than or equal to θ u , then the evaluator can classify the element's membership as positive, and if it is less than or equal to θ l, then it is classified as negative. It should also be understood that the choice of thresholds (upper and lower) may play a crucial role in determining the performance of the evaluation process. The thresholds can significantly affect the trade-off between inference accuracy and the number of inferred element memberships.

[0096] For example, if the upper threshold θ u is set close to 1, and the lower threshold θ l If θ is set close to 0, the inference evaluation can achieve high accuracy, but the number of element memberships inferred within the constraint (the maximum allowable number of calls to the PSI operation) may be limited. This is because the evaluator may need a high level of certainty before classifying an element as a positive or negative member. Therefore, only elements with posterior probabilities close to extreme values ​​can be confidently classified, which may result in a reduced number of identified elements. On the other hand, if the upper threshold θ is set to u and the lower threshold θ l All are set to be close to 0.5, then the element membership of larger proportion can be inferred.But, the reasoning accuracy may be affected.Utilize lower threshold value (for example, upper threshold value and lower threshold value both are close to 0.5), the element with relatively uncertain posterior probability can be classified, causes more number of elements to be identified.But, compare with using stricter threshold value (for example, upper threshold value is close to 1, and lower threshold value is close to 0), the accuracy of these classifications may be lower.

[0097] It should be understood that the features in the embodiments disclosed herein can quantify the privacy leakage of sets (or data sets) (such as member privacy leakage, etc.) for intersection size revealing protocols (e.g., PSI-CA, PSI-SUM, etc.). The features in the embodiments disclosed herein can also utilize both positive membership and negative membership elements to significantly enhance the efficiency of membership measurement. The features in the embodiments disclosed herein can provide a primary method for evaluating and solving set membership leakage by treating the membership of each individual as a random variable. The features in the embodiments disclosed herein can measure set membership leakage in the context of privacy-preserving protocols, including those protected by, for example, differential privacy protocols or algorithms (e.g., where noise is introduced into (multiple) data sets and / or result intersections). The features in the embodiments disclosed herein can incorporate active learning and Bayesian reasoning (including Bayesian posterior updates) and provide a dynamic and robust method for evaluating the effectiveness of privacy-preserving protocols.

[0098] Figure 6 is arranged according to at least some embodiments described herein and is suitable for implementing an electronic device (e.g., Figure 1 Schematic diagram of the structure of an example computer system 600 (a terminal device in a server or terminal device shown). It should be understood that Figure 6The computer system shown is for illustration purposes only and does not limit the functionality and application of the embodiments described herein.

[0099] As shown, the computer system 600 may include a central processing unit (CPU) 605. The CPU 605 may perform various operations and processes based on a program stored in a read-only memory (ROM) 610 or a program loaded from a storage device 640 to a random access memory (RAM) 615. The RAM 615 may also store various data and programs required for the operation of the system 600. The CPU 605, the ROM 610, and the RAM 615 may be connected to each other via a bus 620. An input / output (I / O) interface 625 may also be connected to the bus 620.

[0100] Components connected to the I / O interface 625 may also include an input device 630 including a keyboard, a mouse, a digital pen, a drawing tablet, etc.; an output device 635 including a display such as a liquid crystal display (LCD), a speaker, etc.; a storage device 640 including a hard disk, etc.; and a communication device 645 including a network interface card such as a LAN card, a modem, etc. The communication device 645 may perform communication processing via a network such as the Internet, a WAN, a LAN, a LIN, a cloud, etc. In an embodiment, a drive 650 may also be connected to the I / O interface 625. Removable media 655 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. may be installed on the drive 650 as needed, so that a computer program read from the removable media 655 may be installed in the storage device 640.

[0101] It should be understood that reference Figures 3 to 5 The process described in the flowchart and / or the process described in other figures can be implemented as a computer software program or hardware. The computer program product may include a computer program stored in a computer-readable non-volatile medium. The computer program includes program code for executing the method and / or GUI shown in the flowchart. In this embodiment, the computer program can be downloaded and installed from the network via a communication device 645, and / or installed from a removable medium 655. The computer program, when executed by a central processing unit (CPU) 605, can implement the above-mentioned functions specified in the method in the embodiment described herein.

[0102] It should be understood that the solutions, examples, embodiments, modules and functional operations disclosed herein can be implemented in digital electronic circuit systems, or in computer software, firmware or hardware, including the structures disclosed herein and their structural equivalents, or in a combination of one or more of them. The embodiments disclosed herein and other embodiments can be implemented as one or more computer program products, that is, one or more computer program instruction modules encoded on a computer-readable medium for execution by a data processing device or controlling the operation of the data processing device. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter that affects a machine-readable propagation signal, or a combination of one or more of them. The term "data processing device" covers all devices, equipment and machines for processing data, including, by example, a programmable processor, a computer or multiple processors or computers. In addition to hardware, the device may also include code that creates an execution environment for the computer program in question, for example, code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0103] A computer program (also referred to as a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions). A computer program may be deployed to execute on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

[0104] The processes and logic flows described in this document may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by, and apparatus may be implemented as, special purpose logic circuitry (e.g., field programmable gate arrays, application specific integrated circuits, etc.).

[0105] Processors suitable for executing computer programs include, by way of example, both general and special purpose microprocessors, and any one or more processors of any type of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include or be operably coupled to receive data from or send data to or both of one or more mass storage devices (e.g., magnetic disks, magneto-optical disks, or optical disks) for storing data. However, a computer need not have such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, by way of example, semiconductor memory devices, such as erasable programmable read-only memory, electrically erasable programmable read-only memory, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and optical disk read-only memory and digital video disk read-only memory disks. The processor and memory may be supplemented or incorporated into a dedicated logic circuit system.

[0106] It should be understood that different features, variations, and multiple different embodiments have been shown and described in various details. The content that has sometimes been described in this application with specific embodiments is for illustrative purposes only and is not intended to limit or imply that only one specific embodiment or specific embodiments are envisioned. It should be understood that the present disclosure is not limited to any single specific embodiment or enumerated variations. Those skilled in the art will think of many modifications, variations, and other embodiments that are intended to and are actually covered by the present disclosure. In fact, the scope of the present disclosure should be determined by the appropriate legal interpretation and construction (including equivalents) of the present disclosure as understood by those skilled in the art based on the complete disclosure that exists at the time of submission.

[0107] aspect:

[0108] It can be understood that any aspects can be combined with each other.

[0109] Aspect 1. A method for evaluating privacy protection, the method comprising: determining a first sub-dataset of a first data set based on a sampling rate, a tolerance and a first threshold; determining a second sub-dataset of the first data set based on the sampling rate, the tolerance and a second threshold; determining a first distance between the first sub-dataset and the first threshold; determining a second distance between the second sub-dataset and the second threshold; when the first distance is less than or equal to the second distance, generating a first intersection of the first sub-dataset and the second data set and updating a posteriori for elements of the first data set based on the first intersection; and determining positive membership and negative membership of elements of the first data set in the second data set based on the posteriori for elements of the first data set.

[0110] Aspect 2. The method of aspect 1 further includes: evaluating the privacy protection protocol based on the number of positive memberships and the number of negative memberships of the elements of the first data set in the second data set.

[0111] Aspect 3. The method of aspect 1 or aspect 2 further includes: when the first distance is greater than the second distance, generating a second intersection of the second sub-dataset and the second data set, and updating the posteriori for the elements of the first data set based on the second intersection.

[0112] Aspect 4. The method of Aspect 3, wherein updating the posteriori for the elements of the first data set based on the second intersection comprises: updating the posteriori for the elements of the second sub-data set based on a ratio of a size of the second intersection to a size of the second sub-data set.

[0113] Aspect 5. A method according to any one of Aspects 1 to 4, wherein the determination of positive membership and negative membership of an element of the first data set in a second data set based on a posteriori for the elements of the first data set comprises: determining the positive membership of the elements of the first data set in the second data set based on a first threshold and a posteriori for the elements of the first data set; and determining the negative membership of the elements of the first data set in the second data set based on a second threshold and a posteriori for the elements of the first data set.

[0114] Aspect 6. The method of any one of Aspects 1 to 5 further includes: initializing the posteriori for the elements of the first data set using random variables.

[0115] Aspect 7. The method of any one of Aspects 1 to 6, wherein updating the posteriori for the elements of the first data set based on the first intersection comprises: updating the posteriori for the elements of the first sub-data set based on a ratio of a size of the first intersection to a size of the first sub-data set.

[0116] Aspect 8. A privacy protection assessment system, the system comprising: a memory for storing a first data set, a sampling rate, a tolerance, a first threshold and a second threshold; a processor for: determining a first sub-dataset of the first data set based on the sampling rate, the tolerance and the first threshold; determining a second sub-dataset of the first data set based on the sampling rate, the tolerance and the second threshold; determining a first distance between the first sub-dataset and the first threshold; determining a second distance between the second sub-dataset and the second threshold; when the first distance is less than or equal to the second distance, generating a first intersection of the first sub-dataset and the second data set and updating a posteriori for elements of the first data set based on the first intersection; and determining positive membership and negative membership of elements of the first data set in the second data set based on the posteriori for elements of the first data set.

[0117] Aspect 9. The system of aspect 8, wherein the processor is further configured to: evaluate the privacy preservation protocol based on the number of positive memberships and the number of negative memberships of the elements of the first data set in the second data set.

[0118] Aspect 10. The system of aspect 8 or aspect 9, wherein the processor is further configured to: when the first distance is greater than the second distance, generate a second intersection of the second sub-dataset and the second data set, and update the posteriori for the elements of the first data set based on the second intersection.

[0119] Aspect 11. The system of aspect 10, wherein the processor is further configured to: update the posteriori for the elements of the second sub-dataset based on a ratio of the size of the second intersection to the size of the second sub-dataset.

[0120] Aspect 12. A system of any one of Aspects 8 to 11, wherein the processor is further used to: determine positive membership of an element of the first data set in a second data set based on a first threshold and a posteriori for the elements of the first data set; and determine negative membership of an element of the first data set in the second data set based on a second threshold and a posteriori for the elements of the first data set.

[0121] Aspect 13. The system of any one of Aspects 8 to 12, wherein the processor is further configured to: initialize the posteriori for the elements of the first data set using random variables.

[0122] Aspect 14. The system of any one of aspects 8 to 13, wherein the processor is further configured to: update the posteriori for the elements of the first sub-dataset based on a ratio of a size of the first intersection to a size of the first sub-dataset.

[0123] Aspect 15. A non-transitory computer-readable medium having computer executable instructions stored thereon, which when executed cause one or more processors to perform operations, the operations comprising: determining a first sub-dataset of a first data set based on a sampling rate, a tolerance, and a first threshold; determining a second sub-dataset of the first data set based on the sampling rate, the tolerance, and a second threshold; determining a first distance between the first sub-dataset and the first threshold; determining a second distance between the second sub-dataset and the second threshold; when the first distance is less than or equal to the second distance, generating a first intersection of the first sub-dataset and the second data set, and updating a posteriori for elements of the first data set based on the first intersection; and determining positive membership and negative membership of elements of the first data set in the second data set based on the posteriori for elements of the first data set.

[0124] Aspect 16. The computer-readable medium of aspect 15, wherein the operations further comprise: evaluating the privacy preservation protocol based on the number of positive memberships and the number of negative memberships of the elements of the first data set in the second data set.

[0125] Aspect 17. The computer-readable medium of aspect 15 or aspect 16, wherein the operation further comprises: when the first distance is greater than the second distance, generating a second intersection of the second sub-dataset and the second data set, and updating the posteriori for the elements of the first data set based on the second intersection.

[0126] Aspect 18. The computer-readable medium of aspect 17, wherein updating the posteriori for the elements of the first data set based on the second intersection comprises: updating the posteriori for the elements of the second sub-data set based on a ratio of a size of the second intersection to a size of the second sub-data set.

[0127] Aspect 19. A computer-readable medium of any one of Aspects 15 to 18, wherein the determination of positive membership and negative membership of an element of a first data set in a second data set based on a posteriori for the elements of the first data set comprises: determining the positive membership of an element of the first data set in the second data set based on a first threshold and a posteriori for the elements of the first data set; and determining the negative membership of an element of the first data set in the second data set based on a second threshold and a posteriori for the elements of the first data set.

[0128] Aspect 20. The computer-readable medium of any one of Aspects 15 to 19, wherein updating the posteriori for the elements of the first data set based on the first intersection comprises updating the posteriori for the elements of the first sub-data set based on a ratio of a size of the first intersection to a size of the first sub-data set.

[0129] The terms used in this specification are intended to describe specific embodiments and are not intended to be limiting. Unless expressly stated otherwise, the terms "a", "an" and "the" also include plural forms. When the terms "comprises" and / or "comprising" are used in this specification, the presence of the features, integers, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, integers, steps, operations, elements and / or components is not excluded.

[0130] With respect to the foregoing description, it should be understood that changes may be made in detail, particularly in the construction materials employed and the shapes, sizes and arrangements of parts without departing from the scope of the present disclosure. This specification and the described embodiments are merely exemplary, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Claims

1. A method for evaluating privacy protection, the method comprising: Determine a first sub-dataset of the first data set based on the sampling rate, the tolerance and the first threshold; Determine a second sub-dataset of the first data set based on the sampling rate, the tolerance and a second threshold; determining a first distance between the first sub-dataset and the first threshold; determining a second distance between the second sub-dataset and the second threshold; When the first distance is less than or equal to the second distance, generating a first intersection of the first sub-dataset and the second data set, and updating the posterior for the elements of the first data set based on the first intersection; as well as Based on the posteriori for the element of the first data set, positive membership and negative membership in the second data set for the element of the first data set are determined.

2. The method according to claim 1, further comprising: A privacy preserving protocol is evaluated based on the number of positive memberships and the number of negative memberships in the second data set for the element of the first data set.

3. The method according to claim 1, further comprising: When the first distance is greater than the second distance, a second intersection of the second sub-dataset and the second data set is generated, and the posterior for the element of the first data set is updated based on the second intersection.

4. The method of claim 3, wherein the updating of the posteriori for the elements of the first data set based on the second intersection comprises: Based on a ratio of a size of the second intersection set to a size of the second sub-dataset, a posteriori for elements of the second sub-dataset is updated.

5. The method of claim 1 , wherein the determination of positive membership and negative membership of the element of the first data set in the second data set based on the a posteriori for the element of the first data set comprises: determining positive membership of the element of the first data set in the second data set based on the first threshold and the posterior for the element of the first data set; as well as Based on the second threshold and the posterior for the element of the first data set, a negative membership of the element of the first data set in the second data set is determined.

6. The method according to claim 1, further comprising: The posterior for the elements of the first data set is initialized using a random variable.

7. The method of claim 1 , wherein the updating of the posteriori for the elements of the first data set based on the first intersection comprises: Based on a ratio of a size of the first intersection to a size of the first sub-dataset, a posteriori for the elements of the first sub-dataset is updated.

8. A privacy protection evaluation system, the system comprising: A memory, for storing a first data set, a sampling rate, a tolerance, a first threshold, and a second threshold; A processor for: Determine a first sub-dataset of the first data set based on the sampling rate, the tolerance and the first threshold; Determine a second sub-dataset of the first data set based on the sampling rate, the tolerance and the second threshold; determining a first distance between the first sub-dataset and the first threshold; determining a second distance between the second sub-dataset and the second threshold; When the first distance is less than or equal to the second distance, generating a first intersection of the first sub-dataset and the second data set, and updating the posterior for the elements of the first data set based on the first intersection; as well as Based on the posteriori for the element of the first data set, positive membership and negative membership in the second data set for the element of the first data set are determined.

9. The system of claim 8, wherein the processor is further configured to: A privacy preserving protocol is evaluated based on the number of positive memberships and the number of negative memberships in the second data set for the element of the first data set.

10. The system of claim 8, wherein the processor is further configured to: When the first distance is greater than the second distance, a second intersection of the second sub-dataset and the second data set is generated, and the posterior for the element of the first data set is updated based on the second intersection.

11. The system of claim 10, wherein the processor is further configured to: Based on a ratio of a size of the second intersection set to a size of the second sub-dataset, a posteriori for elements of the second sub-dataset is updated.

12. The system of claim 8, wherein the processor is further configured to: determining positive membership of the element of the first data set in the second data set based on the first threshold and the posterior for the element of the first data set; and Based on the second threshold and the posterior for the element of the first data set, a negative membership of the element of the first data set in the second data set is determined.

13. The system of claim 8, wherein the processor is further configured to: The posterior for the elements of the first data set is initialized using a random variable.

14. The system of claim 8, wherein the processor is further configured to: Based on a ratio of a size of the first intersection to a size of the first sub-dataset, a posteriori for the elements of the first sub-dataset is updated.

15. A non-transitory computer readable medium having computer executable instructions stored thereon, the computer executable instructions, when executed, causing one or more processors to perform operations comprising: Determine a first sub-dataset of the first data set based on the sampling rate, the tolerance and the first threshold; Determine a second sub-dataset of the first data set based on the sampling rate, the tolerance and a second threshold; determining a first distance between the first sub-dataset and the first threshold; determining a second distance between the second sub-dataset and the second threshold; When the first distance is less than or equal to the second distance, generating a first intersection of the first sub-dataset and the second data set, and updating the posterior for the elements of the first data set based on the first intersection; as well as Based on the posteriori for the element of the first data set, positive membership and negative membership in the second data set for the element of the first data set are determined.

16. The computer-readable medium of claim 15, wherein the operations further comprise: A privacy preserving protocol is evaluated based on the number of positive memberships and the number of negative memberships in the second data set for the element of the first data set.

17. The computer-readable medium of claim 15, wherein the operations further comprise: When the first distance is greater than the second distance, a second intersection of the second sub-dataset and the second data set is generated, and the posterior for the element of the first data set is updated based on the second intersection.

18. The computer-readable medium of claim 17, wherein the updating of the posteriori for the elements of the first data set based on the second intersection comprises: Based on a ratio of a size of the second intersection set to a size of the second sub-dataset, a posteriori for elements of the second sub-dataset is updated.

19. The computer-readable medium of claim 15, wherein the determination of positive membership and negative membership of the element of the first data set in the second data set based on the a posteriori for the element of the first data set comprises: determining positive membership of the element of the first data set in the second data set based on the first threshold and the posterior for the element of the first data set; as well as Based on the second threshold and the posterior for the element of the first data set, a negative membership of the element of the first data set in the second data set is determined.

20. The computer-readable medium of claim 15, wherein the updating of the posteriori for the elements of the first data set based on the first intersection comprises: Based on a ratio of a size of the first intersection to a size of the first sub-dataset, a posteriori for the elements of the first sub-dataset is updated.

Images