Efficient unbalanced psi based on bloom filter and hash

Abstract

The application provides an efficient unbalanced PSI based on a Bloom filter and a hash, wherein in a PSI calculation process, a Bloom filter is used to perform one round of screening on a large data set to reduce the overall complexity; a step of sending Bloom filter parameters by a large data set party to a small data set party; a step of calculating, by the small data set party, an index set of bit positions required to be set to 1 in the Bloom filter according to the data set of the small data set party, and sending the index set to the large data set party; a step of initializing the Bloom filter by the large data set party according to the Bloom filter parameters and the index set of the small data set party; a step of screening, by the large data set party, using the Bloom filter to obtain an element set that may be in a private intersection in the data set of the large data set party, wherein the screened set is used as a new private data set for subsequent calculation of the private intersection; and compared with an original PSI based on a hash and a semi-honest third party, the efficiency is greatly improved in an unbalanced scenario.

Description

TECHNICAL FIELD

[0001] The present application relates to a high-efficiency unbalanced PSI based on Bloom filter and hash. BACKGROUND

[0002] PSI is a kind of multi-party computation, and also a kind of multi-party computation scenario that has been studied very deeply. The earliest PSI algorithm based on public key modular exponentiation calculation consumes a lot, and is not practical. With the increasing performance of computers, related algorithms are constantly improving and developing, and the performance has been greatly improved. There has been practical application in real life. For example, the application of personal password leakage in Microsoft Edge browser uses PSI algorithm based on homomorphic encryption. In addition, in social networks, it can also be used to protect the complete friend information of both parties from being leaked when comparing common friends between two people. In addition, PSI can also be used in botnet discovery, similarity detection, gene detection, etc., and has great potential value in practical application.

[0003] The recently rapidly developed federated machine learning to protect user privacy also uses PSI algorithm to align the training samples between the federated learning participants.

[0004] Private Set Intersection (PSI) involves the following concepts:

[0005] 1. Hash algorithm

[0006] Also known as a hash function, it is a function that transforms any length of input into a fixed length output. Hash algorithms can generally be used to improve the utilization of storage space and improve data query efficiency. In cryptography, hash functions can also be used for digital signatures and unique digests of data to ensure the security of data transmission. Cryptographic hash functions are irreversible, that is, the output can be easily calculated from the input, but the input cannot be calculated from the output. In addition, hash functions have the property of determinism, that is, when the input is determined, the output is also determined.

[0007] 2. PSI

[0008] PSI is the abbreviation of private set intersection, that is, two or more entities each have a private data set. These entities want to calculate the common intersection between all entities, but do not want to reveal the content of their respective private sets to each other. PSI is a technology that realizes this kind of demand, and there are many ways to realize PSI, which can be based on public key encryption or inadvertent transmission, etc.

[0009] PSI belongs to a kind of multi-party computation, very promising applications, such as privacy contact discovery, genetic testing, etc. PSI is used between two or more parties, each party has a private data set, and these parties want to calculate the common intersection between all parties, but do not want to reveal the intersection to others. There are many ways to achieve PSI, which can be based on hash, public key encryption, homomorphic encryption, garbled circuit, or oblivious transfer, etc.

[0010] In the application of PSI, there are balanced PSI and unbalanced PSI:

[0011] In balanced PSI, the two parties participating in the calculation have private data sets with similar sizes.

[0012] In unbalanced PSI, the private data sets of the participants have different data set sizes, often with a hundred times difference in size.

[0013] 3. Bloom filter

[0014] Bloom filter (Bloom Filter) was proposed by Bloom in 1970. It consists of a long bit vector and a series of random mapping functions, which is used to check whether an element is in a set. The main advantages are that the space efficiency and query time are much better than general algorithms, and it can be used to save communication overhead when transmitting data over the network.

[0015] A Bloom filter of a data set is initialized by a set of hash functions and the specific element values of the data set. This set of hash functions can also be used to detect whether an element is in the data set. When detecting, check whether the bit value of the bit vector position of the index position generated by the hash of the element is all 1.

[0016] 4. Semi-honest security model

[0017] In multi-party computation, a common multi-party secure computation security model, in the semi-honest model, the attacker will strictly follow the multi-party computation protocol flow, but it will be curious about the privacy data of other participants, trying to restore the privacy data of other participants through the received intermediate data.

[0018] Currently, the existing technology of private set intersection (PSI):

[0019] (1) Hash-based PSI: By calculating the hash value of both parties' original data, both parties exchange and compare, and further obtain the private intersection. This method is relatively simple and direct, and the efficiency is the fastest. But there is a security problem, because the plaintext space used for intersection matching is often limited, such as ID card, mobile phone number, etc. Through the hash value of the other party's private data, the interactive parties can easily deduce the original data through exhaustive attack.

[0020] The security problem of hash-based PSI can be solved by introducing a semi-honest third party. By negotiating a shared key between the parties involved in PSI as the key of HMAC, and calculating the hmac hash value, the third party is asked to calculate the PSI hash value and return it to the parties to restore the final PSI. Since the third party cannot know the HMAC key, it cannot attack by exhaustive attack.

[0021] Under the semi-honest security model, although the introduction of a third party based on hash-based PSI can solve the security problem while retaining the efficiency of hash-based PSI, in the case of imbalance, the computational and communication complexity is still linearly related to the size of the large data set. For participants with smaller data sets, the overall efficiency is still low.

[0022] (2) Public key-based PSI: Early PSI algorithms are generally based on popular public key cryptography algorithms such as RSA and discrete logarithm. However, this type of algorithm often involves large integer modular exponentiation, so it is usually slow and inefficient when the data sets of both parties are large. In summary, a large number of public key modular exponentiation calculations consume a lot of computing resources and are very slow.

[0023] (3) Based on OT: This type of algorithm is based on OT (Oblivious Transfer), which can construct a one-to-many efficient OT group with a small amount of public key calculation and symmetric encryption calculation, and perform efficient PSI algorithm.

[0024] (4) Based on garbled circuit: Garbled circuit was first proposed by Yao to solve the millionaire problem and is a general secure multi-party computation framework that can be used to convert any computation into a Boolean computation and then run secure multi-party computation. Its performance has improved significantly in recent years, and it can convert PSI calculation into garbled circuit secure multi-party computation.

[0025] (5) Based on homomorphic encryption: Microsoft uses this technology to solve the problem of password leakage checking in scenarios where the size of the data sets of both parties differs greatly. This type of PSI technology converts PSI calculation into polynomial calculation and further uses homomorphic encryption to protect the privacy of the data sets of both parties.

[0026] Although the PSI of the latter three methods reduces the public key calculation, the performance is greatly improved compared with the public key calculation-based PSI, and there is an efficient scheme for privacy intersection seeking in unbalanced scenarios, but the performance still cannot match the hash-based PSI method.

[0027] On the other hand, the current hash PSI based on a semi-honest third party requires the party with a large data set to calculate and transmit the hash values of all elements in the large set, and the overall calculation complexity and communication complexity are linearly related to the size of the large data set. Meanwhile, the calculation complexity of the third party in calculating the hash values of the two parties is also linearly related to the size of the large data set. SUMMARY

[0028] In order to optimize the PSI efficiency problem based on hash and semi-honest third party in unbalanced scenarios, the application provides a high-efficiency unbalanced PSI based on Bloom filter and hash.

[0029] In order to achieve the above-mentioned purpose, the application provides a high-efficiency unbalanced PSI based on Bloom filter and hash, which comprises a B party with a large data set and an A party with a small data set; the PSI calculation method comprises the following steps:

[0030] S1, the B party sends Bloom filter parameters to the A party;

[0031] S2, the A party calculates an index set of bit positions that need to be set to 1 in the Bloom filter according to its own data set, and sends the index set to the B party;

[0032] S3, the B party initializes the Bloom filter according to the Bloom filter parameters and the index set of the B party;

[0033] S4, the B party uses the Bloom filter for screening, and obtains a set of elements in its own data set that may be in the private intersection, and this screened set is used as a new private data set for subsequent private intersection seeking;

[0034] S5, the A party and the B party each calculate the hash value of each data in their own private data set after adding a salt value by using a hash algorithm and save the hash value;

[0035] S6, the A party and the B party each transmit the saved hash value to a semi-honest third party;

[0036] S7, the semi-honest third party compares the hash values of the two parties, screens out hash values that are equal from the hash value sets of the two parties, and sends the screened hash values that are equal to the two parties participating in the PSI calculation;

[0037] S8, A party and B party receive the third party sent filtered value equal to the hash value, respectively with the local saved hash value do a comparison, with the third party forwarding the hash value equal to the local hash value of the original data output, as the final output of the PSI algorithm.

[0038] Further, the above-mentioned efficient unbalanced PSI based on Bloom filter and hash includes the following preparation steps before step S1:

[0039] B party according to its own data set size, Bloom filter fault tolerance, initialization of Bloom filter parameters.

[0040] Further, the above-mentioned efficient unbalanced PSI based on Bloom filter and hash includes:

[0041] The Bloom filter fault tolerance is set to 0.25.

[0042] Further, the above-mentioned efficient unbalanced PSI based on Bloom filter and hash includes: the Bloom filter parameters include Bloom filter length, hash function set.

[0043] Further, the above-mentioned efficient unbalanced PSI based on Bloom filter and hash includes: in step S5, the salt value is generated by the following steps:

[0044] A party and B party generate a shared random key through key exchange protocol, as the salt value of hash calculation.

[0045] Further, the above-mentioned efficient unbalanced PSI based on Bloom filter and hash includes: A party and B party negotiate to use Diffie-Hellman for key exchange.

[0046] Further, the above-mentioned efficient unbalanced PSI based on Bloom filter and hash includes: the parameters used by A party and B party when performing key exchange are: 2048 bit MODP group with 224 bit prime number order subgroup.

[0047] Further, the above-mentioned efficient unbalanced PSI based on Bloom filter and hash includes: in step S5:

[0048] If one of A party or B party does not require to obtain PSI, the semi-honest third party can not send these filtered hash values to the party which does not require to obtain PSI.

[0049] In the present application, a bloom filter is used to perform a round of screening on a large data set to reduce the overall complexity. After screening, the communication complexity is greatly reduced, and the calculation complexity of the third party is also reduced. After reduction, the complexity is linearly related to the size of the small data set. The party with a large data set only needs to transmit the bloom filter parameters to the party with a small data set, and the party with a small data set only needs to transmit the bloom filter index to the party with a large data set. In terms of calculation complexity, the bloom filter index is mainly calculated on the party with a large data set. The efficiency of the PSI based on hash and semi-honest third party is greatly improved.

[0050] The present application will be further described below in combination with the drawings and specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 The flow chart of the high-efficiency unbalanced PSI based on the bloom filter and hash in the embodiments of the present application. EMBODIMENT

[0052] Embodiment 1: The present embodiment is a high-efficiency unbalanced PSI based on the bloom filter and hash. In the present embodiment, the party with a large data set is the A party and the party with a small data set is the B party.

[0053] As shown in the following specific steps: Figure 1

[0054] (1) Let the two parties participating in the PSI calculation be the A party and the B party, and the semi-honest third party be the C party. Among them, the B party has a larger data set SET B , and the A party has a smaller data set SET A .

[0055] (2) The A party and the B party negotiate to use Diffie-Hellman for key exchange, and the parameters of the algorithm are (2048-bit MODP Group with 224-bit Prime Order Subgroup). The subgroup generator in the algorithm is represented as g, and the parameter description specific content can be found in some network articles such as the link: https: / / tools.ietf.org / html / rfc5114. In the present embodiment, the key exchange algorithm here can be any secure standard key exchange algorithm.

[0056] (3) The A party generates a 160-bit random number a, the B party generates a 160-bit random number b, the A party calculates and sends g a to the B party, and the B party calculates and sends g b to the A party.

[0057] (4) The A party receives the data g b ​Further calculation results in a shared key ss = (g b ) a = g ab , B also performs similar calculations to obtain a shared key ss = (g a ) b = g ab .

[0058] (5) B initializes the Bloom filter parameters, including the Bloom filter length m, the hash function set h1, h2, h3, …, according to the size of SET B and the Bloom filter error tolerance p, such as p = 0.25. The generation of the hash function set can be a secure hash function with different random salt values, such as h1 is SHA256, and the salt value is 36787. In fact, p can be set to a larger value, which can reduce the amount of data transmitted in communication.

[0059] (6) B sends the Bloom filter parameters {m, h1, h2, h3, …} to A.

[0060] (7) A, after obtaining the Bloom filter parameters, calculates the index set SET index of the bit positions that need to be set to 1 in the Bloom filter for all elements in SET A . For example, if there is an element x in the data set, calculate i1 = h1(x), i2 = h2(x), i3 = h3(x), …, and add i1, i2, i3 to the index set. Finally, SET index is the Bloom filter index set of SET A .

[0061] (8) A sends the index set SET index to B.

[0062] (9) B initializes the Bloom filter BF according to the Bloom filter parameters {m, h1, h2, h3, …} and SET index .

[0063] (10) B uses BF to screen SET B and obtains a set of elements that may be in the private intersection SET C . The screened SET C is used as the new private data set for subsequent private intersection calculation.

[0064] (11) Assuming that A's SET A = {x}, A calculates the hash value hash x = SHA-256(x + ss) of SET A and sends {hash x{hash} is sent to C. Note: SHA256 is the selected hash function, which can be any secure standard hash function.

[0065] (12) Assuming B filters the SET C {y}, B computes the hash value hash B =SHA-256(y+ss) of the SET y and sends the set {hash y} to C. Note: SHA256 is the selected hash function, which must be consistent with the hash algorithm selected by A.

[0066] (13) After C receives the hash values sent by A and B, it compares them and outputs the hash values that exist in both sets: {hash z}, if hash z ⊂{hash x} and hash z ⊂{hash y}. C sends {hash z} to A and B. Note: If A or B does not require the final PSI, C can not send {hash z} to this party.

[0067] (14) After A receives {hash z}, it compares it with {hash x} calculated in (11). For all hash values that exist in {hash z}, i.e. hash x’ =hash z , output its original {x'} as the final PSI output.

[0068] (15) After B receives {hash z}, it compares it with {hash y} calculated in (12). For all hash values that exist in {hash z}, i.e. hash y’ =hash z , output its original {y'} as the final PSI output.

Claims

1. An efficient unbalanced PSI based on Bloom filter and hash, comprising a B party with a large data set and an A party with a small data set; characterized in that: The PSI calculation method comprises the following steps: S1, the B party sends the Bloom filter parameter to the A party; S2, the A party calculates the index set of the data set in the Bloom filter which needs to be set to 1 according to the data set of the A party, and sends the index set to the B party; S3, the B party initializes the Bloom filter according to the Bloom filter parameter and the index set of the A party; S4, the B party uses the Bloom filter to screen, and obtains an element set which is possibly in the privacy intersection set in the data set of the B party. The screened set is used as a new privacy data set for subsequent privacy intersection calculation; S5, the A party and the B party each calculate the hash value of each data in the privacy data set of the A party and the B party after adding a salt value by using a hash algorithm and save the hash value; S6, the A party and the B party each transmit the saved hash value to a semi-honest third party; S7, the semi-honest third party compares the hash values of the A party and the B party, screens the hash values which are equal from the hash value set of the A party and the B party, and sends the screened hash values which are equal to the A party and the B party participating in the PSI calculation; S8, after receiving the screened hash values which are equal sent by the third party, the A party and the B party compare the saved hash values respectively, output the original data corresponding to the hash values which are equal to the hash values forwarded by the third party as the final output of the PSI algorithm.

2. The Bloom filter and hash based efficient unbalanced PSI according to claim 1, wherein: Before step S1, the following preparation steps are further included: The B party initializes the Bloom filter parameter according to the size of the data set of the B party and the Bloom filter fault tolerance.

3. The high-efficiency unbalanced PSI based on the Bloom filter and the hash according to claim 2, wherein: The Bloom filter fault tolerance is less than 0.

25.

4. The Bloom filter and hash based efficient unbalanced PSI according to claim 2, wherein: The Bloom filter parameter comprises a Bloom filter length and a hash function set.

5. The Bloom filter and hash based efficient unbalanced PSI according to claim 1, wherein: In step S5, the salt value is generated by the following steps: The A party and the B party generate a shared random key by using a key exchange protocol as the hash calculation salt value.

6. The Bloom filter and hash based efficient unbalanced PSI according to claim 5, wherein: The A party and the B party negotiate to use Diffie-Hellman for key exchange.

7. The Bloom filter and hash based efficient unbalanced PSI according to claim 6, wherein: The parameters used by the A party and the B party in the key exchange are a 2048-bit MODP group with a 224-bit prime order subgroup.

8. The Bloom filter and hash based efficient unbalanced PSI according to claim 1, wherein: In step S5: If one of the A party and the B party does not require to obtain the PSI, the semi-honest third party can not send the screened hash values to the party which does not require to obtain the PSI.

Images