Privacy intersection method, device and equipment for big data and storage medium

Abstract

The invention provides a big data-oriented privacy intersection method, apparatus and device, and a storage medium. The method provided by the invention comprises the following steps: performing key negotiation to obtain a first key, and obtaining a first bucket hash function; calling a first bucket hash function for elements in the first data set, and calling the first bucket hash function for elements in the second data set to obtain an intersection estimated size; if the estimated intersection size is not greater than the lower limit of the expected intersection size of the receiver, exiting the protocol; otherwise, performing key negotiation again to obtain a second key, and obtaining a second bucket hash function; and calling a protocol executed by the sender in the privacy intersection protocol by using the fifth data set, and calling a protocol executed by the receiver in the privacy intersection protocol by using the sixth data set to obtain an intersection result. According to the method, waste of computing power resources and memory resources can be avoided, and the privacy intersection task between large data volumes can be executed in a small memory space.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a privacy protection method, device, equipment and storage medium for big data. Background Art

[0002] Medical data includes patient data of medical institutions, electronic medical records, imaging data, laboratory reports, etc. With the rapid development of informatization, today's medical data has gradually acquired the characteristics of big data. Because medical data contains personal sensitive information, due to the constraints of regulations and laws, medical data is not allowed to circulate at will. The privacy intersection protocol can obtain the intersection of all sets without leaking the participating data sets. For example, before joint modeling, the participants can use a certain feature to intersect and then use the intersecting data for modeling. In recent years, privacy intersection protocols have been booming. At present, there are many practical and high-performance privacy intersection protocols, such as ECDH-PSI, KKRT-PSI and VOLE-PSI. However, the existing privacy intersection schemes do not fully consider the big data scenario and cannot perform privacy intersection tasks between large amounts of data in a smaller memory space. For example, the terminal machine has only 64GB of memory, but has to perform privacy intersection tasks at the level of tens of billions, making the privacy intersection task impossible to achieve; in the joint modeling scenario, there are often requirements for the data size of the intersection. If the intersection is too small, the model effect will deteriorate or even fail to be modeled. However, the existing privacy intersection schemes cannot estimate the size of the intersection in advance, but require the complete execution of the protocol to know the size of the intersection, which may waste computing power and memory resources. Summary of the invention

[0003] The purpose of the present invention is to provide a privacy intersection method, device, equipment and storage medium for big data, so as to solve the technical problems in the prior art that the privacy intersection task between large amounts of data cannot be performed in a smaller memory space and the size of the intersection cannot be estimated in advance.

[0004] The technical solution of the present invention is as follows, which provides a privacy intersection method for big data, including:

[0005] Obtaining a first data set of a sender and a second data set of a receiver, causing the sender and the receiver to negotiate a key to obtain a first key, and obtaining a first bucket hash function according to the first key;

[0006] For each element in the first data set, the first bucket hash function is called respectively to form a third data set, for each element in the second data set, the first bucket hash function is called respectively to form a fourth data set, and an estimated size of the intersection is obtained according to the third data set and the fourth data set;

[0007] If the estimated intersection size is not greater than the lower limit of the intersection size expected by the receiver, the receiver and the sender are caused to exit the protocol;

[0008] If the estimated size of the intersection is greater than the lower limit of the size of the intersection expected by the receiver, the sender and the receiver re-negotiate the key to obtain a second key, and obtain a second bucket hash function according to the second key;

[0009] For each element in the first data set, the second bucket hash function is called respectively to form a fifth data set, so that the sender uses the fifth data set to call the protocol executed by the sender in the privacy intersection protocol. For each element in the second data set, the second bucket hash function is called respectively to form a sixth data set, so that the receiver uses the sixth data set to call the protocol executed by the receiver in the privacy intersection protocol to obtain an intersection result.

[0010] Further, obtaining a first bucket hash function according to the first key includes:

[0011] The first bucket number is obtained according to the size of the first data set, the size of the second data set and a bucket number calculation formula, and the first bucket hash function is obtained according to the first bucket number and the first key. The bucket number calculation formula includes

[0012]

[0013] Among them, B is the number of first buckets, is the size of the second data set, is the size of the first data set, w is the lower limit of the set size that the sender can run the privacy intersection protocol at a single time, It means round up.

[0014] Further, for each element in the first data set, the first bucket hash function is called respectively to form a third data set, and for each element in the second data set, the first bucket hash function is called respectively to form a fourth data set, including:

[0015] For each element in the first data set, the first bucket hash function is called respectively, and a third data set is formed with all elements in the first data set that satisfy the conditions that the calculation result of the first bucket hash function is 0. For each element in the second data set, the first bucket hash function is called respectively, and a fourth data set is formed with all elements in the second data set that satisfy the conditions that the calculation result of the first bucket hash function is 0.

[0016] Further, obtaining an estimated size of the intersection according to the third data set and the fourth data set includes:

[0017] For each element in the third data set, the mapping hash function is called respectively to form a seventh data set. For each element in the fourth data set, the mapping hash function is called respectively to form an eighth data set. According to the intersection of the seventh data set and the eighth data set, the estimated size of the intersection is obtained.

[0018] Further, for each element in the third data set, respectively call the mapping hash function to form a seventh data set, and for each element in the fourth data set, respectively call the mapping hash function to form an eighth data set, including:

[0019] For each element in the third data set, respectively call the mapping hash function to obtain a calculation result of the mapping hash function, obtain a first random number power of each calculation result of the mapping hash function, form a ninth data set with the first random number power of all calculation results of the mapping hash function, and calculate a second random number power for each element of the ninth data set to form a seventh data set;

[0020] For each element in the fourth data set, call the mapping hash function respectively to obtain the calculation result of the mapping hash function, obtain the third random number power of each calculation result of the mapping hash function, add the third random number power of the calculation results of all the mapping hash functions to form a tenth data set, and calculate the fourth random number power for each element of the tenth data set to form an eighth data set.

[0021] Further, obtaining an estimated size of the intersection according to the intersection of the seventh data set and the eighth data set includes:

[0022] According to the intersection of the seventh data set and the eighth data set, and the intersection estimation formula, the intersection estimation size is obtained, and the intersection estimation formula includes Among them, M is the estimated size of the intersection, is the size of the second data set, is the size of the tenth data set, and m′ is the size of the intersection of the seventh data set and the eighth data set.

[0023] Further, for each element in the first data set, respectively calling the second bucket hash function to form a fifth data set, including, for each element in the first data set, respectively calling the second bucket hash function to form a fifth data set with all elements in the first data set that satisfy the calculation result of the second bucket hash function to be i, where i∈{0,...,B'-1}, B' is the number of buckets of the second bucket hash function;

[0024] For each element in the second data set, the second bucket hash function is called respectively to form a sixth data set, including, for each element in the second data set, the second bucket hash function is called respectively to form a sixth data set with all elements in the second data set that satisfy the calculation result of the second bucket hash function being i.

[0025] Another technical solution of the present invention is as follows: a privacy intersection device for big data is provided, comprising a first bucket hash function acquisition module, an intersection size estimation module, a protocol exit module, a second bucket hash function acquisition module, and an intersection result acquisition module;

[0026] The first bucket hash function acquisition module is used to acquire a first data set of a sender and a second data set of a receiver, so that the sender and the receiver negotiate a key to obtain a first key, and acquire a first bucket hash function according to the first key;

[0027] The intersection size estimation module is used to call the first bucket hash function for each element in the first data set to form a third data set, call the first bucket hash function for each element in the second data set to form a fourth data set, and obtain the estimated size of the intersection according to the third data set and the fourth data set;

[0028] The protocol exit module is used to make the receiver and the sender exit the protocol if the estimated intersection size is not greater than the lower limit of the intersection size expected by the receiver;

[0029] The second bucket hash function acquisition module is used to enable the sender and the receiver to renegotiate the key if the estimated intersection size is greater than the lower limit of the intersection size expected by the receiver, to obtain a second key, and to obtain a second bucket hash function according to the second key;

[0030] The intersection result acquisition module is used to call the second bucket hash function for each element in the first data set to form a fifth data set, so that the sender uses the fifth data set to call the protocol executed by the sender in the privacy intersection protocol, and to call the second bucket hash function for each element in the second data set to form a sixth data set, so that the receiver uses the sixth data set to call the protocol executed by the receiver in the privacy intersection protocol to obtain an intersection result.

[0031] Another technical solution of the present invention is as follows: an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program executable by the processor, and when the processor executes the computer program, the privacy intersection method for big data as described in any of the above technical solutions is implemented.

[0032] Another technical solution of the present invention is as follows: a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the privacy intersection method for big data as described in any of the above technical solutions is implemented.

[0033] The beneficial effects of the present invention are: obtaining a first data set of a sender and a second data set of a receiver, allowing the sender and the receiver to negotiate a key to obtain a first key, and obtaining a first bucket hash function according to the first key; for each element in the first data set, calling the first bucket hash function respectively to form a third data set, for each element in the second data set, calling the first bucket hash function respectively to form a fourth data set, and obtaining an estimated intersection size according to the third data set and the fourth set; if the estimated intersection size is not greater than the lower limit of the intersection size expected by the receiver, causing the receiver and the sender to exit the protocol; if the estimated intersection size is greater than the lower limit of the intersection size expected by the receiver, causing the sender and the receiver to re-negotiate a key to obtain a second key, and obtaining a second bucket hash function according to the second key; for the first data set For each element in the combination, the second bucket hash function is called respectively to form a fifth data set, so that the sender uses the fifth data set to call the protocol executed by the sender in the privacy intersection protocol. For each element in the second data set, the second bucket hash function is called respectively to form a sixth data set, so that the receiver uses the sixth data set to call the protocol executed by the receiver in the privacy intersection protocol to obtain the intersection result. Through the above scheme, the intersection size can be estimated in advance to obtain the estimated intersection size. If the estimated intersection size is not greater than the lower limit of the intersection size expected by the receiver, the receiver and the sender exit the protocol to end the modeling in advance in the joint modeling scenario, thereby avoiding the waste of computing power resources and memory resources. If the estimated intersection size is greater than the lower limit of the intersection size expected by the receiver, the intersection result is obtained, which can realize the execution of the privacy intersection task between large data volumes in a smaller memory space. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A flowchart of a privacy-enforcing method for big data provided by an embodiment of the present invention;

[0035] Figure 2 A schematic diagram of the structure of a privacy-enforcing device for big data provided by an embodiment of the present invention;

[0036] Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] In the description of this application, the words "first", "second", etc. are only used for the purpose of distinguishing descriptions, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying an order. In this specification, the terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0039] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0040] Figure 1 is a flow chart of the privacy intersection method for big data according to an embodiment of the present invention. It should be noted that if there is substantially the same result, the privacy intersection method for big data according to the present invention is not based on Figure 1 The process sequence shown is limited. Figure 1 As shown, the privacy intersection method for big data mainly includes the following steps:

[0041] S101, obtaining a first data set of a sender and a second data set of a receiver, causing the sender and the receiver to negotiate a key to obtain a first key, and obtaining a first bucket hash function according to the first key;

[0042] It should be noted that the privacy intersection protocol calculates the intersection of all data sets without revealing the data sets owned by the participants. Without loss of generality, it can be considered that the privacy intersection protocol is run between two participants, namely the sender and the receiver. Among them, the sender only needs to provide its own data set, while the receiver will provide its own data set and obtain the intersection between it and the sender. The first data set of the sender and the second data set of the receiver mentioned above can be a collection of medical data, which can include patient data of medical institutions, electronic medical records, imaging data, laboratory reports, etc.

[0043] In an optional implementation, obtaining a first bucket hash function according to the first key includes:

[0044] The first bucket number is obtained according to the size of the first data set, the size of the second data set and a bucket number calculation formula, and the first bucket hash function is obtained according to the first bucket number and the first key. The bucket number calculation formula includes

[0045]

[0046] Among them, B is the number of first buckets, is the size of the second data set, is the size of the first data set, w is the lower limit of the set size that the sender can run the privacy intersection protocol at a single time, It means rounding up.

[0047] It should be noted that In the formula, x and y are both positive integers. In the bucket number calculation formula, x is y is w.

[0048] In some embodiments, without loss of generality, assume that the first data set of the sender is The size of the set of nodes that the sender can run the privacy intersection protocol on at a time is a positive integer n S , and w can be the lower limit of the set size that the sender can run the privacy intersection protocol at a time, for example, w = 10 5 , q is the upper limit of the set size that can be used by the sender to run the privacy intersection protocol at one time, for example, q = 10 8 , is the size of the first data set (i.e., the number of elements in the set). Without loss of generality, let the second data set of the receiver be is the size of the second data set, a positive integer m is the lower limit of the intersection size expected by the receiver, n R Indicates the size of the set that the receiver can run the privacy intersection protocol for at one time, a positive integer Among them, the parameters also need to satisfy the relationship t≤n R ≤10 8 ≤u, t can be the lower limit of the set size that the receiver can run the privacy intersection protocol for a single time, t and w can be equal, for example, t = 10 5 , u is the upper limit of the size of the set that can be used by the receiver to run the privacy intersection protocol at one time, u and q can be equal, for example, u = 10 8 .

[0049] In a specific embodiment, the sender sends and n S To the receiver, so that the receiver can send and n R To the sender, so that the sender receives and n R , so that the receiver receives and n S , so that the sender and the receiver can negotiate the key and obtain the first key According to the first bucket number B and the first key Get the first bucket hash function in, is the first key, the first bucket hash function Accept the first key e is an element in the first data set or an element in the second data set, which can be a string of any length, e∈{0,1} * , output b∈{0,...,B-1}.

[0050] S102, for each element in the first data set, respectively call the first bucket hash function to form a third data set, for each element in the second data set, respectively call the first bucket hash function to form a fourth data set, and obtain an estimated size of the intersection according to the third data set and the fourth data set;

[0051] In an optional embodiment, for each element in the first data set, the first bucket hash function is called respectively to form a third data set, and for each element in the second data set, the first bucket hash function is called respectively to form a fourth data set, including:

[0052] For each element in the first data set, the first bucket hash function is called respectively, and a third data set is formed with all elements in the first data set that satisfy the conditions that the calculation result of the first bucket hash function is 0. For each element in the second data set, the first bucket hash function is called respectively, and a fourth data set is formed with all elements in the second data set that satisfy the conditions that the calculation result of the first bucket hash function is 0.

[0053] In some embodiments, without loss of generality, let w = 10 5 , so that the sender calculates the first bucket number For the first data set For each element e1 in, call the first bucket hash function And satisfy all Add the elements of the third data set The third data set is formed Let t = 10 5 , so that the receiver calculates the first bucket number For the second data set For each element e2 in, call the first bucket hash function And satisfy all Add the elements of The fourth data set is formed

[0054] In an optional implementation, obtaining an estimated size of the intersection according to the third data set and the fourth data set includes:

[0055] For each element in the third data set, the mapping hash function is called respectively to form a seventh data set. For each element in the fourth data set, the mapping hash function is called respectively to form an eighth data set. According to the intersection of the seventh data set and the eighth data set, the estimated size of the intersection is obtained.

[0056] In an optional embodiment, for each element in the third data set, respectively calling the mapping hash function to form a seventh data set, and for each element in the fourth data set, respectively calling the mapping hash function to form an eighth data set, including:

[0057] For each element in the third data set, respectively call the mapping hash function to obtain a calculation result of the mapping hash function, obtain a first random number power of each calculation result of the mapping hash function, form a ninth data set with the first random number power of all calculation results of the mapping hash function, and calculate a second random number power for each element of the ninth data set to form a seventh data set;

[0058] For each element in the fourth data set, call the mapping hash function respectively to obtain the calculation result of the mapping hash function, obtain the third random number power of each calculation result of the mapping hash function, and form a tenth data set with the third random number power of the calculation results of all the mapping hash functions. For each element of the tenth data set, calculate the fourth random number power to form an eighth data set.

[0059] In some embodiments, the sender randomly selects a positive integer (a first random number) For the third data set For each element e3 in And calculate ε s , forming as The data set is taken, and the order of the elements in the data set is disrupted to form the ninth data set The ninth data set Send to the receiver. The receiver randomly selects a positive integer (the third random number) For the fourth data set For each element e4 in And calculate e r , forming a r data set, disrupting the order of the elements in the data set to form the tenth data set And the tenth data set Sent to the sender, the above is a mapping hash function, function Will accept a group description and any length string x∈{0,1} * , output an element The Deffie-Hellman group is described as It should be noted that for any element and positive integers Using symbols Represents a element Multiply, that is, element to the power of a.

[0060] In some embodiments, the receiving party receives the ninth data set For the ninth data set For each element e5 in, calculate ε1=e5 s , forming the seventh data set including ε1 At this time, s is the second random number, which is equal to the first random number; the sender receives the tenth data set For the tenth data set For each element e6 in, calculate ε2=e6 s , forming the eighth data set including ε2 And the eighth data set Sent to the recipient, at this time s is the fourth random number, and the fourth random number is equal to the first random number.

[0061] In an optional implementation, obtaining an estimated size of the intersection according to the intersection of the seventh data set and the eighth data set includes:

[0062] According to the intersection of the seventh data set and the eighth data set, and the intersection estimation formula, the intersection estimation size is obtained, and the intersection estimation formula includes Among them, M is the estimated size of the intersection, is the size of the second data set, is the size of the tenth data set, and m′ is the size of the intersection of the seventh data set and the eighth data set.

[0063] In some embodiments, the receiving party obtains the seventh data set and the eighth data set The size of the intersection m' is estimated according to the intersection estimation formula Get the estimated size M of the intersection.

[0064] S103, if the estimated intersection size is not greater than the lower limit of the intersection size expected by the receiver, causing the receiver and the sender to exit the protocol;

[0065] In a specific embodiment, if the estimated intersection size is not greater than the lower limit of the intersection size expected by the receiver, that is, M≤m, the receiver sends a protocol termination instruction to the sender and exits the protocol, and the sender terminates the protocol and exits the protocol.

[0066] S104, if the estimated size of the intersection is greater than the lower limit of the size of the intersection expected by the receiver, the sender and the receiver re-negotiate the key to obtain a second key, and obtain a second bucket hash function according to the second key;

[0067] In a specific embodiment, the sender and the receiver re-negotiate the key to obtain a second key. Calculate n = min (n S ,n R ), and calculate the number of buckets Get the second bucket hash function

[0068] S105. For each element in the first data set, the second bucket hash function is called respectively to form a fifth data set, so that the sender uses the fifth data set to call the protocol executed by the sender in the privacy intersection protocol. For each element in the second data set, the second bucket hash function is called respectively to form a sixth data set, so that the receiver uses the sixth data set to call the protocol executed by the receiver in the privacy intersection protocol to obtain an intersection result.

[0069] In an optional embodiment, for each element in the first data set, respectively calling the second bucket hash function to form a fifth data set includes, for each element in the first data set, respectively calling the second bucket hash function to form a fifth data set with all elements in the first data set that satisfy the calculation result of the second bucket hash function to be i, where i∈{0,...,B'-1}, and B' is the number of buckets of the second bucket hash function;

[0070] For each element in the second data set, the second bucket hash function is called respectively to form a sixth data set, including, for each element in the second data set, the second bucket hash function is called respectively to form a sixth data set with all elements in the second data set that satisfy the calculation result of the second bucket hash function being i.

[0071] In some embodiments, the privacy request protocol is PSI =(Send,Recv), where Π PSI .Send is the protocol executed by the sender in the privacy-seeking protocol, which requires a set to be provided as input; and Π PSI .Recv is the protocol executed by the receiver in the privacy intersection protocol. It needs to provide a set as input and get a set as output at the end of the protocol.

[0072] For the first data set For each element e1 in, call the second bucket hash function All elements in the first data set that satisfy the calculation result of the second bucket hash function is i are used to form a fifth data set. Wherein, i∈{0,...,B'-1}, B' is the number of buckets of the second bucket hash function; for all i∈{0,...,B'-1}, the sender uses the fifth data set Call That is, the protocol executed by the sender in the privacy request protocol; for the second data set For each element e2 in the bucket, the second bucket hash function is called The second data set All elements in the bucket that satisfy the condition that the calculation result of the second bucket hash function is i, to form a sixth data set For all i∈{0,...,B'-1}, the receiver uses the set Call That is, the protocol executed by the receiver in the privacy intersection protocol, thereby obtaining the intersection of the fifth data set and the sixth data set Make the receiver calculate the set Output the intersection result.

[0073] The privacy intersection method for big data provided by the embodiment of the present invention obtains a first data set of a sender and a second data set of a receiver, and enables the sender and the receiver to negotiate a key to obtain a first key, and obtain a first bucket hash function according to the first key; for each element in the first data set, the first bucket hash function is called respectively to form a third data set, and for each element in the second data set, the first bucket hash function is called respectively to form a fourth data set, and an estimated intersection size is obtained according to the third data set and the fourth set; if the estimated intersection size is not greater than the lower limit of the intersection size expected by the receiver, the receiver and the sender exit the protocol; if the estimated intersection size is greater than the lower limit of the intersection size expected by the receiver, the sender and the receiver re-negotiate a key to obtain a second key, and obtain a second bucket hash function according to the second key; for For each element in the first data set, the second bucket hash function is called respectively to form a fifth data set, so that the sender uses the fifth data set to call the protocol executed by the sender in the privacy intersection protocol. For each element in the second data set, the second bucket hash function is called respectively to form a sixth data set, so that the receiver uses the sixth data set to call the protocol executed by the receiver in the privacy intersection protocol to obtain an intersection result. The intersection size can be estimated in advance to obtain an estimated intersection size. If the estimated intersection size is not greater than the lower limit of the intersection size expected by the receiver, the receiver and the sender exit the protocol to end the modeling early in the joint modeling scenario, thereby avoiding the waste of computing power resources and memory resources. If the estimated intersection size is greater than the lower limit of the intersection size expected by the receiver, the intersection result is obtained, which can realize the execution of the privacy intersection task between large data volumes in a smaller (limited) memory space.

[0074] The privacy intersection method for big data provided by the embodiment of the present invention can evaluate the intersection size in a relatively light way, saving computing power and time. The process of estimating the intersection size does not leak the data in the data set. If the intersection size does not meet the modeling requirements, the protocol can be terminated in advance without continuing to perform the privacy intersection task, which can efficiently solve the problems of privacy intersection and joint modeling in medical big data scenarios. The privacy intersection method for big data provided by the embodiment of the present invention can support privacy intersection between big data, including but not limited to privacy intersection between tens of billions of data, and can complete the privacy intersection of medical big data under limited resource configuration, thereby releasing the value of medical data.

[0075] The privacy intersection method for big data provided by the embodiment of the present invention can be constructed based on artificial intelligence, and relevant data can be acquired and processed based on artificial intelligence technology to realize unattended privacy intersection for big data by artificial intelligence. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0076] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0077] Figure 2 is a schematic diagram of the structure of a privacy protection device for big data according to an embodiment of the present invention. Figure 2 As shown, the privacy intersection device 20 for big data includes a first bucket hash function acquisition module 21, an intersection size estimation module 22, a protocol exit module 23, a second bucket hash function acquisition module 24 and an intersection result acquisition module 25;

[0078] The first bucket hash function acquisition module 21 is used to acquire a first data set of a sender and a second data set of a receiver, so that the sender and the receiver negotiate a key to obtain a first key, and acquire a first bucket hash function according to the first key;

[0079] The intersection size estimation module 22 is used to call the first bucket hash function for each element in the first data set to form a third data set, call the first bucket hash function for each element in the second data set to form a fourth data set, and obtain the intersection estimated size according to the third data set and the fourth data set;

[0080] The protocol exit module 23 is used to make the receiver and the sender exit the protocol if the estimated intersection size is not greater than the lower limit of the intersection size expected by the receiver;

[0081] The second bucket hash function acquisition module 24 is used to enable the sender and the receiver to renegotiate the key if the estimated intersection size is greater than the lower limit of the intersection size expected by the receiver, to obtain a second key, and to obtain a second bucket hash function according to the second key;

[0082] The intersection result acquisition module 25 is used to call the second bucket hash function for each element in the first data set to form a fifth data set, so that the sender uses the fifth data set to call the protocol executed by the sender in the privacy intersection protocol, and to call the second bucket hash function for each element in the second data set to form a sixth data set, so that the receiver uses the sixth data set to call the protocol executed by the receiver in the privacy intersection protocol to obtain an intersection result.

[0083] In an optional implementation, the first bucket hash function acquisition module 21 acquires the first bucket hash function according to the first key, including:

[0084] The first bucket number is obtained according to the size of the first data set, the size of the second data set and a bucket number calculation formula, and the first bucket hash function is obtained according to the first bucket number and the first key. The bucket number calculation formula includes

[0085]

[0086] Among them, B is the number of first buckets, is the size of the second data set, is the size of the first data set, w is the lower limit of the set size that the sender can run the privacy intersection protocol at a single time, It means round up.

[0087] In an optional embodiment, the intersection size estimation module 22 calls the first bucket hash function for each element in the first data set to form a third data set, and calls the first bucket hash function for each element in the second data set to form a fourth data set, including:

[0088] For each element in the first data set, the first bucket hash function is called respectively, and a third data set is formed with all elements in the first data set that satisfy the conditions that the calculation result of the first bucket hash function is 0. For each element in the second data set, the first bucket hash function is called respectively, and a fourth data set is formed with all elements in the second data set that satisfy the conditions that the calculation result of the first bucket hash function is 0.

[0089] In an optional implementation, the intersection size estimation module 22 obtains the estimated intersection size according to the third data set and the fourth data set, including:

[0090] For each element in the third data set, the mapping hash function is called respectively to form a seventh data set. For each element in the fourth data set, the mapping hash function is called respectively to form an eighth data set. According to the intersection of the seventh data set and the eighth data set, the estimated size of the intersection is obtained.

[0091] In an optional implementation, the intersection size estimation module 22 calls the mapping hash function for each element in the third data set to form a seventh data set, and calls the mapping hash function for each element in the fourth data set to form an eighth data set, including:

[0092] For each element in the third data set, respectively call the mapping hash function to obtain a calculation result of the mapping hash function, obtain a first random number power of each calculation result of the mapping hash function, form a ninth data set with the first random number power of all calculation results of the mapping hash function, and calculate a second random number power for each element of the ninth data set to form a seventh data set;

[0093] For each element in the fourth data set, call the mapping hash function respectively to obtain the calculation result of the mapping hash function, obtain the third random number power of each calculation result of the mapping hash function, add the third random number power of the calculation results of all the mapping hash functions to form a tenth data set, and calculate the fourth random number power for each element of the tenth data set to form an eighth data set.

[0094] In an optional implementation, the intersection size estimation module 22 obtains the estimated intersection size according to the intersection of the seventh data set and the eighth data set, including:

[0095] According to the intersection of the seventh data set and the eighth data set, and the intersection estimation formula, the intersection estimation size is obtained, and the intersection estimation formula includes Among them, M is the estimated size of the intersection, is the size of the second data set, is the size of the tenth data set, and m′ is the size of the intersection of the seventh data set and the eighth data set.

[0096] In an optional embodiment, the intersection result acquisition module 25 calls the second bucket hash function for each element in the first data set to form a fifth data set, including calling the second bucket hash function for each element in the first data set to form a fifth data set with all elements in the first data set that satisfy the conditions that the calculation result of the second bucket hash function is i, where i∈{0,...,B'-1}, B' is the number of buckets of the second bucket hash function; the intersection result acquisition module 25 calls the second bucket hash function for each element in the second data set to form a sixth data set, including calling the second bucket hash function for each element in the second data set to form a sixth data set with all elements in the second data set that satisfy the conditions that the calculation result of the second bucket hash function is i.

[0097] Figure 3 Schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Figure 3 As shown, the electronic device 30 includes a processor 31 and a memory 32 that is communicatively connected to the processor 31 .

[0098] The memory 32 stores program instructions for implementing the privacy intersection method for big data in any of the above embodiments.

[0099] The processor 31 is used to execute program instructions stored in the memory 32 to perform privacy protection for big data.

[0100] The processor 31 may also be referred to as a CPU (Central Processing Unit). The processor 31 may be an integrated circuit chip having signal processing capabilities. The processor 31 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0101] An embodiment of the present invention provides a storage medium, and the storage medium of the embodiment of the present invention stores program instructions that can implement all the above-mentioned privacy-seeking methods for big data, and the storage medium can be non-volatile or volatile. Among them, the program instructions can be stored in the above-mentioned storage medium in the form of a software product, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes, or terminal devices such as computers, servers, mobile phones, and tablets.

[0102] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0103] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, or each module may exist physically separately, or two or more modules may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of a software functional unit. The above is only an implementation mode of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the specification and drawings of the present invention, or directly or indirectly used in other related technical fields, is also included in the patent protection scope of the present invention.

[0104] The above description is only an implementation mode of the present invention. It should be pointed out that, for ordinary technicians in this field, improvements can be made without departing from the creative concept of the present invention, but these all belong to the protection scope of the present invention.

Claims

1. A privacy intersection method for big data, characterized in that: include: Obtaining a first data set of a sender and a second data set of a receiver, causing the sender and the receiver to negotiate a key to obtain a first key, and obtaining a first bucket hash function according to the first key; For each element in the first data set, the first bucket hash function is called respectively to form a third data set, for each element in the second data set, the first bucket hash function is called respectively to form a fourth data set, and an estimated size of the intersection is obtained according to the third data set and the fourth data set; If the estimated intersection size is not greater than the lower limit of the intersection size expected by the receiver, the receiver and the sender are caused to exit the protocol; If the estimated size of the intersection is greater than the lower limit of the size of the intersection expected by the receiver, the sender and the receiver re-negotiate the key to obtain a second key, and obtain a second bucket hash function according to the second key; For each element in the first data set, the second bucket hash function is called respectively to form a fifth data set, so that the sender uses the fifth data set to call the protocol executed by the sender in the privacy intersection protocol. For each element in the second data set, the second bucket hash function is called respectively to form a sixth data set, so that the receiver uses the sixth data set to call the protocol executed by the receiver in the privacy intersection protocol to obtain an intersection result.

2. The privacy intersection method for big data according to claim 1 is characterized in that: Obtaining a first bucket hash function according to the first key includes: The first bucket number is obtained according to the size of the first data set, the size of the second data set and a bucket number calculation formula, and the first bucket hash function is obtained according to the first bucket number and the first key. The bucket number calculation formula includes Among them, B is the number of first buckets, is the size of the second data set, is the size of the first data set, w is the lower limit of the set size that the sender can run the privacy intersection protocol at a single time, It means rounding up.

3. The privacy intersection method for big data according to claim 1, characterized in that: For each element in the first data set, the first bucket hash function is called respectively to form a third data set, and for each element in the second data set, the first bucket hash function is called respectively to form a fourth data set, including: For each element in the first data set, the first bucket hash function is called respectively, and a third data set is formed with all elements in the first data set that satisfy the conditions that the calculation result of the first bucket hash function is 0. For each element in the second data set, the first bucket hash function is called respectively, and a fourth data set is formed with all elements in the second data set that satisfy the conditions that the calculation result of the first bucket hash function is 0.

4. The privacy intersection method for big data according to claim 1, characterized in that: Obtaining an estimated size of the intersection according to the third data set and the fourth data set includes: For each element in the third data set, the mapping hash function is called respectively to form a seventh data set. For each element in the fourth data set, the mapping hash function is called respectively to form an eighth data set. According to the intersection of the seventh data set and the eighth data set, the estimated size of the intersection is obtained.

5. The privacy intersection method for big data according to claim 4 is characterized in that: For each element in the third data set, respectively call the mapping hash function to form a seventh data set, and for each element in the fourth data set, respectively call the mapping hash function to form an eighth data set, including: For each element in the third data set, respectively call the mapping hash function to obtain a calculation result of the mapping hash function, obtain a first random number power of each calculation result of the mapping hash function, form a ninth data set with the first random number power of all calculation results of the mapping hash function, and calculate a second random number power for each element of the ninth data set to form a seventh data set; For each element in the fourth data set, call the mapping hash function respectively to obtain the calculation result of the mapping hash function, obtain the third random number power of each calculation result of the mapping hash function, add the third random number power of the calculation results of all the mapping hash functions to form a tenth data set, and calculate the fourth random number power for each element of the tenth data set to form an eighth data set.

6. The privacy intersection method for big data according to claim 4, characterized in that: Obtaining an estimated size of the intersection according to the intersection of the seventh data set and the eighth data set, including: According to the intersection of the seventh data set and the eighth data set, and the intersection estimation formula, the intersection estimation size is obtained, and the intersection estimation formula includes Among them, M is the estimated size of the intersection, is the size of the second data set, is the size of the tenth data set, and m′ is the size of the intersection of the seventh data set and the eighth data set.

7. The privacy intersection method for big data according to claim 1, characterized in that: For each element in the first data set, respectively calling the second bucket hash function to form a fifth data set, including, for each element in the first data set, respectively calling the second bucket hash function to form a fifth data set with all elements in the first data set that satisfy the calculation result of the second bucket hash function to be i, where i∈{0,...,B'-1}, B' is the number of buckets of the second bucket hash function; For each element in the second data set, the second bucket hash function is called respectively to form a sixth data set, including, for each element in the second data set, the second bucket hash function is called respectively to form a sixth data set with all elements in the second data set that satisfy the calculation result of the second bucket hash function being i.

8. A privacy-enforcing device for big data, characterized in that: It includes a first bucket hash function acquisition module, an intersection size estimation module, a protocol exit module, a second bucket hash function acquisition module and an intersection result acquisition module; The first bucket hash function acquisition module is used to acquire a first data set of a sender and a second data set of a receiver, so that the sender and the receiver negotiate a key to obtain a first key, and acquire a first bucket hash function according to the first key; The intersection size estimation module is used to call the first bucket hash function for each element in the first data set to form a third data set, call the first bucket hash function for each element in the second data set to form a fourth data set, and obtain the estimated size of the intersection according to the third data set and the fourth data set; The protocol exit module is used to make the receiver and the sender exit the protocol if the estimated intersection size is not greater than the lower limit of the intersection size expected by the receiver; The second bucket hash function acquisition module is used to enable the sender and the receiver to renegotiate the key if the estimated intersection size is greater than the lower limit of the intersection size expected by the receiver, to obtain a second key, and to obtain a second bucket hash function according to the second key; The intersection result acquisition module is used to call the second bucket hash function for each element in the first data set to form a fifth data set, so that the sender uses the fifth data set to call the protocol executed by the sender in the privacy intersection protocol, and to call the second bucket hash function for each element in the second data set to form a sixth data set, so that the receiver uses the sixth data set to call the protocol executed by the receiver in the privacy intersection protocol to obtain an intersection result.

9. An electronic device, comprising a memory and a processor, wherein the memory stores a computer program executable by the processor, wherein: When the processor executes the computer program, the privacy intersection method for big data is implemented as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the privacy-enforcing method for big data as described in any one of claims 1 to 7 is implemented.

Images