A privacy intersection method, device and equipment for big data and storage medium

Abstract

The application provides a privacy intersection method and device for big data, equipment and a storage medium. The method provided by the application comprises: performing key negotiation to obtain a first key and acquiring a first bucketed hash function; calling the first bucketed hash function for elements in a first data set, calling the first bucketed hash function for elements in a second data set, and acquiring an intersection estimate size; if the intersection estimate size is not greater than a lower limit of an expected intersection size of a receiver, then the protocol is exited; otherwise, the key negotiation is performed again to obtain a second key and acquire a second bucketed hash function; a protocol executed by a sender in a privacy intersection protocol is called using a fifth data set, and a protocol executed by a receiver in the privacy intersection protocol is called using a sixth data set, so as to obtain an intersection result. The method of the application can avoid waste of computing resource and memory resource, and can perform a privacy intersection task between big data in a small memory space.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a privacy-preserving intersection method, apparatus, device, and storage medium for big data. Background Technology

[0002] Medical data encompasses patient data from healthcare institutions, electronic medical records, imaging data, laboratory reports, and more. With the rapid development of information technology, today's medical data is increasingly exhibiting characteristics of big data. Because medical data contains sensitive personal information, its free circulation is prohibited by regulations and laws. Privacy-preserving intersection protocols can obtain the intersection of all sets without disclosing the participating datasets. For example, before joint modeling, participants can use a specific feature to find the intersection and then use the intersecting data for modeling. In recent years, privacy-preserving intersection protocols have seen rapid development, and many practical and high-performance protocols already exist, such as ECDH-PSI, KKRT-PSI, and VOLE-PSI. However, existing privacy-preserving intersection schemes do not fully consider big data scenarios and cannot execute privacy-preserving intersection tasks between large amounts of data in a limited memory space. For example, a terminal machine with only 64GB of memory may be unable to perform privacy-preserving intersection tasks involving tens of billions of data points. In joint modeling scenarios, there are often requirements on the size of the intersection data. If the intersection is too small, it will lead to poor model performance or even failure to model. However, existing privacy-preserving intersection schemes cannot predict the intersection size in advance and require the complete execution of the protocol to know the intersection size, which may waste computing and memory resources. Summary of the Invention

[0003] The purpose of this invention is to provide a privacy-preserving intersection method, apparatus, device, and storage medium for big data, so as to solve the technical problems in the prior art that it is impossible to perform privacy-preserving intersection tasks between large amounts of data in a small memory space and that it is impossible to predict the size of the intersection in advance.

[0004] The technical solution of this invention is as follows: a privacy-preserving intersection method for big data is provided, comprising:

[0005] Obtain the first data set of the sender and the second data set of the receiver, enable the sender and the receiver to perform key negotiation to obtain the first key, and obtain the first bucket hash function based on the first key;

[0006] For each element in the first data set, the first bucket hash function is called to form the third data set. For each element in the second data set, the first bucket hash function is called to form the fourth data set. The estimated size of the intersection of the third data set and the fourth data set is obtained.

[0007] If the estimated size of the intersection is not greater than the lower limit of the expected size of the intersection by the receiver, then the receiver and the sender shall exit the protocol.

[0008] If the estimated size of the intersection is greater than the lower limit of the expected size of the intersection by the receiver, the sender and the receiver shall re-negotiate the key to obtain the second key, and obtain the second bucket hash function based on the second key;

[0009] For each element in the first data set, the second bucket hash function is called to form a fifth data set, which the sender uses to call the protocol executed by the sender in the privacy intersection protocol. For each element in the second data set, the second bucket hash function is called to form a sixth data set, which the receiver uses to call the protocol executed by the receiver in the privacy intersection protocol to obtain the intersection result.

[0010] Further, obtaining the first bucket hash function based on the first key includes:

[0011] The first bucket number is obtained based on the size of the first data set, the size of the second data set, and the bucket number calculation formula. The first bucket hash function is obtained based on the first bucket number and the first key. The bucket number calculation formula includes...

[0012]

[0013] Where B is the number of the first bucket. The size of the second data set, Let w be the size of the first data set, and w be the lower limit of the set size that the sender can run the privacy intersection protocol in a single operation. This indicates rounding up.

[0014] Furthermore, for each element in the first data set, the first bucket hash function is called to form a third data set; for each element in the second data set, the first bucket hash function is called to form a fourth data set, including:

[0015] For each element in the first data set, the first bucket hash function is called respectively, and a third data set is formed by all elements in the first data set that satisfy the condition that the calculation result of the first bucket hash function is 0. For each element in the second data set, the first bucket hash function is called respectively, and a fourth data set is formed by all elements in the second data set that satisfy the condition that the calculation result of the first bucket hash function is 0.

[0016] Further, obtaining the estimated size of the intersection based on the third data set and the fourth data set includes:

[0017] For each element in the third data set, the mapping hash function is called to form the seventh data set. For each element in the fourth data set, the mapping hash function is called to form the eighth data set. The estimated size of the intersection is obtained based on the intersection of the seventh data set and the eighth data set.

[0018] Furthermore, for each element in the third data set, a mapping hash function is invoked to form a seventh data set; for each element in the fourth data set, the mapping hash function is invoked to form an eighth data set, including:

[0019] For each element in the third data set, the mapping hash function is called to obtain the calculation result of the mapping hash function. The first random number power of the calculation result of each mapping hash function is obtained. The first random number power of the calculation results of all the mapping hash functions is used to form the ninth data set. The second random number power is calculated for each element in the ninth data set to form the seventh data set.

[0020] For each element in the fourth data set, the mapping hash function is called to obtain the calculation result of the mapping hash function. The third random number power of the calculation result of each mapping hash function is obtained. The third random number power of the calculation results of all the mapping hash functions is added to form the tenth data set. The fourth random number power is calculated for each element in the tenth data set to form the eighth data set.

[0021] Further, based on the intersection of the seventh data set and the eighth data set, the estimated size of the intersection is obtained, including:

[0022] Based on the intersection of the seventh and eighth data sets, and the intersection prediction formula, the estimated size of the intersection is obtained. The intersection prediction formula includes... Where M is the estimated size of the intersection. The size of the second data set, m' is the size of the tenth data set, and m' is the size of the intersection of the seventh and eighth data sets.

[0023] Further, for each element in the first data set, the second bucket hash function is called to form a fifth data set, including calling the second bucket hash function for each element in the first data set, and forming the fifth data set with all elements in the first data set that satisfy the condition that the calculation result of the second bucket hash function is i, where i∈{0,...,B'-1}, and B' is the number of buckets in the second bucket hash function;

[0024] For each element in the second data set, the second bucket hash function is called to form the sixth data set. This includes calling the second bucket hash function for each element in the second data set to form the sixth data set by including all elements in the second data set that satisfy the condition that the calculation result of the second bucket hash function is i.

[0025] Another technical solution of the present invention is as follows: a privacy intersection device for big data is provided, including a first bucket hash function acquisition module, an intersection size estimation module, a protocol exit module, a second bucket hash function acquisition module, and an intersection result acquisition module;

[0026] The first bucket hash function acquisition module is used to acquire the first data set of the sender and the second data set of the receiver, enable the sender and the receiver to negotiate the key to obtain the first key, and acquire the first bucket hash function based on the first key;

[0027] The intersection size estimation module is used to call the first bucket hash function for each element in the first data set to form a third data set, and to call the first bucket hash function for each element in the second data set to form a fourth data set, and to obtain the intersection size estimation based on the third data set and the fourth data set.

[0028] The protocol exit module is used to cause the receiver and the sender to exit the protocol if the estimated size of the intersection is not greater than the lower limit of the expected size of the intersection by the receiver.

[0029] The second bucket hash function acquisition module is used to, if the estimated size of the intersection is greater than the lower limit of the expected size of the intersection by the receiver, cause the sender and the receiver to re-negotiate the key to obtain the second key, and obtain the second bucket hash function based on the second key;

[0030] The intersection result acquisition module is used to call the second bucket hash function for each element in the first data set to form a fifth data set, so that the sender uses the fifth data set to call the protocol executed by the sender in the privacy intersection protocol. For each element in the second data set, the module calls the second bucket hash function to form a sixth data set, so that the receiver uses the sixth data set to call the protocol executed by the receiver in the privacy intersection protocol to obtain the intersection result.

[0031] Another technical solution of the present invention is as follows: an electronic device is provided, including a memory and a processor. The memory stores a computer program that can be executed by the processor. When the processor executes the computer program, it implements the privacy-preserving intersection method for big data as described in any of the above technical solutions.

[0032] Another technical solution of the present invention is as follows: a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and the computer program, when executed by a processor, implements the privacy-preserving intersection method for big data as described in any of the above technical solutions.

[0033] The beneficial effects of this invention are as follows: A first data set from the sender and a second data set from the receiver are obtained; the sender and receiver perform key negotiation to obtain a first key; a first bucket hash function is obtained based on the first key; for each element in the first data set, the first bucket hash function is called to form a third data set; for each element in the second data set, the first bucket hash function is called to form a fourth data set; the estimated intersection size is obtained based on the third and fourth data sets; if the estimated intersection size is not greater than the lower limit of the receiver's expected intersection size, the receiver and sender exit the protocol; if the estimated intersection size is greater than the lower limit of the receiver's expected intersection size, the sender and receiver renegotiate the key to obtain a second key; a second bucket hash function is obtained based on the second key; for the first data set... Each element in the second data set is processed by calling the second bucket hash function to form a fifth data set. The sender then uses this fifth data set to invoke the protocol executed by the sender in the privacy intersection protocol. For each element in the second data set, the second bucket hash function is called to form a sixth data set. The receiver then uses this sixth data set to invoke the protocol executed by the receiver in the privacy intersection protocol to obtain the intersection result. Through this scheme, the intersection size can be estimated in advance. If the estimated intersection size is not greater than the lower limit of the receiver's expected intersection size, the receiver and sender exit the protocol, thus ending the modeling process early in the joint modeling scenario and avoiding waste of computing and memory resources. If the estimated intersection size is greater than the lower limit of the receiver's expected intersection size, the intersection result is obtained. This allows for the execution of privacy intersection tasks between large amounts of data in a relatively small memory space. Attached Figure Description

[0034] Figure 1 A flowchart illustrating the privacy-preserving intersection method for big data provided in an embodiment of the present invention;

[0035] Figure 2 This is a schematic diagram of the privacy-preserving intersection device for big data provided in an embodiment of the present invention;

[0036] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0038] In the description of this application, the terms "first," "second," etc., are used only for distinguishing purposes and should not be construed as indicating or implying relative importance or order. In this specification, the terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0039] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0040] Figure 1 This is a flowchart illustrating a privacy-preserving intersection method for big data according to an embodiment of the present invention. It should be noted that if substantially the same result is obtained, the privacy-preserving intersection method for big data of the present invention does not necessarily reflect the intended outcome. Figure 1 The illustrated process sequence is limited. For example... Figure 1 As shown, this privacy-preserving intersection method for big data mainly includes the following steps:

[0041] S101, obtain the first data set of the sender and the second data set of the receiver, enable the sender and the receiver to perform key negotiation to obtain the first key, and obtain the first bucket hash function based on the first key;

[0042] It's important to note that the privacy intersection protocol calculates the intersection of all datasets without revealing the datasets possessed by the participating parties. Without loss of generality, the privacy intersection protocol can be considered to operate between two parties: a sender and a receiver. The sender only needs to provide its own dataset, while the receiver provides its own dataset and obtains the intersection of its dataset with the sender's dataset. The sender's first dataset and the receiver's second dataset can be sets of medical data, which may include patient data from medical institutions, electronic medical records, imaging data, laboratory reports, etc.

[0043] In an optional implementation, obtaining the first bucket hash function based on the first key includes:

[0044] The first bucket number is obtained based on the size of the first data set, the size of the second data set, and the bucket number calculation formula. The first bucket hash function is obtained based on the first bucket number and the first key. The bucket number calculation formula includes...

[0045]

[0046] Where B is the number of the first bucket. The size of the second data set, Let w be the size of the first data set, and w be the lower limit of the set size that the sender can run the privacy intersection protocol in a single operation. This indicates rounding up.

[0047] It should be noted that, In the formula for calculating the number of buckets, x and y are both positive integers. y is w.

[0048] In some embodiments, without loss of generality, let the sender's first data set be... The size of the set that the sender can run in a single instance of the privacy intersection protocol is a positive integer n. S , and w can be a lower bound on the size of the set that the sender can run in a single instance of the privacy intersection protocol, for example, w = 10 5 q is the upper limit of the set size for which the privacy intersection protocol can be run by the sender in a single run, for example, q = 10. 8 , Let be the size of the first data set (i.e., the number of elements in the set). Similarly, without loss of generality, let the receiver's second data set be... The size of the second data set is a positive integer. m is the lower bound of the size of the intersection expected by the receiver, n R This represents the size of the set that the receiver can run the privacy intersection protocol in a single run, a positive integer. The parameters also need to satisfy the relation t≤n R ≤10 8 ≤u, where t can be the lower bound of the set size that the receiver can run the privacy intersection protocol in a single run. t and w can be equal, for example, t = 10. 5 u is the upper limit of the set size for which the privacy intersection protocol can be run once for the receiver. u and q can be equal, for example, u = 10. 8 .

[0049] In one specific embodiment, the sender sends... and n S To the recipient, so that the recipient can send and n R Give it to the sender so that the sender can receive it. and n R To enable the recipient to receive and n S This enables the sender and receiver to negotiate a key and obtain the first key. Based on the first bucket number B and the first key Obtain the first bucket hash function in, The first key, the first bucket hash function Accept the first key e is an element in either the first data set or the second data set, and it can be a string of any length, e∈{0,1}. * The output is b∈{0,...,B-1}.

[0050] S102, for each element in the first data set, the first bucket hash function is called to form a third data set; for each element in the second data set, the first bucket hash function is called to form a fourth data set; and the estimated size of the intersection of the third data set and the fourth data set is obtained.

[0051] In an optional implementation, for each element in the first data set, the first bucket hash function is called to form a third data set; for each element in the second data set, the first bucket hash function is called to form a fourth data set, including:

[0052] For each element in the first data set, the first bucket hash function is called respectively, and a third data set is formed by all elements in the first data set that satisfy the condition that the calculation result of the first bucket hash function is 0. For each element in the second data set, the first bucket hash function is called respectively, and a fourth data set is formed by all elements in the second data set that satisfy the condition that the calculation result of the first bucket hash function is 0.

[0053] In some embodiments, without loss of generality, w = 10 5 This allows the sender to calculate the number of the first bucket. For the first data set For each element e1, call the first bucket hash function. and all satisfied Add the element to the third data set In this way, a third data set is formed. Make t=10 5 This allows the receiver to calculate the number of the first bucket. For the second data set For each element e2, call the first bucket hash function. and all satisfied Add the element to the fourth data set In this way, the fourth data set is formed.

[0054] In an optional implementation, obtaining the estimated size of the intersection based on the third data set and the fourth data set includes:

[0055] For each element in the third data set, the mapping hash function is called to form the seventh data set. For each element in the fourth data set, the mapping hash function is called to form the eighth data set. The estimated size of the intersection is obtained based on the intersection of the seventh data set and the eighth data set.

[0056] In an optional implementation, for each element in the third data set, a mapping hash function is invoked to form a seventh data set, and for each element in the fourth data set, the mapping hash function is invoked to form an eighth data set, including:

[0057] For each element in the third data set, the mapping hash function is called to obtain the calculation result of the mapping hash function. The first random number power of the calculation result of each mapping hash function is obtained. The first random number power of the calculation results of all the mapping hash functions is used to form the ninth data set. The second random number power is calculated for each element in the ninth data set to form the seventh data set.

[0058] For each element in the fourth data set, the mapping hash function is called to obtain the calculation result of the mapping hash function. The third random number power of the calculation result of each mapping hash function is obtained. The third random number power of the calculation results of all the mapping hash functions is used to form the tenth data set. The fourth random number power is calculated for each element in the tenth data set to form the eighth data set.

[0059] In some embodiments, the sender randomly selects a positive integer (a first random number). For the third data set For each element e3 in the equation, calculate... And calculate ε s , forming including εs The data set is shuffled, and the order of the elements in the data set is scrambled to form the ninth data set. The ninth data set Send to the recipient. The recipient randomly selects a positive integer (a third random number). For the fourth data set For each element e4 in the equation, calculate... And calculate e r , forming including e r The data set is shuffled, and the order of the elements in the data set is scrambled to form the tenth data set. and the tenth data set Send to the sender, the above For mapping hash functions, the function Will accept a group description and strings of arbitrary length x∈{0,1} * Output an element The description of the Deffie-Hellman group is as follows: It should be noted that for any element and positive integers Use symbols Represents a elements Multiplication, i.e., element a to the power of.

[0060] In some embodiments, the receiver receives the ninth data set. For the ninth data set For each element e5 in the array, calculate ε1 = e5. s This forms the seventh data set, which includes ε1. At this point, s is the second random number, which is equal to the first random number; the sender receives the tenth data set. For the tenth data set For each element e6 in the equation, calculate ε2 = e6. s This forms the eighth data set, which includes ε2. and the eighth data set Send it to the receiver. At this time, s is the fourth random number, which is equal to the first random number.

[0061] In an optional implementation, the estimated size of the intersection of the seventh data set and the eighth data set is obtained, including:

[0062] Based on the intersection of the seventh and eighth data sets, and the intersection prediction formula, the estimated size of the intersection is obtained. The intersection prediction formula includes... Where M is the estimated size of the intersection. The size of the second data set, m' is the size of the tenth data set, and m' is the size of the intersection of the seventh and eighth data sets.

[0063] In some embodiments, the receiver obtains the seventh data set. and the eighth data set The size m' of the intersection is determined by the intersection prediction formula. Obtain the estimated size M of the intersection.

[0064] S103, if the estimated size of the intersection is not greater than the lower limit of the expected size of the intersection by the receiver, then the receiver and the sender shall exit the protocol.

[0065] In one specific embodiment, if the estimated size of the intersection is not greater than the lower limit of the expected size of the intersection by the receiver, i.e., M≤m, the receiver sends a protocol termination instruction to the sender and exits the protocol. The sender then sends a protocol termination instruction and exits the protocol.

[0066] S104, if the estimated size of the intersection is greater than the lower limit of the expected size of the intersection by the receiver, the sender and the receiver shall re-negotiate the key to obtain the second key, and obtain the second bucket hash function based on the second key;

[0067] In one specific embodiment, the sender and receiver renegotiate the key to obtain a second key. Calculate n = min(n S ,n R ), and calculate the number of buckets. Get the second bucket hash function

[0068] S105, for each element in the first data set, the second bucket hash function is called to form a fifth data set, so that the sender uses the fifth data set to call the protocol executed by the sender in the privacy intersection protocol. For each element in the second data set, the second bucket hash function is called to form a sixth data set, so that the receiver uses the sixth data set to call the protocol executed by the receiver in the privacy intersection protocol to obtain the intersection result.

[0069] In an optional implementation, for each element in the first data set, the second bucket hash function is called to form a fifth data set. This includes calling the second bucket hash function for each element in the first data set, and forming the fifth data set with all elements in the first data set that satisfy the condition that the calculation result of the second bucket hash function is i, where i∈{0,...,B'-1} and B' is the number of buckets in the second bucket hash function.

[0070] For each element in the second data set, the second bucket hash function is called to form the sixth data set. This includes calling the second bucket hash function for each element in the second data set to form the sixth data set by including all elements in the second data set that satisfy the condition that the calculation result of the second bucket hash function is i.

[0071] In some embodiments, the privacy request protocol is Π PSI =(Send,Recv), where Π PSI .Send is the protocol executed by the sender in a privacy-preserving intersection protocol, requiring a set as input; while Π PSI .Recv is a protocol executed by the receiver in a privacy intersection protocol. It requires a set as input and outputs a set at the end of the protocol.

[0072] For the first data set For each element e1 in the dataset, the second bucket hash function is called respectively. The fifth data set is formed by taking all elements from the first data set that satisfy the condition that the result of the second bucket hash function is i. Where i∈{0,...,B'-1}, and B' is the number of buckets in the second bucket hash function; for all i∈{0,...,B'-1}, the sender uses the fifth data set. Call That is, the protocol executed by the sender in the privacy-preserving intersection protocol; for the second data set For each element e2, the second bucket hash function is called respectively. With the second data set The sixth data set is formed by combining all elements that satisfy the condition that the hash function of the second bucket results in value i. For all i∈{0,...,B'-1}, the receiver uses the set Call That is, the protocol executed by the receiver in the privacy intersection protocol, thereby obtaining the intersection of the fifth and sixth data sets. Enable the receiver to compute the set Output the intersection result.

[0073] The privacy-preserving intersection method for big data provided in this invention involves obtaining a first data set from the sender and a second data set from the receiver. The sender and receiver then negotiate a key to obtain a first key. A first bucket hash function is obtained based on the first key. For each element in the first data set, the first bucket hash function is called to form a third data set. For each element in the second data set, the first bucket hash function is called to form a fourth data set. The estimated intersection size is obtained based on the third and fourth data sets. If the estimated intersection size is not greater than the lower limit of the receiver's expected intersection size, the receiver and sender exit the protocol. If the estimated intersection size is greater than the lower limit of the receiver's expected intersection size, the sender and receiver renegotiate a key to obtain a second key. A second bucket hash function is obtained based on the second key. Each element in the first data set calls the second bucket hash function to form a fifth data set, which the sender uses to call the protocol executed by the sender in the privacy intersection protocol. For each element in the second data set, the second bucket hash function is called to form a sixth data set, which the receiver uses to call the protocol executed by the receiver in the privacy intersection protocol to obtain the intersection result. The intersection size can be estimated in advance. If the estimated intersection size is not greater than the lower limit of the receiver's expected intersection size, the receiver and sender exit the protocol to end the modeling early in the joint modeling scenario, thereby avoiding the waste of computing and memory resources. If the estimated intersection size is greater than the lower limit of the receiver's expected intersection size, the intersection result is obtained. This allows privacy intersection tasks between large amounts of data to be executed in a relatively small (limited) memory space.

[0074] The privacy-preserving intersection method for big data provided in this invention can lightweightly evaluate the intersection size, saving computing power and time. The process of estimating the intersection size does not leak data within the dataset. If the intersection size does not meet the modeling requirements, the protocol can be terminated early, eliminating the need to continue the privacy-preserving intersection task. This method efficiently solves the problems of privacy-preserving intersection and joint modeling in medical big data scenarios. The privacy-preserving intersection method for big data provided in this invention supports privacy-preserving intersections between large datasets, including but not limited to privacy-preserving intersections between billions of data points. It can complete privacy-preserving intersections for medical big data with limited resource allocation, unlocking the value of medical data.

[0075] The privacy-preserving intersection method for big data provided in this invention can be constructed based on artificial intelligence. It acquires and processes relevant data using AI technology, achieving unattended, AI-based privacy-preserving intersection for big data. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0076] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0077] Figure 2 This is a schematic diagram of the privacy-preserving intersection device for big data according to an embodiment of the present invention, as shown below. Figure 2 As shown, the privacy intersection device 20 for big data includes a first bucket hash function acquisition module 21, an intersection size estimation module 22, a protocol exit module 23, a second bucket hash function acquisition module 24, and an intersection result acquisition module 25;

[0078] The first bucket hash function acquisition module 21 is used to acquire the first data set of the sender and the second data set of the receiver, enable the sender and the receiver to negotiate the key to obtain the first key, and acquire the first bucket hash function based on the first key;

[0079] The intersection size estimation module 22 is used to call the first bucket hash function for each element in the first data set to form a third data set, and to call the first bucket hash function for each element in the second data set to form a fourth data set, and to obtain the intersection size estimation based on the third data set and the fourth data set.

[0080] The protocol exit module 23 is used to cause the receiver and the sender to exit the protocol if the estimated size of the intersection is not greater than the lower limit of the expected size of the intersection by the receiver.

[0081] The second bucket hash function acquisition module 24 is used to, if the estimated size of the intersection is greater than the lower limit of the expected size of the intersection by the receiver, cause the sender and the receiver to re-negotiate the key to obtain the second key, and obtain the second bucket hash function based on the second key;

[0082] The intersection result acquisition module 25 is used to call the second bucket hash function for each element in the first data set to form a fifth data set, so that the sender uses the fifth data set to call the protocol executed by the sender in the privacy intersection protocol. For each element in the second data set, the second bucket hash function is called to form a sixth data set, so that the receiver uses the sixth data set to call the protocol executed by the receiver in the privacy intersection protocol to obtain the intersection result.

[0083] In an optional implementation, the first bucket hash function acquisition module 21 acquires the first bucket hash function based on the first key, including:

[0084] The first bucket number is obtained based on the size of the first data set, the size of the second data set, and the bucket number calculation formula. The first bucket hash function is obtained based on the first bucket number and the first key. The bucket number calculation formula includes...

[0085]

[0086] Where B is the number of the first bucket. The size of the second data set, Let w be the size of the first data set, and w be the lower limit of the set size that the sender can run the privacy intersection protocol in a single operation. This indicates rounding up.

[0087] In an optional implementation, the intersection size estimation module 22 calls the first bucket hash function for each element in the first data set to form a third data set, and calls the first bucket hash function for each element in the second data set to form a fourth data set, including:

[0088] For each element in the first data set, the first bucket hash function is called respectively, and a third data set is formed by all elements in the first data set that satisfy the condition that the calculation result of the first bucket hash function is 0. For each element in the second data set, the first bucket hash function is called respectively, and a fourth data set is formed by all elements in the second data set that satisfy the condition that the calculation result of the first bucket hash function is 0.

[0089] In an optional implementation, the intersection size estimation module 22 obtains the intersection size estimate based on the third data set and the fourth data set, including:

[0090] For each element in the third data set, the mapping hash function is called to form the seventh data set. For each element in the fourth data set, the mapping hash function is called to form the eighth data set. The estimated size of the intersection is obtained based on the intersection of the seventh data set and the eighth data set.

[0091] In an optional implementation, the intersection size estimation module 22 calls a mapping hash function for each element in the third data set to form a seventh data set, and calls the mapping hash function for each element in the fourth data set to form an eighth data set, including:

[0092] For each element in the third data set, the mapping hash function is called to obtain the calculation result of the mapping hash function. The first random number power of the calculation result of each mapping hash function is obtained. The first random number power of the calculation results of all the mapping hash functions is used to form the ninth data set. The second random number power is calculated for each element in the ninth data set to form the seventh data set.

[0093] For each element in the fourth data set, the mapping hash function is called to obtain the calculation result of the mapping hash function. The third random number power of the calculation result of each mapping hash function is obtained. The third random number power of the calculation results of all the mapping hash functions is added to form the tenth data set. The fourth random number power is calculated for each element in the tenth data set to form the eighth data set.

[0094] In an optional implementation, the intersection size estimation module 22 obtains the intersection estimate size based on the intersection of the seventh data set and the eighth data set, including:

[0095] Based on the intersection of the seventh and eighth data sets, and the intersection prediction formula, the estimated size of the intersection is obtained. The intersection prediction formula includes... Where M is the estimated size of the intersection. The size of the second data set, m' is the size of the tenth data set, and m' is the size of the intersection of the seventh and eighth data sets.

[0096] In an optional implementation, the intersection result acquisition module 25, for each element in the first data set, calls the second bucket hash function to form a fifth data set, including, for each element in the first data set, calling the second bucket hash function to form the fifth data set with all elements in the first data set that satisfy the condition that the calculation result of the second bucket hash function is i, where i∈{0,...,B'-1}, and B' is the number of buckets in the second bucket hash function; the intersection result acquisition module 25, for each element in the second data set, calls the second bucket hash function to form a sixth data set, including, for each element in the second data set, calling the second bucket hash function to form the sixth data set with all elements in the second data set that satisfy the condition that the calculation result of the second bucket hash function is i.

[0097] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Figure 3 As shown, the electronic device 30 includes a processor 31 and a memory 32 communicatively connected to the processor 31.

[0098] The memory 32 stores program instructions for implementing the privacy-preserving intersection method for big data described in any of the above embodiments.

[0099] The processor 31 is used to execute program instructions stored in the memory 32 to perform privacy-preserving intersections for big data.

[0100] The processor 31 can also be referred to as a CPU (Central Processing Unit). The processor 31 may be an integrated circuit chip with signal processing capabilities. The processor 31 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor.

[0101] This invention provides a storage medium storing program instructions capable of implementing all the aforementioned privacy-preserving intersection methods for big data. The storage medium can be non-volatile or volatile. These program instructions can be stored in the storage medium as a software product, including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, or terminal devices such as computers, servers, mobile phones, and tablets.

[0102] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0103] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. The above are merely embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

[0104] The above description is merely an embodiment of the present invention. It should be noted that those skilled in the art can make improvements without departing from the inventive concept of the present invention, but these improvements all fall within the protection scope of the present invention.

Claims

1. A privacy-preserving intersection method for big data, characterized in that, include: Obtain the first data set of the sender and the second data set of the receiver, enable the sender and the receiver to perform key negotiation to obtain the first key, and obtain the first bucket hash function based on the first key; For each element in the first data set, the first bucket hash function is called respectively. A third data set is formed by all elements in the first data set that satisfy the condition that the calculation result of the first bucket hash function is 0. For each element in the second data set, the first bucket hash function is called respectively. A fourth data set is formed by all elements in the second data set that satisfy the condition that the calculation result of the first bucket hash function is 0. The estimated size of the intersection is obtained based on the third data set and the fourth data set. If the estimated size of the intersection is not greater than the lower limit of the expected size of the intersection by the receiver, then the receiver and the sender shall exit the protocol. If the estimated size of the intersection is greater than the lower limit of the expected size of the intersection by the receiver, the sender and the receiver shall re-negotiate the key to obtain the second key, and obtain the second bucket hash function based on the second key; For each element in the first data set, the second bucket hash function is called to form a fifth data set, which the sender uses to call the protocol executed by the sender in the privacy intersection protocol. For each element in the second data set, the second bucket hash function is called to form a sixth data set, which the receiver uses to call the protocol executed by the receiver in the privacy intersection protocol to obtain the intersection result.

2. The privacy-preserving intersection method for big data according to claim 1, characterized in that, Obtaining the first bucket hash function based on the first key includes: The first bucket number is obtained based on the size of the first data set, the size of the second data set, and the bucket number calculation formula. The first bucket hash function is obtained based on the first bucket number and the first key. The bucket number calculation formula includes... Where B is the number of the first bucket. , The size of the second data set, Let w be the size of the first data set, and w be the lower limit of the set size that the sender can run the privacy intersection protocol in a single operation. This indicates rounding up.

3. The privacy-preserving intersection method for big data according to claim 1, characterized in that, Obtaining the estimated size of the intersection of the third and fourth data sets includes: For each element in the third data set, the mapping hash function is called to form the seventh data set. For each element in the fourth data set, the mapping hash function is called to form the eighth data set. The estimated size of the intersection is obtained based on the intersection of the seventh data set and the eighth data set.

4. The privacy-preserving intersection method for big data according to claim 3, characterized in that, For each element in the third data set, a mapping hash function is called to form the seventh data set; for each element in the fourth data set, the mapping hash function is called to form the eighth data set, including: For each element in the third data set, the mapping hash function is called to obtain the calculation result of the mapping hash function. The first random number power of the calculation result of each mapping hash function is obtained. The first random number power of the calculation results of all the mapping hash functions is used to form the ninth data set. The second random number power is calculated for each element in the ninth data set to form the seventh data set. For each element in the fourth data set, the mapping hash function is called to obtain the calculation result of the mapping hash function. The third random number power of the calculation result of each mapping hash function is obtained. The third random number power of the calculation results of all the mapping hash functions is added to form the tenth data set. The fourth random number power is calculated for each element in the tenth data set to form the eighth data set.

5. The privacy-preserving intersection method for big data according to claim 3, characterized in that, Based on the intersection of the seventh data set and the eighth data set, obtain the estimated size of the intersection, including: Based on the intersection of the seventh and eighth data sets, and the intersection prediction formula, the estimated size of the intersection is obtained. The intersection prediction formula includes... ,in, To estimate the size of the intersection, The size of the second data set, The size of the tenth data set, The size of the intersection of the seventh data set and the eighth data set.

6. The privacy-preserving intersection method for big data according to claim 1, characterized in that, For each element in the first data set, the second bucket hash function is called to form a fifth data set. This includes calling the second bucket hash function on each element in the first data set, taking all elements in the first data set that satisfy the condition that the result of the second bucket hash function is... i The elements are used to form the fifth data set, where, , The number of buckets in the second bucket hash function; For each element in the second data set, the second bucket hash function is called to form the sixth data set. This includes calling the second bucket hash function on each element in the second data set, taking all elements in the second data set that satisfy the condition that the result of the second bucket hash function is... i The elements are used to form the sixth data set.

7. A privacy-preserving intersection device for big data, characterized in that, It includes a first bucket hash function acquisition module, an intersection size estimation module, a protocol exit module, a second bucket hash function acquisition module, and an intersection result acquisition module; The first bucket hash function acquisition module is used to acquire the first data set of the sender and the second data set of the receiver, enable the sender and the receiver to negotiate the key to obtain the first key, and acquire the first bucket hash function based on the first key; The intersection size estimation module is used to call the first bucket hash function for each element in the first data set, and form a third data set with all elements in the first data set that satisfy the condition that the calculation result of the first bucket hash function is 0. For each element in the second data set, the first bucket hash function is called, and form a fourth data set with all elements in the second data set that satisfy the condition that the calculation result of the first bucket hash function is 0. The intersection size is then estimated based on the third data set and the fourth data set. The protocol exit module is used to cause the receiver and the sender to exit the protocol if the estimated size of the intersection is not greater than the lower limit of the expected size of the intersection by the receiver. The second bucket hash function acquisition module is used to, if the estimated size of the intersection is greater than the lower limit of the expected size of the intersection by the receiver, cause the sender and the receiver to re-negotiate the key to obtain the second key, and obtain the second bucket hash function based on the second key; The intersection result acquisition module is used to call the second bucket hash function for each element in the first data set to form a fifth data set, so that the sender uses the fifth data set to call the protocol executed by the sender in the privacy intersection protocol. For each element in the second data set, the module calls the second bucket hash function to form a sixth data set, so that the receiver uses the sixth data set to call the protocol executed by the receiver in the privacy intersection protocol to obtain the intersection result.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program executable by the processor, characterized in that, When the processor executes the computer program, it implements the privacy-preserving intersection method for big data as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the privacy-preserving intersection method for big data as described in any one of claims 1 to 6.

Images