Data acquisition method and device, and storage medium

Abstract

Embodiments of the present disclosure provide a data acquisition method and device, and a storage medium. The method comprises: determining a data identifier intersection between databases of data providers, wherein the data identifier intersection comprises a same data identifier between the databases of the data providers; constructing a Bloom vector of a Bloom filter on the basis of the data identifier intersection, and sending the Bloom vector to the data providers; receiving candidate data sent by the data providers, wherein the candidate data is data corresponding to a target data identifier, and the target data identifier is determined by the data providers from data identifiers of the respective databases thereof by means of the Bloom filter on the basis of the Bloom vector; and selecting target data corresponding to the data identifier intersection from the candidate data. In a scenario of joint data processing of multiple parties, obfuscation is added to the data identifier intersection by means of the Bloom filter such that the data providers cannot acquire an original data identifier intersection, thereby avoiding the leakage of the data identifier intersection, and improving data security.

Description

Data acquisition method, device and storage medium

[0001] This application claims priority to Chinese Patent Application No. 202310994857.9 filed on August 8, 2023, and the contents of the above-mentioned Chinese patent application disclosure are hereby incorporated by reference in their entirety as a part of this application. Technical Field

[0002] The embodiments of the present disclosure relate to a data acquisition method, device, and storage medium. Background Art

[0003] With the rapid development of Internet technology and information services, data in all walks of life has shown explosive growth, creating opportunities for the joint processing of multi-party data, such as joint modeling based on multi-party data and federated learning.

[0004] In scenarios where multiple data providers are jointly processed, data from each provider must be aligned. This means that each provider must provide data with the same data identifier (ID). This is typically done using a Private Set Intersection (PSI) approach to determine the intersection of each provider's data identifiers. This intersection is then sent to each provider, who then provides the corresponding data based on the data identifiers in the intersection.

[0005] However, the data identification intersection is shared among the data providers, which results in the data identification intersection being leaked to all data providers and makes it impossible to achieve privacy protection for the data identification intersection.

[0006] Summary of the Invention

[0007] The embodiments of the present disclosure provide a data acquisition method, device, and storage medium to avoid leakage of data identifier intersections and improve data security.

[0008] In a first aspect, an embodiment of the present disclosure provides a data acquisition method, comprising:

[0009] Determine an intersection of data identifiers between databases of various data providers, wherein the intersection of data identifiers includes identical data identifiers between databases of various data providers;

[0010] Constructing a Bloom vector of a Bloom filter according to the data identifier intersection, and sending the Bloom vector to each data provider;

[0011] Receiving candidate data sent by each data provider, wherein the candidate data is data corresponding to a target data identifier, and the target data identifier is determined by each data provider from data identifiers in their respective databases based on the Bloom vector using a Bloom filter;

[0012] Target data corresponding to the intersection of the data identifiers is filtered out from the candidate data.

[0013] In a second aspect, an embodiment of the present disclosure provides a data acquisition method, including:

[0014] receiving a Bloom vector of a Bloom filter sent by a data processing device, where the Bloom vector of the Bloom filter is a Bloom vector corresponding to an intersection of data identifiers between databases of various data providers;

[0015] Determine a target data identifier from data identifiers in a database based on the Bloom vector using a Bloom filter;

[0016] Acquire data corresponding to the target data identifier from the database and determine it as candidate data;

[0017] The candidate data is sent to the data processing device.

[0018] In a third aspect, an embodiment of the present disclosure provides a data acquisition device, including:

[0019] An intersection determination unit, configured to determine an intersection of data identifiers between databases of various data providers, wherein the intersection of data identifiers includes identical data identifiers between databases of various data providers;

[0020] a vector construction unit, configured to construct a Bloom vector of a Bloom filter according to the data identifier intersection;

[0021] a sending unit, configured to send the Bloom vector to each data provider;

[0022] a receiving unit, configured to receive candidate data sent by each data provider, wherein the candidate data is data corresponding to a target data identifier, and the target data identifier is determined by each data provider from data identifiers in their respective databases based on the Bloom vector through a Bloom filter;

[0023] A data processing unit is used to filter out target data corresponding to the intersection of the data identifiers from the candidate data.

[0024] In a fourth aspect, an embodiment of the present disclosure provides a data acquisition device, including:

[0025] A receiving unit, configured to receive a Bloom vector of a Bloom filter sent by a data processing device, wherein the Bloom vector of the Bloom filter is a Bloom vector corresponding to an intersection of data identifiers between databases of various data providers;

[0026] a filtering unit, configured to determine a target data identifier from data identifiers in a database based on the Bloom vector by using a Bloom filter;

[0027] A data acquisition unit, configured to acquire data corresponding to the target data identifier from the database and determine the data as candidate data;

[0028] A sending unit is used to send the candidate data to the data processing device for joint data processing.

[0029] In a fifth aspect, an embodiment of the present disclosure provides an electronic device, including: at least one processor and a memory;

[0030] The memory stores computer-executable instructions;

[0031] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the data acquisition method described in the first aspect and various possible designs of the first aspect, or the data acquisition method described in the second aspect and various possible designs of the second aspect.

[0032] In a sixth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the data acquisition method described in the first aspect and various possible designs of the first aspect, or the data acquisition method described in the second aspect and various possible designs of the second aspect is implemented.

[0033] In the seventh aspect, an embodiment of the present disclosure provides a computer program product, including computer execution instructions. When a processor executes the computer execution instructions, it implements the data acquisition method described in the first aspect and various possible designs of the first aspect, or the data acquisition method described in the second aspect and various possible designs of the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, a brief introduction to the drawings required for the embodiments will be given below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0035] FIG1a is an example diagram of a data acquisition method in the related art;

[0036] FIG1b is an example diagram of another data acquisition method in the related art;

[0037] FIG2 is a schematic diagram of an application scenario of a data acquisition method provided by an embodiment of the present disclosure;

[0038] FIG3 is a flow chart of a data acquisition method according to an embodiment of the present disclosure;

[0039] FIG4 is a flow chart of a data acquisition method according to another embodiment of the present disclosure;

[0040] FIG5 is a flow chart of a data acquisition method according to another embodiment of the present disclosure;

[0041] FIG6 is a structural block diagram of a data acquisition device provided by an embodiment of the present disclosure;

[0042] FIG7 is a structural block diagram of a data acquisition device provided by another embodiment of the present disclosure; and

[0043] FIG8 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0044] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0045] First, let’s explain the terms involved in this application:

[0046] Privacy-preserving computing, also known as "privacy computing," refers to a class of technologies that analyze and compute data while protecting its privacy. Privacy computing involves analyzing and computing data in multi-party scenarios, ensuring that the original data is not disclosed by the data provider. This ensures that data can be securely circulated in a "available but invisible" manner. In machine learning scenarios, joint modeling and prediction based on multi-party data is generally required. Implementation techniques can be categorized into multi-party secure computing (MPC), federated learning, and trusted computing environments.

[0047] Vertical machine learning refers to a joint modeling scenario in which data from multiple or two parties have the same sample space but different feature spaces. In this scenario, the data from multiple or two parties modeled need to be aligned according to the data identifier (ID) first.

[0048] The Bloom filter was proposed by Bloom in 1970. It consists of a binary vector and a series of random mapping (hash) functions. Bloom filters can be used to determine whether an element is in a set. Its advantage is efficient query efficiency, but its disadvantage is a certain false positive rate (that is, data that is not in a set may be mistakenly identified as being in the set, while data that is in the set will not be identified as not being in the set). This false positive rate is adjustable (by controlling the size of the binary vector space).

[0049] A Trusted Execution Environment (TEE), a hardware-based security mechanism, loads the code and data involved in computing into a trusted environment protected by the CPU, providing confidentiality and integrity protection. TEEs offer a higher level of security than operating systems, making them suitable for processing sensitive data.

[0050] In scenarios where data from multiple parties is jointly processed, alignment of data from various data providers is often required. This means that each data provider must provide data with the same data identifier (ID). Related technologies typically use a Private Set Intersection (PSI) approach to align data from various data providers, determining the intersection of each provider's data identifiers. This intersection is then sent to each data provider, who then provides the corresponding data X based on the data identifiers in the intersection.

[0051] As shown in Figure 1a, taking two parties as an example, in the scenario of multi-party joint data processing, especially vertical joint modeling, when modeling through multi-party secure computing and federated learning, the data of each data provider is aligned to obtain the intersection of the data identifiers (IDs) between the data providers. Then, each data provider provides the corresponding data X (such as feature data) based on the data identifiers in the data identifier intersection for model training.

[0052] In another related technology, in a trusted computing environment (TEE), if the amount of data provided by each data provider is relatively large and cannot be loaded into the TEE environment all at once, or if data privacy protection requirements do not allow all data to be loaded into the TEE environment, as shown in Figure 1b, the full IDs of both parties are first loaded into the TEE environment, and data alignment is performed within the TEE environment. Then, each data provider loads the data X (such as feature data) corresponding to the intersection of the data identifiers (IDs) into the TEE environment for subsequent model training.

[0053] However, the intersection of data identifiers is shared among all data providers, resulting in the leakage of the intersection of data identifiers to all data providers. Any data provider can know part of the data identifiers of other data providers. If the data identifiers of a data provider are more comprehensive, it may know all the data identifiers of other data providers based on the intersection of data identifiers. Therefore, the privacy protection of the intersection of data identifiers cannot be achieved.

[0054] In order to solve the above technical problems, the present disclosure provides a data acquisition method. After obtaining the data identifier intersection between the databases of each data provider, the data identifier can be retrieved by each data provider based on the Bloom filter. In addition, considering that the Bloom filter has a false recognition rate, with the help of the false recognition rate of the Bloom filter, obfuscation is added to the target data identifier retrieved by each data provider, thereby reducing the probability of each data provider inferring the original data identifier intersection, making it impossible for each data provider to obtain the original data identifier intersection, avoiding the leakage of the data identifier intersection, and improving data security.

[0055] Furthermore, the length of the Bloom vector can be adjusted to make the false recognition rate of the Bloom filter sufficiently large, thereby further reducing the probability of each data provider inferring the original data identifier intersection.

[0056] The application scenario of the data acquisition method provided by the present disclosure is shown in Figure 2. The data processing device determines the intersection of data identifiers (IDs) between the databases of each data provider through data alignment. The data identifier intersection includes the same data identifiers between the databases of each data provider; constructs the Bloom vector of the Bloom filter based on the data identifier intersection, and sends the Bloom vector to each data provider; receives candidate data sent by each data provider, where the candidate data is the data corresponding to the target data identifier (target ID), and the target data identifier is determined by each data provider from the data identifier of each database based on the Bloom vector through the Bloom filter; and filters out the target data corresponding to the data identifier intersection from the candidate data. The data processing device can be a data processing device in a TEE environment.

[0057] It should be noted that the user information and data involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0058] The data acquisition method disclosed herein will be described in detail below with reference to specific embodiments.

[0059] Referring to FIG3 , FIG3 is a flow chart of a data acquisition method provided by an embodiment of the present disclosure. The method of this embodiment can be applied in a data processing device, and the data acquisition method includes:

[0060] S301: Determine the intersection of data identifiers between databases of various data providers, where the intersection of data identifiers includes the same data identifiers between databases of various data providers.

[0061] In this embodiment, in the application scenario of joint processing of multi-party data, such as joint modeling and prediction based on multi-party data, it is necessary to align the multi-party data according to the data identifier (ID) in the multi-party scenario while ensuring that the data provider does not leak the original data. In this embodiment, the same data identifier (ID) existing between the databases of each data provider can be determined first, and the data identifier intersection between the databases of each data provider can be obtained. The method for obtaining the data identifier intersection between the databases of each data provider can adopt any possible method, such as the privacy set intersection (PSI) method, or loading the data identifier of each data provider into the trusted computing environment (TEE) to calculate the intersection, which is not limited here.

[0062] S302: Construct a Bloom vector of a Bloom filter according to the data identifier intersection, and send the Bloom vector to each data provider.

[0063] In this embodiment, after obtaining the data identifier intersection between the databases of each data provider, each data provider needs to search for the data identifier in its own database that falls into the data identifier intersection. Considering that the Bloom filter can be used to retrieve whether an element is in a set, the Bloom filter is applied in this embodiment to realize the retrieval of data identifiers. The Bloom vector (also called bit vector, that is, a bit array composed of 0 and 1) of the Bloom filter can be constructed according to the data identifier intersection, and the Bloom vector can be sent to each data provider without directly sending the data identifier intersection to each data provider.

[0064] Among them, the process of constructing the Bloom vector of the Bloom filter according to the intersection of data identifiers can be as follows: first, a Bloom vector (bit vector, which can be all 0 initially) of a preset length can be constructed, and each data identifier in the intersection of data identifiers is hashed according to a preset hash function. If the hash calculation result of a certain data identifier corresponds to a certain position in the Bloom vector, for example, the hash calculation result of a certain data identifier is 2, which corresponds to the second position of the Bloom vector, the value of this position of the Bloom vector is set to 1.

[0065] S303: Receive candidate data sent by each data provider, wherein the candidate data is data corresponding to a target data identifier, and the target data identifier is determined by each data provider from data identifiers in their respective databases through a Bloom filter based on the Bloom vector.

[0066] In this embodiment, after receiving the Bloom vector of the Bloom filter sent by the data processing device, each data provider determines the target data identifier from the data identifiers of its own database through the Bloom filter based on the Bloom vector, that is, filtering of the data identifier is implemented through the Bloom filter, wherein the hit data identifier is recorded as the target data identifier, and the data corresponding to the target data identifier is obtained from the database, determined as candidate data, and the candidate data is sent to the data processing device.

[0067] Among them, the specific process of filtering the data identifier by the Bloom filter can be to perform hash calculation on any data identifier in the database according to a preset hash function, query the numerical value corresponding to the hash calculation result from the Bloom vector according to the hash calculation result, and determine whether the data identifier is the target data identifier based on the numerical value. If the numerical value is 1, it is determined that the data identifier is the target data identifier. If the numerical value is 0, it is determined that the data identifier is not the target data identifier.

[0068] In this embodiment, since the Bloom filter has a certain false recognition rate, specifically, the Bloom filter actually maps an element to a position in the Bloom vector through a hash function. Just by looking at whether the value of this position is 1, you can know whether the element is in the set. Multiple elements will face the problem of conflict during hash calculation, that is, multiple elements may be mapped to the same position in the Bloom vector. Therefore, the Bloom filter has the following characteristics: if a position in the Bloom vector is 0, the element mapped to this position must not be in the set. If a position in the Bloom vector is 1, the element mapped to this position is not necessarily in the set. Therefore, there is a certain false recognition rate, and the smaller the length of the Bloom vector, the greater the probability of hash conflict, and the greater the false recognition rate of the Bloom filter. In this embodiment, with the help of the false recognition rate of the Bloom filter, when any data provider filters the data tags based on the Bloom filter, in addition to filtering out the data identifiers belonging to the data identifier intersection, some data identifiers that do not belong to the data identifier intersection will also be filtered out, that is, the above-mentioned target data identifier includes the data identifier of the data identifier intersection and is greater than the data identifier intersection. This is equivalent to adding confusing data identifiers on top of the data identifier intersection, reducing the probability of each data provider inferring the original data identifier intersection, so that the data identifier intersection is protected.

[0069] S304: Filter out target data corresponding to the intersection of the data identifiers from the candidate data.

[0070] In this embodiment, after obtaining the candidate data sent by each data provider, since the candidate data includes data corresponding to data identifiers belonging to the intersection of data identifiers and data corresponding to data identifiers that do not belong to the intersection of data identifiers, based on the data identifier intersection, the data corresponding to the data identifiers belonging to the intersection of data identifiers can be filtered out from the candidate data, that is, the target data corresponding to the data identifier intersection.

[0071] Furthermore, after filtering out the target data corresponding to the intersection of the data identifiers from the candidate data, the target data with the same data identifier can be fused, and data processing can be performed based on the fused target data, such as modeling based on the fused target data, to achieve multi-party joint modeling.

[0072] The data acquisition method provided in this embodiment determines the intersection of data identifiers between the databases of each data provider, wherein the intersection of data identifiers includes the same data identifiers between the databases of each data provider; constructs a Bloom vector of a Bloom filter based on the intersection of data identifiers, and sends the Bloom vector to each data provider; receives candidate data sent by each data provider, wherein the candidate data is data corresponding to the target data identifier, and the target data identifier is determined by each data provider from the data identifiers of their respective databases through a Bloom filter based on the Bloom vector; and filters out the target data corresponding to the intersection of data identifiers from the candidate data. In this embodiment, based on the intersection of data identifiers between the databases of each data provider, each data provider retrieves the data identifier through a Bloom filter. By taking advantage of the fact that the Bloom filter has a false recognition rate, obfuscation is added to the target data identifier retrieved by each data provider, reducing the probability of each data provider inferring the original data identifier intersection, making it impossible for each data provider to obtain the original data identifier intersection, avoiding the leakage of the data identifier intersection, and improving data security.

[0073] Based on any of the above embodiments, constructing the bloom vector of the bloom filter according to the data identifier intersection in S302 may specifically include:

[0074] A hash calculation is performed on each data identifier in the intersection of the data identifiers according to a preset hash function, and the Bloom vector is constructed based on the hash calculation result and a preset Bloom vector length.

[0075] In this embodiment, when constructing a Bloom vector for the intersection of data identifiers, a hash calculation can be performed on each data identifier in the intersection of data identifiers according to a preset hash function, and then a Bloom vector is constructed based on the hash calculation result. Taking into account the fact that the Bloom filter has a false recognition rate in this embodiment, and the smaller the Bloom vector length, the greater the false recognition rate of the Bloom filter, a preset Bloom vector length can be set in advance when constructing the Bloom vector to control the false recognition rate of the Bloom filter, thereby controlling the degree of confusion added to the intersection of data identifiers.

[0076] Specifically, an initial Bloom vector (bit vector, which can be all 0 initially) can be constructed based on the preset Bloom vector length, and each data identifier in the intersection of the data identifiers can be hashed according to the preset hash function. If the hash calculation result of a certain data identifier corresponds to a certain position in the Bloom vector, for example, the hash calculation result of a certain data identifier is 2, which corresponds to the second position of the Bloom vector, then the value of this position of the Bloom vector is set to 1, and finally a Bloom vector of the preset Bloom vector length is obtained.

[0077] Optionally, the preset Bloom vector length is a preset multiple of the total number of data identifiers in the data identifier intersection, and the preset multiple is less than 1, that is, the preset Bloom vector length is less than the total number of data identifiers in the data identifier intersection. The ideal situation for the Bloom filter to have no false recognition rate is that each data identifier in the data identifier intersection corresponds to a position of the Bloom vector, that is, the length of the Bloom vector needs to be equal to or greater than the total number of data identifiers in the data identifier intersection. On the contrary, if the preset Bloom vector length is less than the total number of data identifiers in the data identifier intersection, the Bloom filter will inevitably have a false recognition rate, and the smaller the preset Bloom vector length is relative to the total number of data identifiers in the data identifier intersection, the greater the false recognition rate of the Bloom filter, that is, the smaller the above-mentioned preset multiple is, the greater the false recognition rate of the Bloom filter.

[0078] In another optional embodiment, before constructing the Bloom vector based on the hash calculation result and the preset Bloom vector length, the following steps may also be included:

[0079] Determine the total number of data identifiers in the data identifier intersection; and determine the preset Bloom vector length based on the total number of data identifiers and an adjustment factor for the misrecognition rate of the Bloom filter.

[0080] In this embodiment, the total number of data identifiers can be determined based on the intersection of the data identifiers, and the preset Bloom vector length can be determined based on the total number of data identifiers combined with the adjustment factor of the misrecognition rate of the Bloom filter (or it can also be called the adjustment factor for the confusion situation).

[0081] Specifically, the total number of data identifiers in the data identifier intersection can be first determined as the initial Bloom vector length. At this time, the false recognition rate of the Bloom filter is small or there is no false recognition rate. On the basis of the initial Bloom vector length, it is reduced based on the adjustment ratio to obtain the preset Bloom vector length. For example, if the adjustment ratio of the false recognition rate of the Bloom filter is 2 times, it means that the false recognition rate of the Bloom filter needs to be increased by 2 times, or the confusion situation needs to be increased by 2 times. At this time, it can be reduced by 2 times based on the initial Bloom vector length, that is, the preset Bloom vector length obtained at this time is 1 / 2 of the total number of data identifiers in the data identifier intersection.

[0082] Based on any of the above embodiments, the data acquisition method of this embodiment can be executed in a trusted computing environment TEE, that is, the above data processing device is a data processing device in the TEE environment. As shown in the figure, in the TEE environment, the data processing device acts as a neutral third party to determine the data identification intersection between the databases of each data provider; constructs the Bloom vector of the Bloom filter based on the data identification intersection, and sends the Bloom vector to each data provider; receives the candidate data sent by each data provider; filters out the target data corresponding to the data identification intersection from the candidate data; further, model training can be performed based on the target data corresponding to the data identification intersection, which can avoid data leakage and improve data security.

[0083] Refer to Figure 4, which is a flow chart of a data acquisition method provided by an embodiment of the present disclosure. The method of this embodiment can be applied to a device of a data provider, and the data acquisition method includes:

[0084] S401: Receive a Bloom vector of a Bloom filter sent by a data processing device, where the Bloom vector of the Bloom filter is a Bloom vector corresponding to an intersection of data identifiers between databases of various data providers;

[0085] S402, determining a target data identifier from data identifiers in a database based on the Bloom vector using a Bloom filter;

[0086] S403, obtaining data corresponding to the target data identifier from the database and determining it as candidate data;

[0087] S404: Send the candidate data to the data processing device.

[0088] Furthermore, determining the target data identifier from the data identifiers in the database based on the Bloom vector by using the Bloom filter includes:

[0089] A hash calculation is performed on any data identifier in the database according to a preset hash function, and a numerical value corresponding to the hash calculation result is queried from the Bloom vector according to the hash calculation result, and whether the data identifier is the target data identifier is determined according to the numerical value.

[0090] This embodiment is a method on the data provider side in the above embodiments. Its principles and technical effects can be found in the above embodiments and will not be described again here.

[0091] Referring to FIG5 , FIG5 is a flow chart of a data acquisition method provided in this embodiment, the data acquisition method including:

[0092] 1) The data processing device obtains the intersection of data identifications (IDs);

[0093] 2) The data processing device caches the intersection of data identifiers (IDs);

[0094] 3) The data processing device determines the preset Bloom vector length;

[0095] 4) The data processing device performs a hash calculation on each data identifier in the intersection of the data identifiers according to a preset hash function, and constructs a Bloom vector based on the hash calculation result and a preset Bloom vector length;

[0096] 5) The data processing device sends the Bloom vector to each data provider;

[0097] 6) The data provider receives the Bloom vector;

[0098] 7) The data provider performs a hash calculation on any data identifier in the database according to a preset hash function, and determines the target data identifier based on the Bloom vector through a Bloom filter;

[0099] 8) The data provider obtains the data (X) corresponding to the target data identifier from the database and determines it as candidate data;

[0100] 9) The data provider sends the candidate data to the data processing device;

[0101] 10) The data processing device determines whether the data from the data provider has been received. If it is received, the data processing device continues to execute the subsequent steps; if not, the data processing device waits;

[0102] 11) Filtering the target data corresponding to the intersection of data identifiers (IDs) from the candidate data to achieve deobfuscation;

[0103] 12) Data processing, such as fusing target data with the same data identifier and performing model training based on the fused target data.

[0104] Corresponding to the data acquisition method on the data processing device side in the above embodiments, FIG6 is a block diagram of the data acquisition device provided in an embodiment of the present disclosure. For ease of illustration, only the portions relevant to the embodiment of the present disclosure are shown. Referring to FIG6 , the data acquisition device 600 includes an intersection determination unit 601, a vector construction unit 602, a sending unit 603, a receiving unit 604, and a data processing unit 605.

[0105] The intersection determination unit 601 is configured to determine the intersection of data identifiers between the databases of the data providers, wherein the intersection of data identifiers includes the same data identifiers between the databases of the data providers;

[0106] A vector construction unit 602 is configured to construct a Bloom vector of a Bloom filter according to the data identifier intersection;

[0107] A sending unit 603 is configured to send the Bloom vector to each data provider;

[0108] A receiving unit 604 is configured to receive candidate data sent by each data provider, wherein the candidate data is data corresponding to a target data identifier, and the target data identifier is determined by each data provider from data identifiers in their respective databases based on the Bloom vector using a Bloom filter;

[0109] The data processing unit 605 is configured to filter out target data corresponding to the intersection of the data identifiers from the candidate data.

[0110] In one or more embodiments of the present disclosure, when constructing the Bloom vector of the Bloom filter according to the data identifier intersection, the vector construction unit 602 is configured to:

[0111] A hash calculation is performed on each data identifier in the intersection of the data identifiers according to a preset hash function, and the Bloom vector is constructed based on the hash calculation result and a preset Bloom vector length.

[0112] In one or more embodiments of the present disclosure, the preset Bloom vector length is a preset multiple of the total number of data identifiers in the data identifier intersection, and the preset multiple is less than 1.

[0113] In one or more embodiments of the present disclosure, before constructing the Bloom vector based on the hash calculation result and the preset Bloom vector length, the vector construction unit 602 is further configured to:

[0114] Determining the total number of data identifiers in the intersection of the data identifiers;

[0115] The preset Bloom vector length is determined based on the total number of data identifiers and an adjustment factor for the misrecognition rate of the Bloom filter.

[0116] In one or more embodiments of the present disclosure, when determining the preset Bloom vector length based on the total number of data identifiers and the adjustment factor for the false recognition rate of the Bloom filter, the vector construction unit 602 is configured to:

[0117] Determine the total number of data identifiers as the initial Bloom vector length;

[0118] The initial Bloom vector length is reduced based on the adjustment factor to obtain the preset Bloom vector length.

[0119] In one or more embodiments of the present disclosure, after filtering out the target data corresponding to the intersection of the data identifiers from the candidate data, the data processing unit 605 is further configured to:

[0120] Target data with the same data identifier are fused, and data processing is performed based on the fused target data.

[0121] In one or more embodiments of the present disclosure, the method is applied to a data processing device in a trusted computing environment (TEE).

[0122] The data acquisition device provided in this embodiment can be used to execute the technical solution of the method embodiment on the data processing device side. Its implementation principle and technical effects are similar and will not be described in detail in this embodiment.

[0123] Corresponding to the data acquisition method on the data provider side in the above embodiments, Figure 7 is a block diagram of the data acquisition device provided in the embodiments of the present disclosure. For ease of illustration, only the portions relevant to the embodiments of the present disclosure are shown. Referring to Figure 7 , the data acquisition device 700 includes a receiving unit 701, a filtering unit 702, a data acquisition unit 703, and a sending unit 704.

[0124] The receiving unit 701 is configured to receive a Bloom vector of a Bloom filter sent by a data processing device, where the Bloom vector of the Bloom filter is a Bloom vector corresponding to an intersection of data identifiers between databases of various data providers;

[0125] A filtering unit 702 is configured to determine a target data identifier from data identifiers in a database based on the Bloom vector using a Bloom filter;

[0126] A data acquisition unit 703 is configured to acquire data corresponding to the target data identifier from the database and determine the data as candidate data;

[0127] The sending unit 704 is configured to send the candidate data to the data processing device for joint data processing.

[0128] In one or more embodiments of the present disclosure, when determining the target data identifier from the data identifiers in the database based on the Bloom vector through the Bloom filter, the filtering unit 702 is configured to:

[0129] A hash calculation is performed on any data identifier in the database according to a preset hash function, and a numerical value corresponding to the hash calculation result is queried from the Bloom vector according to the hash calculation result, and whether the data identifier is the target data identifier is determined according to the numerical value.

[0130] The data acquisition device provided in this embodiment can be used to execute the technical solution of the method embodiment on the data provider side. Its implementation principle and technical effects are similar and will not be described in detail in this embodiment.

[0131] Referring to FIG8 , a schematic diagram of the structure of an electronic device 800 suitable for implementing an embodiment of the present disclosure is shown. The electronic device 800 may be a terminal device or a server. The terminal device may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle-mounted terminals (e.g., vehicle-mounted navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device shown in FIG8 is merely an example and should not limit the functionality and scope of use of the embodiments of the present disclosure.

[0132] As shown in Figure 8, the electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 into a random access memory (RAM) 803. Various programs and data required for the operation of the electronic device 800 are also stored in the RAM 803. The processing device 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0133] Typically, the following devices may be connected to the I / O interface 805: an input device 806 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 808 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 809. The communication device 809 may allow the electronic device 800 to communicate with other devices wirelessly or by wire to exchange data. Although FIG8 shows an electronic device 800 having various devices, it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0134] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 809, or installed from the storage device 808, or installed from the ROM 802. When the computer program is executed by the processing device 801, the above-mentioned functions defined in the data acquisition method on the data processing device side or the data acquisition method on the data provider side of the embodiment of the present disclosure are executed.

[0135] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0136] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0137] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0138] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider).

[0139] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0140] The units involved in the embodiments described in this disclosure may be implemented in software or hardware. In some cases, the name of a unit does not limit the unit itself. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses."

[0141] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0142] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0143] In a first aspect, according to one or more embodiments of the present disclosure, a data acquisition method is provided, comprising:

[0144] Determine an intersection of data identifiers between databases of various data providers, wherein the intersection of data identifiers includes identical data identifiers between databases of various data providers;

[0145] Constructing a Bloom vector of a Bloom filter according to the data identifier intersection, and sending the Bloom vector to each data provider;

[0146] Receiving candidate data sent by each data provider, wherein the candidate data is data corresponding to a target data identifier, and the target data identifier is determined by each data provider from data identifiers in their respective databases based on the Bloom vector using a Bloom filter;

[0147] Target data corresponding to the intersection of the data identifiers is filtered out from the candidate data.

[0148] According to one or more embodiments of the present disclosure, constructing a Bloom vector of a Bloom filter according to the data identifier intersection includes:

[0149] A hash calculation is performed on each data identifier in the intersection of the data identifiers according to a preset hash function, and the Bloom vector is constructed based on the hash calculation result and a preset Bloom vector length.

[0150] According to one or more embodiments of the present disclosure, the preset Bloom vector length is a preset multiple of the total number of data identifiers in the data identifier intersection, and the preset multiple is less than 1.

[0151] According to one or more embodiments of the present disclosure, before constructing the Bloom vector based on the hash calculation result and the preset Bloom vector length, the method further includes:

[0152] Determining the total number of data identifiers in the intersection of the data identifiers;

[0153] The preset Bloom vector length is determined based on the total number of data identifiers and an adjustment factor for the misrecognition rate of the Bloom filter.

[0154] According to one or more embodiments of the present disclosure, determining the preset Bloom vector length based on the total number of data identifiers and an adjustment factor for the misrecognition rate of the Bloom filter includes:

[0155] Determine the total number of data identifiers as the initial Bloom vector length;

[0156] The initial Bloom vector length is reduced based on the adjustment factor to obtain the preset Bloom vector length.

[0157] According to one or more embodiments of the present disclosure, after filtering out the target data corresponding to the intersection of the data identifiers from the candidate data, the method further includes:

[0158] Target data with the same data identifier are fused, and data processing is performed based on the fused target data.

[0159] According to one or more embodiments of the present disclosure, the method is applied to a data processing device in a trusted computing environment (TEE).

[0160] In a second aspect, according to one or more embodiments of the present disclosure, a data acquisition method is provided, comprising:

[0161] receiving a Bloom vector of a Bloom filter sent by a data processing device, where the Bloom vector of the Bloom filter is a Bloom vector corresponding to an intersection of data identifiers between databases of various data providers;

[0162] Determine a target data identifier from data identifiers in a database based on the Bloom vector using a Bloom filter;

[0163] Acquire data corresponding to the target data identifier from the database and determine it as candidate data;

[0164] The candidate data is sent to the data processing device.

[0165] According to one or more embodiments of the present disclosure, determining the target data identifier from the data identifiers in the database based on the Bloom vector by using a Bloom filter includes:

[0166] A hash calculation is performed on any data identifier in the database according to a preset hash function, and a numerical value corresponding to the hash calculation result is queried from the Bloom vector according to the hash calculation result, and whether the data identifier is the target data identifier is determined according to the numerical value.

[0167] In a third aspect, according to one or more embodiments of the present disclosure, a data acquisition device is provided, comprising:

[0168] An intersection determination unit, configured to determine an intersection of data identifiers between databases of various data providers, wherein the intersection of data identifiers includes identical data identifiers between databases of various data providers;

[0169] a vector construction unit, configured to construct a Bloom vector of a Bloom filter according to the data identifier intersection;

[0170] a sending unit, configured to send the Bloom vector to each data provider;

[0171] a receiving unit, configured to receive candidate data sent by each data provider, wherein the candidate data is data corresponding to a target data identifier, and the target data identifier is determined by each data provider from data identifiers in their respective databases based on the Bloom vector through a Bloom filter;

[0172] A data processing unit is used to filter out target data corresponding to the intersection of the data identifiers from the candidate data.

[0173] According to one or more embodiments of the present disclosure, when constructing the Bloom vector of the Bloom filter according to the data identifier intersection, the vector construction unit is configured to:

[0174] A hash calculation is performed on each data identifier in the intersection of the data identifiers according to a preset hash function, and the Bloom vector is constructed based on the hash calculation result and a preset Bloom vector length.

[0175] According to one or more embodiments of the present disclosure, the preset Bloom vector length is a preset multiple of the total number of data identifiers in the data identifier intersection, and the preset multiple is less than 1.

[0176] According to one or more embodiments of the present disclosure, before constructing the Bloom vector based on the hash calculation result and the preset Bloom vector length, the vector construction unit is further configured to:

[0177] Determining the total number of data identifiers in the intersection of the data identifiers;

[0178] The preset Bloom vector length is determined based on the total number of data identifiers and an adjustment factor for the misrecognition rate of the Bloom filter.

[0179] According to one or more embodiments of the present disclosure, when determining the preset Bloom vector length based on the total number of data identifiers and the adjustment factor for the false recognition rate of the Bloom filter, the vector construction unit is configured to:

[0180] Determine the total number of data identifiers as the initial Bloom vector length;

[0181] The initial Bloom vector length is reduced based on the adjustment factor to obtain the preset Bloom vector length.

[0182] According to one or more embodiments of the present disclosure, after the data processing unit filters out the target data corresponding to the intersection of the data identifiers from the candidate data, it is further configured to:

[0183] Target data with the same data identifier are fused, and data processing is performed based on the fused target data.

[0184] According to one or more embodiments of the present disclosure, the method is applied to a data processing device in a trusted computing environment (TEE).

[0185] In a fourth aspect, according to one or more embodiments of the present disclosure, a data acquisition device is provided, comprising:

[0186] A receiving unit, configured to receive a Bloom vector of a Bloom filter sent by a data processing device, wherein the Bloom vector of the Bloom filter is a Bloom vector corresponding to an intersection of data identifiers between databases of various data providers;

[0187] a filtering unit, configured to determine a target data identifier from data identifiers in a database based on the Bloom vector by using a Bloom filter;

[0188] A data acquisition unit, configured to acquire data corresponding to the target data identifier from the database and determine the data as candidate data;

[0189] A sending unit is used to send the candidate data to the data processing device for joint data processing.

[0190] According to one or more embodiments of the present disclosure, when the filtering unit determines the target data identifier from the data identifiers in the database based on the Bloom vector through the Bloom filter, it is configured to:

[0191] A hash calculation is performed on any data identifier in the database according to a preset hash function, and a numerical value corresponding to the hash calculation result is queried from the Bloom vector according to the hash calculation result, and whether the data identifier is the target data identifier is determined according to the numerical value.

[0192] In a fifth aspect, according to one or more embodiments of the present disclosure, there is provided an electronic device, comprising: at least one processor and a memory;

[0193] The memory stores computer-executable instructions;

[0194] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the data acquisition method described in the first aspect and various possible designs of the first aspect, or the data acquisition method described in the second aspect and various possible designs of the second aspect.

[0195] In the sixth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, in which computer-executable instructions are stored. When the processor executes the computer-executable instructions, the data acquisition method described in the first aspect and various possible designs of the first aspect, or the data acquisition method described in the second aspect and various possible designs of the second aspect is implemented.

[0196] In the seventh aspect, according to one or more embodiments of the present disclosure, a computer program product is provided, comprising computer-executable instructions. When a processor executes the computer-executable instructions, the data acquisition method described in the first aspect and various possible designs of the first aspect, or the data acquisition method described in the second aspect and various possible designs of the second aspect is implemented.

[0197] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0198] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0199] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A data acquisition method, comprising: Determine the data identifier intersection between the databases of the data providers, wherein the data identifier intersection includes the same data identifiers between the databases of the data providers; Constructing a Bloom vector of a Bloom filter according to the data identifier intersection, and sending the Bloom vector to each data provider; Receiving candidate data sent by each data provider, wherein the candidate data is data corresponding to a target data identifier, and the target data identifier is determined by each data provider from data identifiers of respective databases through the Bloom filter based on the Bloom vector; and Target data corresponding to the intersection of the data identifiers is screened out from the candidate data.

2. The data acquisition method according to claim 1, wherein: The step of constructing a Bloom vector of a Bloom filter according to the data identifier intersection includes: A hash calculation is performed on each data identifier in the intersection of the data identifiers according to a preset hash function, and the Bloom vector is constructed based on the hash calculation result and a preset Bloom vector length.

3. The data acquisition method according to claim 2, wherein: The preset Bloom vector length is a preset multiple of the total number of data identifiers in the data identifier intersection, and the preset multiple is less than 1.

4. The data acquisition method according to claim 2 or 3, wherein: Before constructing the Bloom vector based on the hash calculation result and the preset Bloom vector length, the method further includes: Determine the total number of data identifiers in the intersection of the data identifiers; The preset Bloom vector length is determined based on the total number of data identifiers and an adjustment factor for the misrecognition rate of the Bloom filter.

5. The data acquisition method according to claim 4, wherein: The determining the preset Bloom vector length based on the total number of data identifiers and the adjustment factor of the misrecognition rate of the Bloom filter includes: Determine the total number of data identifiers as the initial Bloom vector length; The initial Bloom vector length is reduced based on the adjustment factor to obtain the preset Bloom vector length.

6. The data acquisition method according to any one of claims 1 to 5, wherein: After the target data corresponding to the intersection of the data identifiers is screened out from the candidate data, the method further includes: Target data with the same data identifier are fused, and data processing is performed based on the fused target data.

7. The data acquisition method according to any one of claims 1 to 6, wherein: The data acquisition method is applied to a data processing device in a trusted computing environment TEE.

8. A data acquisition method, comprising: receiving a Bloom vector of a Bloom filter sent by a data processing device, where the Bloom vector of the Bloom filter is a Bloom vector corresponding to an intersection of data identifiers between databases of various data providers; Determining a target data identifier from data identifiers in the database based on the Bloom vector by using the Bloom filter; Acquire data corresponding to the target data identifier from the database and determine it as candidate data; as well as The candidate data is sent to the data processing device.

9. The data acquisition method according to claim 8, wherein: The determining the target data identifier from the data identifiers of the database based on the Bloom vector by using the Bloom filter includes: A hash calculation is performed on any data identifier of the database according to a preset hash function, and a numerical value at a position corresponding to the hash calculation result is queried from the Bloom vector according to the hash calculation result, and whether the data identifier is the target data identifier is determined according to the numerical value.

10. A data acquisition device, comprising: An intersection determination unit is configured to determine an intersection of data identifiers between databases of various data providers, wherein the intersection of data identifiers includes the same data identifiers between databases of various data providers; a vector construction unit configured to construct a Bloom vector of a Bloom filter according to the data identifier intersection; A sending unit, configured to send the Bloom vector to each data provider; A receiving unit is configured to receive candidate data sent by each data provider, wherein the candidate data is data corresponding to a target data identifier, and the target data identifier is determined by each data provider from data identifiers of respective databases through the Bloom filter based on the Bloom vector; as well as The data processing unit is configured to filter out target data corresponding to the intersection of the data identifiers from the candidate data.

11. A data acquisition device, comprising: a receiving unit configured to receive a Bloom vector of a Bloom filter sent by a data processing device, The Bloom vector of the Bloom filter is a Bloom vector corresponding to the intersection of the data identifiers between the databases of the data providers; a filtering unit configured to determine a target data identifier from data identifiers in the database based on the Bloom vector by using the Bloom filter; A data acquisition unit is configured to acquire data corresponding to the target data identifier from the database and determine the data as candidate data; as well as The sending unit is configured to send the candidate data to the data processing device for joint data processing.

12. An electronic device comprising: at least one processor and memory, wherein The memory is configured to store computer-executable instructions; The at least one processor is configured to execute the computer-executable instructions stored in the memory, so that the at least one processor performs the data acquisition method according to any one of claims 1 to 9.

13. A computer-readable storage medium storing computer-executable instructions, wherein: When the processor executes the computer-executable instruction, the data acquisition method according to any one of claims 1 to 9 is implemented.

14. A computer program product comprising computer executable instructions, wherein: When the processor executes the computer-executable instruction, the data acquisition method according to any one of claims 1 to 9 is implemented.

Images