Parallel computing method and system for set intersection computing integrity verification

Abstract

The application discloses a parallel computing method and system for set intersection computing integrity verification, and the method comprises the following steps: scanning all appearing elements in a set data set to form a value domain space, and dividing the value domain space into multiple non-overlapping subspaces; dividing each set into corresponding sub-sets according to the subspace division scheme, and constructing intersection computing integrity verification auxiliary information for different sub-sets in parallel; numbering all non-empty sub-sets of each set to form a meta-set of the set, and constructing intersection computing integrity verification auxiliary information for the meta-set; generating intersection computing integrity verification proofs for the meta-set and corresponding sub-sets according to a user-submitted intersection computing request, and checking the proofs on a client side; and performing the proof generation and checking processes of different sub-sets in parallel. The application can reduce the time cost required for constructing verification auxiliary information, generating verification proofs and checking the proofs in the process of set intersection computing integrity verification through a data parallel mode.

Description

Technical Field

[0001] This invention belongs to the field of parallel computing technology, specifically relating to a parallel computing method and computing system for verifying the integrity of set intersection calculations. Background Technology

[0002] With the rapid development of information technology, information systems have become the foundation of people's daily social life. In an open environment, information systems face various security attacks, including data tampering. To ensure the data security of information systems and the reliability of search results, it is essential to verify the integrity of the search results returned by the information system.

[0003] Set intersection calculation is one of the core operations of information systems. Many information system operations can be modeled as intersection operations on two or more sets. For example, equi-join retrieval between two data tables can be modeled as set intersection calculation. An equi-join connects two relational data tables through a common attribute. All values ​​of each data table on the attribute involved in the join can be modeled as a set of attribute values ​​for that table, with each attribute value corresponding to an element in the set. All attribute values ​​that appear in both relational tables should be included in the equi-join result; that is, the equi-join result should contain all elements of the intersection of the attribute value sets of the two tables. Verifying the integrity of the equi-join retrieval result is equivalent to verifying the integrity of the intersection calculation result of the attribute value sets of the two tables. Therefore, implementing integrity verification of set intersection calculation results is the foundation for implementing integrity verification of more complex retrieval results.

[0004] The completeness of the result of set intersection calculation has two aspects: correctness and completeness. Correctness means that the result of set intersection calculation is correct, does not contain erroneous results, and all set elements in the result come from both original sets. Completeness means that the result has no repetitions or omissions and contains all set elements that meet the requirements.

[0005] Existing methods for verifying the integrity of set intersection computations can be categorized into three types: verification methods based on cryptographic accumulators, methods based on verification data structures, and methods based on verifiable computation. Among these, verification methods based on cryptographic accumulators (e.g., Ferretti L, Colajanni M, Marchetti M. Implementation of verified setoperation protocols based on bilinear accumulators [C] / / Cryptology and NetworkSecurity: 15th International Conference, CANS 2016, Milan, Italy, November 14-16, 2016, Proceedings 15. Springer International Publishing, 2016: 626-636) utilize cryptographic accumulator technology to generate verification auxiliary information and integrity proofs for set intersection computations. Methods based on verification data structures use verification data structures such as Merkle hash trees and Merkle B+ trees as verification auxiliary information. Typical methods include Li F, Hadjieleftheriou M, Kollios G, et al. Dynamic authenticated index structures for outsourced databases[C] / / Proceedings of the 2006 ACM SIGMOD international conference on Management of data.2006:121-132. and Papadopoulos S, Papadias D, Cheng W, et al. Separating authentication from query execution in outsourced databases[C] / / 2009 IEEE 25th International Conference on Data Engineering.IEEE,2009:1148-1151.Verifiable computation-based methods utilize fully homomorphic encryption (e.g., Gennaro R, Gentry C, Parno B. Non-interactive verifiable computing: Outsourcing computation to untrusted workers [C] / / Advances in Cryptology–CRYPTO 2010:30th Annual Cryptology Conference, Santa Barbara, CA, USA, August 15-19, 2010. Proceedings 30. Springer Berlin Heidelberg, 2010:465-482) or verifiable set computation theory (e.g., Papadopoulos D, Papadopoulos S, Triandopoulos N. Taking authenticated range queries to arbitrary dimensions [C] / / Proceedings of the 2014 ACM SIGSACConference on Computer and Communications Security. 2014:819-830) to verify the integrity of computation results.

[0006] However, existing methods for verifying the integrity of sets by calculating their intersection suffer from performance issues when handling large-scale datasets. These methods are all designed for single-machine, serial environments, and as the set size increases, the computation time for constructing auxiliary information for integrity verification becomes excessive. Among the three existing methods, those based on verification data structures have the lowest computational complexity and the least overhead in constructing auxiliary information. However, limited by the single-core computing power of CPUs, existing methods can only handle sets with millions of elements in a short time. When the number of elements in the set reaches tens of millions or even hundreds of millions, the computation time of existing methods becomes excessively long. Summary of the Invention

[0007] Purpose of the invention: The purpose of this invention is to provide a parallel computing method and system for integrity verification of intersection calculations of large-scale datasets, reducing the time overhead of constructing verification auxiliary information (i.e. verification data structure) for integrity verification, thereby supporting integrity verification of intersection calculations of datasets with a scale of tens of millions in a shorter time.

[0008] Technical solution: In a first aspect, the present invention provides a parallel computing method for verifying the integrity of set intersection calculations, comprising the following steps:

[0009] (1) Obtain a dataset D from an input device, disk file, or computer network. The dataset consists of several sets, each identified by a unique keyword.

[0010] (2) Obtain all elements contained in all sets in the dataset to form the value space. Divide the value space into n disjoint subspaces according to the given hyperparameter n. Merge all subspaces together to form the complete value space. Assign integer numbers to the subspaces and generate a subspace partitioning scheme.

[0011] (3) For each set of the dataset, divide it into n subsets according to the subspace partitioning scheme described in step (2), and assign the subsets the same integer number as the corresponding subspace;

[0012] (4) For each non-empty subset of each set generated in step (3), use the set intersection computation integrity verification technology to construct intersection computation integrity verification auxiliary information for each subset; construct the integrity verification auxiliary information of each non-empty subset as an independent computation task, and execute multiple computation tasks in parallel using a multi-threaded or multi-process approach;

[0013] (5) Create a meta set for each set in the dataset, add the integer numbers of all non-empty subsets of the n subsets of the set generated in step (3) to the meta set, and use the set intersection calculation integrity verification technology to generate integrity verification auxiliary information for the meta set;

[0014] (6) Respond to the user's request for set intersection calculation based on the integrity verification auxiliary information of the set dataset, the metaset, and all subsets generated in step (4). The set intersection calculation request consists of several query keywords, and the query response to the set intersection calculation request includes:

[0015] Calculate the intersection of all sets corresponding to the query keywords and return the intersection result.

[0016] Calculate the intersection of the metasets corresponding to all query keywords, and generate a proof of the intersection's integrity based on the auxiliary information for metaset integrity verification. Return the metaset intersection result (META) and the proof of the intersection's integrity. META ;

[0017] For each integer index i appearing in the META result of the intersection calculation of the metasets, obtain the subset containing that integer index in the set corresponding to each query keyword, and generate a proof of the integrity of the intersection calculation of the subsets. iThe process of processing an integer number i is treated as an independent computational task. Multiple computational tasks are executed in parallel using multi-threading or multi-processing, and the integrity proof of the intersection of all generated subsets is returned.

[0018] (7) Calculate the integrity proof Proof based on the intersection of the returned metasets. META The integrity of the intersection calculation result META of the metaset is verified; and the integrity of the returned intersection calculation result Result is verified in parallel based on the intersection calculation result META of the metaset and the integrity proof of the intersection calculation of all subsets. If both verifications pass, the intersection calculation result is valid and the intersection calculation result is output to the user; otherwise, if either verification fails, the user is informed that the calculation result Result or the integrity proof has an error and the received intersection calculation result is invalid.

[0019] Furthermore, in step (2), the subspace partitioning scheme should have an easily computable function for determining the subspace number to which an element belongs, wherein the function for determining the subspace number y to which an element x belongs is y = |hash(x)|modn, and hash() is any hash function.

[0020] Further, step (3) includes: according to the generated subspace partitioning scheme, sequentially determining the subspace number to which the elements in the set belong, merging the elements belonging to the same subspace together to form a subset, and assigning the subset the same integer number as the corresponding subspace.

[0021] Furthermore, in step (4), the following arbitrary set intersection computation integrity verification techniques are used to generate intersection computation integrity verification auxiliary information for non-empty subsets: bilinear accumulator or Merkle hash tree, Merkle B+ tree, or other set intersection computation integrity verification techniques.

[0022] Furthermore, the meta-set generated in step (5) consists of several integers. If the meta-set contains an integer i, it means that the subset numbered i generated in step (3) is non-empty.

[0023] Further, in step (9), the process of generating the integrity proof of the set intersection calculation is as follows: First, calculate the intersection X of the meta-sets of all sets corresponding to the query keywords. Based on the integrity verification auxiliary information of the involved meta-sets, generate an integrity proof for the meta-set intersection calculation. For each number appearing in the meta-set intersection X, extract the subset corresponding to the number from the set corresponding to the query keyword. Based on the integrity verification auxiliary information of the subset, generate an integrity proof for the intersection of subsets with the same number. The intersection calculation and integrity proof generation process between subsets with different numbers are independent of each other and are generated in parallel. The method of generating the integrity proof is determined by the intersection calculation integrity verification technology used in step (4).

[0024] Further, in step (10), the process of parallel verification of the integrity of the intersection calculation result Result is as follows: the intersection calculation result Result is divided into n subsets according to the subspace partitioning scheme described in step (2), and each subset is assigned the same integer number as the corresponding subspace. It is checked whether the integer numbers of all non-empty subsets are included in META. If the integer number of a non-empty subset is not included in META, the verification fails. For each integer number i contained in META, the integrity proof of the subset intersection calculation is obtained based on the subset intersection calculation integrity proof received by the user. i The integrity of the subset numbered i in the intersection calculation result Result is verified. If the verification process of any integer number i fails, the verification fails. The verification process of each integer number i is treated as an independent calculation task, and multiple calculation tasks are executed in parallel using a multi-threaded or multi-process mechanism.

[0025] Secondly, the present invention provides a parallel computing system for verifying the integrity of set intersection calculations, comprising a data owner module, a query service module, and a client module, wherein the data owner module is configured as follows:

[0026] Obtain a dataset D from an input device, disk file, or computer network. The dataset consists of several sets, each identified by a unique keyword. Obtain the value space formed by all elements contained in all sets of the dataset. Divide the value space into n disjoint subspaces according to a given hyperparameter n. Merge all subspaces to form the complete value space. Assign integer numbers to the subspaces to generate a subspace partitioning scheme. For each set in the dataset, partition it into n subsets according to the aforementioned subspace partitioning scheme, assigning each subset the same integer number as its corresponding subspace. For each non-empty subset of each generated set, use set intersection... The dataset employs a set-based integrity verification technique, constructing intersection-based integrity verification auxiliary information for each subset. Each non-empty subset's integrity verification auxiliary information is treated as an independent computational task, with multiple tasks executed in parallel using multi-threading or multi-processing. A meta-set is created for each set in the dataset, and the integer indices of all non-empty subsets from the n generated subsets are added to the meta-set. The set intersection-based integrity verification technique is then used to generate integrity verification auxiliary information for the meta-set. Finally, the dataset, the meta-set's integrity verification auxiliary information, and the integrity verification auxiliary information of all generated subsets are passed to the query service module.

[0027] The query service module is configured to respond to user-initiated requests for set intersection calculations, where each request consists of several query keywords. The query response includes: calculating the intersection of the sets corresponding to all query keywords and returning the intersection calculation result (Result); calculating the intersection of the metasets of all sets corresponding to the query keywords, generating a metaset intersection calculation integrity proof based on metaset integrity verification auxiliary information, and returning the metaset intersection calculation result (META) and the metaset intersection calculation integrity proof (Proof). META For each integer index i appearing in the intersection of the metasets, obtain the subset containing that integer index in the set corresponding to each query keyword, and generate a proof of the completeness of the subset intersection calculation. i The process of processing an integer number i is treated as an independent computational task. Multiple computational tasks are executed in parallel using multi-threading or multi-processing, and the integrity proof of the intersection of all generated subsets is returned.

[0028] The client module is configured to compute the integrity proof Proof based on the intersection of the returned metasets. METAThe integrity of the intersection calculation result META of the metaset is verified; and the integrity of the returned intersection calculation result Result is verified in parallel based on the intersection calculation result META of the metaset and the integrity proof of the intersection calculation of all subsets. If both verifications pass, the intersection calculation result is valid and the intersection calculation result is output to the user; otherwise, if either verification fails, the user is informed that the calculation result Result or the integrity proof has an error and the received intersection calculation result is invalid.

[0029] Thirdly, the present invention provides a computer device comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein when the programs are executed by the processors, they implement the steps of the parallel computing method for verifying the integrity of set intersection computation as described in the first aspect of the present invention.

[0030] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the parallel computing method for verifying the integrity of set intersection calculation as described in the first aspect of the present invention.

[0031] Beneficial Effects: Compared with existing technologies, the significant advantages of this invention are as follows: This invention first divides the set into a series of subsets, and generates integrity verification auxiliary information and integrity proofs in parallel at the subset granularity, as well as verifying the integrity proofs. This avoids the performance problems caused by the serial generation of integrity verification auxiliary information and integrity proofs for the entire set in existing methods. When the set contains a large number of elements and has a high degree of parallelism, this invention can significantly reduce the time required for the data owner module and query service module to generate integrity verification auxiliary information and integrity proofs through parallel computing. As the dataset size increases and the number of elements in the set increases, the performance advantage of this invention compared to the serial generation method used in existing technologies will become even more significant. Simultaneously, during the generation of integrity proofs, the query service module first calculates the intersection of the metasets corresponding to the sets corresponding to the query keywords, and only generates integrity proofs for the subsets corresponding to the numbers appearing in the metaset intersection. This avoids the problem of existing methods needing to generate integrity proofs for the intersection results of the entire set, thereby reducing the data size of the integrity proofs and reducing the communication overhead between the query service module and the client module. Attached Figure Description

[0032] Figure 1 This is the workflow of the parallel computing method for verifying the integrity of set intersection calculation proposed in this invention;

[0033] Figure 2 This is the example set, dataset D, used in the embodiments;

[0034] Figure 3 This is a schematic diagram of the value domain space partitioning scheme in the embodiment;

[0035] Figure 4 This is a schematic diagram of the parallel generation process of integrity verification auxiliary information;

[0036] Figure 5 This is an experimental comparison of the time cost of key AIM steps between the method proposed in this invention and existing methods. Detailed Implementation

[0037] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0038] This invention provides a method and system for parallel intersection computation integrity verification of large-scale datasets, which can reduce the time overhead required to generate integrity verification auxiliary information, integrity proof, and verify integrity proof, and solves the problem that existing methods are unable to achieve intersection computation integrity verification of tens of millions of datasets in a short time.

[0039] The integrity verification method for set intersection calculation and the corresponding computer system workflow are as follows: Figure 1 As shown, a computer system supporting integrity verification of set intersection calculations consists of three modules: a data owner, a query service, and a client. First, the data owner module obtains the set dataset from the organization or company. In a relational database scenario, each column in a data table involved in the join corresponds to a set. The key to the set is a combination of the table name and column name, and all attribute values ​​of that column constitute the elements in the set. Second, the data owner module generates an integrity verification key; it shares this key with the client module via a confidential channel; based on the key, it generates integrity verification auxiliary information for each set in the dataset, which is then passed to the query service module. When the client needs to perform set intersection calculations, it sends the set keywords for the intersection calculation to the query service module. Next, the query service module performs the intersection calculation on the corresponding sets in the dataset based on the keywords sent by the client, obtains the calculation result, and generates an integrity certificate based on the corresponding set's verification auxiliary information; the query service module returns both the integrity certificate and the calculation result to the client module. Finally, the client module verifies the intersection calculation result based on the integrity proof, and outputs the intersection calculation result along with its integrity status, i.e., whether the calculation result meets the correctness and completeness conditions. If the integrity verification fails, it indicates that the data has been tampered with, and the set intersection calculation result is unreliable. Therefore, the client module can choose to discard the invalid calculation result.

[0040] The parallel computing method for integrity verification of set intersection provided by this invention divides the set into a series of subsets, and generates integrity verification auxiliary information and integrity proofs in parallel at the subset granularity, as well as verifies the integrity proofs, with reference to... Figure 1 The method of the present invention includes the following steps:

[0041] Step (1) Obtain a dataset D from an input device, disk file, or computer network. The dataset consists of several sets, each identified by a unique keyword, and each set contains several unique elements.

[0042] In this embodiment of the invention, the data owner reads a dataset D from an input device. This dataset contains four sets, identified by keywords R, S, T, and Q, respectively. Each set contains elements as follows: Figure 2 As shown.

[0043] Step (2): Scan all sets in the dataset to obtain all elements contained in all sets of the dataset, forming the value range space; The data owner module divides the value range space into n subspaces according to the hyperparameter n given by the user, and generates a subspace partitioning scheme.

[0044] According to an embodiment of the present invention, the subspace partitioning scheme has the following properties or satisfies the following conditions: the partitioning scheme divides the entire value domain into n consecutive subspaces, the subspaces do not overlap with each other, and all subspaces are merged together to form a complete value domain space; any set element belongs to only one subspace; all subspaces generated by the partitioning are assigned a unique integer number; there should be an easily computable function to determine the subspace number to which an element belongs.

[0045] In this embodiment of the invention, scanning Figure 2 The example dataset shown yields a set of all elements {0,1,2,5,6,8,9,10,11,12,13,14,15}, which constitutes the value space. Assuming the user-given hyperparameter n = 4, the value space is divided into four groups based on the remainder of the element values ​​modulo n. Each group corresponds to a subspace, and the subspace number is determined by the remainder of the element values ​​in the group modulo n. The resulting subspace partitioning scheme is as follows: Figure 3As shown. The three elements {0, 8, 12} are assigned to the subspace numbered 0 because their remainders after modulo 4 are all 0. The function to determine the subspace number y to which an element x belongs is y = |hash(x)| mod n, where hash() uses an equivalence function, i.e., hash(x) = x. This sub-partitioning scheme satisfies the following conditions: subspaces do not overlap, all subspaces are merged to form a complete value space; any set element belongs to only one subspace; all subspaces generated by the partitioning are assigned unique integer numbers; and there exists an easily computable function to determine the subspace number to which an element belongs.

[0046] Step (3): For each set in the dataset, divide the set into n subsets according to the subspace partitioning scheme, where the elements of the i-th subset all come from the i-th subspace.

[0047] According to an embodiment of the present invention, the process of dividing a subset is as follows: based on the subspace partitioning scheme generated in step (2), the subspace numbers to which the elements in the set belong are determined in turn, the elements belonging to the same subspace are merged together to form a subset, and the subset is assigned the same integer number as the corresponding subspace.

[0048] In this embodiment of the invention, with Figure 2 Taking the set {1,2,5,6,8,9,10,12,13,15} with the keyword R in the dataset shown as an example, according to the subspace partitioning scheme described in step (2), the subspace numbers to which the elements belong are determined sequentially, and elements belonging to the same subspace are numbered together, assigning the subset the same integer number as the corresponding subspace. For the set with the keyword R, elements 8 and 12 belong to the subspace numbered 0, so they are merged into a subset {8,12}, which is numbered 0. Similarly, the set can be further divided into a subset {1,5,9,13} with number 1, a subset {2,6,10} with number 2, and a subset {15} with number 3.

[0049] Step (4): For all non-empty subsets of each set in the dataset, use the existing set intersection computation integrity verification method to construct intersection computation integrity verification auxiliary information for each subset according to the pre-given key sk; construct intersection computation integrity verification auxiliary information in parallel for different sets or different subsets of the same set in a multi-threaded or multi-process manner.

[0050] Any existing set intersection computation integrity verification technique (such as bilinear accumulator, Merkle hash tree, Merkle B+ tree, etc.) can be used to generate auxiliary information for intersection computation integrity verification of non-empty subsets. In this embodiment of the invention, the data owner module uses the existing AIM algorithm (YANG Y, PAPADIAS D, PAPADOPOULOS S, et al. Authenticated join processing in outsourced databases [C] / / Proceedings of the 2009 ACM SIGMOD International Conference on Management of Data. ACM, 2009: 5-18) as the set intersection computation integrity verification method. This method generates integrity verification auxiliary information and integrity verification proof based on a Merkle B+ tree structure. Figure 2 Taking the example dataset as an example, the dataset contains four sets. The data owner module constructs the intersection of the four sets in parallel to calculate integrity verification auxiliary information. For the set with keyword R, all four subsets of this set are non-empty, and the data owner module generates integrity verification auxiliary information for the four subsets in parallel. Similarly, the sets with keywords S, T, and Q have 4, 2, and 2 non-empty subsets respectively, and integrity verification auxiliary information is generated for these subsets, such as... Figure 4 As shown.

[0051] Step (5): For each set in the dataset, the data owner module generates a meta-set for the set based on whether the n subsets of the set are empty. If a subset is not empty, the number of the subset is added to the meta-set. Using the existing set intersection computation integrity verification technology, auxiliary information for set intersection computation integrity verification is generated for the meta-set based on the pre-given key sk.

[0052] The metaset generated in this step consists of several integers. If the metaset contains an integer i, it means that the subset numbered i generated in step (3) is non-empty.

[0053] In this embodiment of the invention, for the set with keyword R, none of its four subsets are empty sets, so the indices of all four subsets are added to the metaset, resulting in the metaset {0,1,2,3}. The set with keyword T contains elements {1,5,9,13,15}. According to the subspace partitioning scheme, this set can be divided into four subsets numbered 0 to 3: {}, {1,5,9,13}, {}, and {15}, respectively. Subsets numbered 1 and 3 are not empty sets, so their corresponding metaset is {1,3}. Similarly, the metaset of the set with keyword S is {0,1,2,3}, and the metaset of the set with keyword Q is {0,2}. Using the existing AIM algorithm, integrity verification auxiliary information based on a Merkle B+ tree structure is generated for the metaset of each set in the dataset.

[0054] Step (6) responds to the user's request for set intersection calculation based on the set dataset, the integrity verification auxiliary information of the metaset, and the integrity verification auxiliary information of all subsets generated in step (4). The set intersection calculation request consists of several query keywords. The query response to the set intersection calculation request includes: calculating the intersection of the sets corresponding to all query keywords and returning the intersection calculation result Result; calculating the intersection of the metasets of the sets corresponding to all query keywords, and generating a metaset intersection calculation integrity proof based on the metaset integrity verification auxiliary information, and returning the metaset intersection calculation result META and the metaset intersection calculation integrity proof Proof. META For each integer index i appearing in the intersection of the metasets (META), obtain the subset containing that integer index in the set corresponding to each query keyword, and generate a proof of the completeness of the subset intersection calculation. i The process of processing an integer number i is treated as an independent computational task. Multiple computational tasks are executed in parallel using multi-threading or multi-processing, and the integrity proof of the intersection of all generated subsets is returned.

[0055] In this embodiment of the invention, the user initiates a set intersection calculation request to the client through an input device. In this embodiment, it is assumed that the query keywords submitted by the user are R and S, that is, the user requests to calculate the intersection of two sets with keywords R and S.

[0056] Upon receiving the calculation request, based on the query keywords R and S, the corresponding sets are retrieved from the dataset: R = {1,2,5,6,8,9,10,12,13,15} and S = {0,1,2,5,6,9,10,11,12,13,14}. The intersection of these sets, Result = R ∩ S = {1,2,5,6,9,10,12,13}, is calculated as the result, generating the query response.

[0057] Meta, based on the set whose keyword is R. R Meta is the metaset of sets with the keyword S and the set {0,1,2,3}. S ={0,1,2,3}, calculate the intersection of the metasets META = Meta R ∩Meta S ={0,1,2,3}; according to the metaset Meta R and Meta S The integrity verification auxiliary information is used to generate the integrity proof of the intersection of metasets using the AIM algorithm. META .

[0058] Each number appearing in the metaset intersection META (0, 1, 2, 3 in this example) is processed sequentially. The processing procedure for each number is the same; here, we take the processing procedure for number 0 as an example. For number 0, the subset with number 0 is extracted from the two sets R and S participating in the intersection calculation. These are the subsets {8, 12} of R and {0, 12} of S. Based on the integrity verification auxiliary information of the two subsets, the integrity proof Proof0 for the intersection calculation of these two subsets is generated using the AIM algorithm. The processing procedures for subsets with different numbers are independent of each other and are processed in parallel using a multi-threaded mechanism to generate integrity proofs Proof0, Proof1, Proof2, and Proof3. The metaset intersection META and the integrity proof Proof0 are then combined. META Proof0, Proof1, Proof2, and Proof3 are used as query responses.

[0059] Step (7): Calculate the integrity proof Proof based on the intersection of the returned metasets. META The integrity of the intersection calculation result META of the metaset is verified; and the integrity of the returned intersection calculation result Result is verified in parallel based on the intersection calculation result META of the metaset and the integrity proof of the intersection calculation of all subsets. If both verifications pass, the intersection calculation result is valid and the intersection calculation result is output to the user; otherwise, if either verification fails, the user is informed that the calculation result Result or the integrity proof has an error and the received intersection calculation result is invalid.

[0060] According to an embodiment of the present invention, the process of verifying the integrity proof is as follows: First, the integrity proof is calculated based on the intersection of the meta-sets, and the integrity of the meta-set intersection META is verified; according to the method described in step (3), the calculation result Result set is divided into n subsets according to the subspace partitioning scheme and numbered. The numbers of all non-empty subsets should appear in the meta-set intersection META. If the number of a non-empty subset does not belong to META, the verification fails; for each number in the meta-set intersection META, the integrity proof of the intersection of the subset corresponding to the number should be returned. If there is no corresponding integrity proof for a number, the verification fails; for each number appearing in the meta-set intersection META, the corresponding numbered subset of the calculation result set is verified based on the integrity proof of the intersection of the subset corresponding to the number. If the verification of any numbered subset fails, the verification fails; the verification method for the integrity proof is determined by the intersection calculation integrity verification technology used in step (4).

[0061] In this embodiment of the invention, the integrity proof Proof is first calculated based on the intersection of the metasets. META The integrity of the meta-set intersection META = {0,1,2,3} is verified using the AIM algorithm. Then, the calculation result set Result = {1,2,5,6,9,10,12,13} returned in step (6) is divided into four subsets numbered from 0 to 3 according to the subspace partitioning method described in step (3), namely {12}, {1,5,9,13}, {2,6,10}, and {}. The non-empty subsets are numbered 0, 1, and 2, all of which appear in the meta-set intersection META. Next, each number in META (0, 1, 2, and 3 in this embodiment) is verified. The verification process for each number is the same. The verification process for number 0 is illustrated below: For number 0, the integrity proof Proof0 is calculated based on the intersection of the subsets corresponding to this number. The AIM algorithm is used to verify the subset {12} with number 0 in the calculation result; similarly, the integrity proofs of other numbers are verified. If all the aforementioned verification processes pass, the entire integrity verification passes, and the verification result is communicated to the user. Conversely, if any verification fails, the entire verification process fails, and the user is also informed that the calculation result or integrity proof contains an error.

[0062] To test the performance of the parallel set intersection computation integrity verification method proposed in this invention, datasets of different sizes containing two sets were randomly generated in the experiment, with all sets in the dataset having the same number of elements. The experiment measured the time cost of key steps of the proposed parallel integrity verification method under different set sizes with 4 threads, and compared it with the existing serial integrity verification method AIM. The experimental results are as follows: Figure 5 As shown. Among them, Figure 5 (a) compares the time cost of generating integrity verification auxiliary information in steps (4) and (5). Figure 5 (b) compares the time cost of generating the intersection computation integrity proof and the intersection computation result in step (6). Figure 5 (c) in the figure compares the time overhead of verifying the integrity proof of the intersection calculation in step (7). Experimental results show that the time overhead of the parallel integrity verification method proposed in this invention is only 30.5% of the average time overhead of the existing serial method AIM. The method proposed in this invention can effectively reduce the time overhead required for verifying the integrity of set intersection calculation.

[0063] Based on the above detailed description of the parallel computing method for verifying the integrity of set intersection calculation proposed in this invention, correspondingly, this invention provides a parallel computing system for verifying the integrity of set intersection calculation, comprising a data owner module, a query service module, and a client module, wherein the data owner module is configured as follows:

[0064] Obtain a dataset D from an input device, disk file, or computer network. The dataset consists of several sets, each identified by a unique keyword. Obtain the value space formed by all elements contained in all sets of the dataset. Divide the value space into n disjoint subspaces according to a given hyperparameter n. Merge all subspaces to form the complete value space. Assign integer numbers to the subspaces to generate a subspace partitioning scheme. For each set in the dataset, partition it into n subsets according to the aforementioned subspace partitioning scheme, assigning each subset the same integer number as its corresponding subspace. For each non-empty subset of each generated set, use set intersection... The dataset employs a set-based integrity verification technique, constructing intersection-based integrity verification auxiliary information for each subset. Each non-empty subset's integrity verification auxiliary information is treated as an independent computational task, with multiple tasks executed in parallel using multi-threading or multi-processing. A meta-set is created for each set in the dataset, and the integer indices of all non-empty subsets from the n generated subsets are added to the meta-set. The set intersection-based integrity verification technique is then used to generate integrity verification auxiliary information for the meta-set. Finally, the dataset, the meta-set's integrity verification auxiliary information, and the integrity verification auxiliary information of all generated subsets are passed to the query service module.

[0065] The query service module is configured to respond to user-initiated requests for set intersection calculations, where each request consists of several query keywords. The query response includes: calculating the intersection of the sets corresponding to all query keywords and returning the intersection calculation result (Result); calculating the intersection of the metasets of all sets corresponding to the query keywords, generating a metaset intersection calculation integrity proof based on metaset integrity verification auxiliary information, and returning the metaset intersection calculation result (META) and the metaset intersection calculation integrity proof (Proof). META For each integer index i appearing in the intersection of the metasets, obtain the subset containing that integer index in the set corresponding to each query keyword, and generate a proof of the completeness of the subset intersection calculation. i The process of processing an integer number i is treated as an independent computational task. Multiple computational tasks are executed in parallel using multi-threading or multi-processing, and the integrity proof of all generated subsets is returned.

[0066] The client module is configured to compute the integrity proof Proof based on the intersection of the returned metasets. META The integrity of the intersection calculation result META of the metaset is verified; and the integrity of the returned intersection calculation result Result is verified in parallel based on the intersection calculation result META of the metaset and the integrity proof of the intersection calculation of all subsets. If both verifications pass, the intersection calculation result is valid and the intersection calculation result is output to the user; otherwise, if either verification fails, the user is informed that the calculation result Result or the integrity proof has an error and the received intersection calculation result is invalid.

[0067] It should be understood that the parallel computing system for verifying the integrity of set intersection calculation in the embodiments of the present invention can implement all the technical solutions in the above method embodiments. The functions of each functional module can be specifically implemented according to the methods in the above method embodiments. The specific implementation process can be referred to the relevant descriptions in the above embodiments, which will not be repeated here.

[0068] The present invention also provides a computer device comprising: one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein when the programs are executed by the processors, they implement the steps of the parallel computing method for verifying the integrity of set intersection computation as proposed in the present invention.

[0069] The present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the parallel computing method for verifying the integrity of set intersection calculation as proposed in the present invention are implemented.

[0070] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0071] This invention is described with reference to flowchart illustrations of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each step in the flowchart, and combinations of steps in the flowchart, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the steps in the flowchart. Figure 1 A device for a function specified in one or more processes.

[0072] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 The function specified in one or more processes.

[0073] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 Steps of a specified function in one or more processes.

Claims

1. A parallel computing method for verifying the integrity of set intersection calculations, characterized in that, Includes the following steps: (1) Obtain a dataset D from an input device, disk file, or computer network. The dataset consists of several sets, each identified by a unique keyword. (2) Obtain all elements contained in all sets in the dataset to form the value space. Divide the value space into n disjoint subspaces according to the given hyperparameter n. Merge all subspaces together to form the complete value space. Assign integer numbers to the subspaces and generate a subspace partitioning scheme. (3) For each set of the dataset, divide it into n subsets according to the subspace partitioning scheme described in step (2), and assign the subsets the same integer number as the corresponding subspace; (4) For each non-empty subset of each set generated in step (3), use the set intersection computation integrity verification technique to construct auxiliary information for intersection computation integrity verification for each subset; The integrity verification auxiliary information of each non-empty subset is constructed as an independent computation task, and multiple computation tasks are executed in parallel using a multi-threaded or multi-process approach. (5) Create a meta set for each set in the dataset, add the integer numbers of all non-empty subsets of the n subsets of the set generated in step (3) to the meta set, and use the set intersection calculation integrity verification technology to generate integrity verification auxiliary information for the meta set; (6) Respond to the user's request for set intersection calculation based on the integrity verification auxiliary information of the set dataset, the metaset, and the integrity verification auxiliary information of all subsets generated in step (4). The set intersection calculation request consists of several query keywords, and the query response to the set intersection calculation request includes: Calculate the intersection of all sets corresponding to the query keywords and return the intersection result. Calculate the intersection of the metasets corresponding to all query keywords, and generate a proof of the intersection's integrity based on the auxiliary information for metaset integrity verification. Return the metaset intersection result (META) and the proof of the intersection's integrity. META ; For each integer index i appearing in the META result of the intersection calculation of the metasets, obtain the subset containing that integer index in the set corresponding to each query keyword, and generate a proof of the integrity of the intersection calculation of the subsets. i The process of processing an integer number i is treated as an independent computational task. Multiple computational tasks are executed in parallel using multi-threading or multi-processing, and the integrity proof of the intersection of all generated subsets is returned. The process of generating the integrity proof of the intersection of sets is as follows: First, calculate the intersection X of the meta-sets of all sets corresponding to the query keywords. Based on the integrity verification auxiliary information of the meta-sets involved, generate an integrity proof for the intersection of the meta-sets. For each number appearing in the intersection X of the meta-sets, extract the subset corresponding to the number from the set corresponding to the query keyword. Based on the integrity verification auxiliary information of the subset, generate an integrity proof for the intersection of the subsets with the same number. The intersection calculation and integrity proof generation process between subsets with different numbers are independent of each other and are generated in parallel. The method of generating the integrity proof is determined by the intersection calculation integrity verification technology used in step (4). (7) Calculate the integrity proof Proof based on the intersection of the returned metasets. META The integrity of the intersection calculation result META of the metaset is verified; and the integrity of the returned intersection calculation result Result is verified in parallel based on the intersection calculation result META of the metaset and the integrity proof of the intersection calculation of all subsets; if both verifications pass, the intersection calculation result Result is valid and the intersection calculation result is output to the user; otherwise, if either verification fails, the user is informed that the calculation result Result or the integrity proof has an error and the received intersection calculation result is invalid. The process for parallel verification of the integrity of the intersection calculation result Result is as follows: the intersection calculation result Result is divided into n subsets according to the subspace partitioning scheme described in step (2), and each subset is assigned the same integer number as the corresponding subspace. It is then checked whether the integer numbers of all non-empty subsets are included in META. If the integer number of a non-empty subset is not included in META, the verification fails. For each integer number i included in META, the integrity proof of the subset intersection calculation is obtained based on the data received by the user. i The integrity of the subset numbered i in the intersection calculation result Result is verified. If the verification process of any integer number i fails, the verification fails. The verification process of each integer number i is treated as an independent calculation task, and multiple calculation tasks are executed in parallel using a multi-threaded or multi-process mechanism.

2. The method according to claim 1, characterized in that, In step (2), the subspace partitioning scheme should have an easily computable function to determine the subspace number to which an element belongs. The function to determine the subspace number y to which an element x belongs is y = |hash(x)| mod n, where hash() is any hash function.

3. The method according to claim 1, characterized in that, Step (3) includes: according to the generated subspace partitioning scheme, sequentially determine the subspace number to which the elements in the set belong, merge the elements belonging to the same subspace together to form a subset, and assign the subset the same integer number as the corresponding subspace.

4. The method according to claim 1, characterized in that, In step (4), the following arbitrary set intersection computation integrity verification techniques are used to generate auxiliary information for non-empty subsets: bilinear accumulator or Merkle hash tree, Merkle B+ tree, or other set intersection computation integrity verification techniques.

5. The method according to claim 1, characterized in that, The metaset generated in step (5) consists of several integers. If the metaset contains an integer i, it means that the subset numbered i generated in step (3) is non-empty.

6. A parallel computing system for verifying the integrity of set intersection calculations, characterized in that, It includes a data owner module, a query service module, and a client module. The data owner module is configured as follows: Obtain a dataset D from an input device, disk file, or computer network. The dataset consists of several sets, each identified by a unique keyword. Obtain the value space formed by all elements contained in all sets of the dataset. Divide the value space into n disjoint subspaces according to a given hyperparameter n. Merge all subspaces to form the complete value space. Assign integer numbers to the subspaces to generate a subspace partitioning scheme. For each set in the dataset, partition it into n subsets according to the aforementioned subspace partitioning scheme, assigning each subset the same integer number as its corresponding subspace. For each non-empty subset of each generated set, use set intersection... The dataset employs a set-based integrity verification technique, constructing intersection-based integrity verification auxiliary information for each subset. Each non-empty subset's integrity verification auxiliary information is treated as an independent computational task, with multiple tasks executed in parallel using multi-threading or multi-processing. A meta-set is created for each set in the dataset, and the integer indices of all non-empty subsets from the n generated subsets are added to the meta-set. The set intersection-based integrity verification technique is then used to generate integrity verification auxiliary information for the meta-set. Finally, the dataset, the meta-set's integrity verification auxiliary information, and the integrity verification auxiliary information of all generated subsets are passed to the query service module. The query service module is configured to respond to user-initiated requests for set intersection calculations, where each request consists of several query keywords. The query response includes: calculating the intersection of the sets corresponding to all query keywords and returning the intersection calculation result (Result); calculating the intersection of the metasets of all sets corresponding to the query keywords, generating a metaset intersection calculation integrity proof based on metaset integrity verification auxiliary information, and returning the metaset intersection calculation result (META) and the metaset intersection calculation integrity proof (Proof). META For each integer index i appearing in the intersection of the metasets, obtain the subset containing that integer index in the set corresponding to each query keyword, and generate a proof of the completeness of the subset intersection calculation. i The process of processing an integer number i is treated as an independent computational task. Multiple computational tasks are executed in parallel using multi-threading or multi-processing, and the integrity proof of the intersection of all generated subsets is returned. The process of generating the integrity proof of the intersection of subsets is as follows: First, the intersection X of the metasets corresponding to all query keywords is calculated. Based on the integrity verification auxiliary information of the involved metasets, an integrity proof is generated for the intersection of the metasets. For each number appearing in the intersection X of the metasets, the subset corresponding to that number is extracted from the set corresponding to the query keyword. Based on the integrity verification auxiliary information of the subset, an integrity proof is generated for the intersection of subsets with the same number. The intersection calculation and integrity proof generation processes between subsets with different numbers are independent of each other and are generated in parallel. The method for generating the integrity proof is determined by the intersection computation integrity verification technology adopted by the data owner module. The client module is configured to compute the integrity proof Proof based on the intersection of the returned metasets. META The integrity of the intersection calculation result META of the metasets is verified. Based on the intersection calculation result META and the integrity proofs of all subset intersection calculations, the integrity of the returned intersection calculation result Result is verified in parallel. If both verifications pass, the intersection calculation result is valid and output to the user; otherwise, if either verification fails, the user is informed that the calculation result Result or the integrity proof contains an error, and the received intersection calculation result is invalid. The parallel verification process for the integrity of the intersection calculation result Result is as follows: the intersection calculation result Result is divided into n subsets according to the subspace partitioning scheme of the data owner module, and each subset is assigned the same integer number as its corresponding subspace. It is checked whether the integer numbers of all non-empty subsets are included in META. If the integer number of any non-empty subset is not included in META, the verification fails. For each integer number i contained in META, the integrity proof of the subset intersection calculation received by the user is verified. i The integrity of the subset numbered i in the intersection calculation result Result is verified. If the verification process of any integer number i fails, the verification fails. The verification process of each integer number i is treated as an independent calculation task, and multiple calculation tasks are executed in parallel using a multi-threaded or multi-process mechanism.

7. A computer device, characterized in that, include: One or more processors; Memory; as well as One or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, wherein when the programs are executed by the processors, they implement the steps of the parallel computing method for verifying the integrity of set intersection computation as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the parallel computing method for verifying the integrity of set intersection calculation as described in any one of claims 1-5.

Images