Data processing method and apparatus

Abstract

Embodiments of the present specification provide a data processing method and device. The method is performed by any target party in multiple parties, comprising: first acquiring two table shards corresponding to two data tables, and a target table shard corresponding to any target table in the two data tables is one of multiple secret sharing shards of the target table; then, for the target table shard, based on a primary key column sub-shard contained in the target table shard and a corresponding generated key shard, a pseudo-random array sub-shard is determined for restoring a pseudo-random array; the target table shard is subjected to permutation processing based on the pseudo-random array to obtain a corresponding sorted table shard; and then, based on two sorted table shards corresponding to the two table shards, a connection table shard is determined, which is one of multiple secret sharing shards of a primary key connection table corresponding to the two data tables.

Description

Technical Field

[0001] This specification relates to the field of data processing technology, and in particular to a data processing method and apparatus. Background Technology

[0002] With the advent of the big data era, it has become a typical data distribution scenario where multiple data providers hold different characteristic data of the same business object. For example, merchants, banks, and payment platforms each hold large amounts of shopping orders, credit data, and payment data of the same users by providing user services. Aggregating the characteristic data held by various data providers and conducting joint analysis to obtain more comprehensive and accurate statistical information is an important way to effectively mine the value of data and significantly improve business quality and user satisfaction.

[0003] However, due to concerns about data privacy and security, data providers often cannot directly hand over their data to other participants for joint analysis. How to achieve joint analysis of multi-party data while ensuring that no party can obtain information other than the analysis results becomes a critical issue. Although general-purpose secure multiparty computation (MPC) technology, applicable to arbitrary circuit computation, provides a general solution to this problem, its communication and computational overhead often hinders its application in practical business. Designing specific MPC protocols for specific computational tasks or operators is the mainstream approach to solving practical problems.

[0004] In database query scenarios, a key operator involved in joint analysis is the equal join with primary keys operator, also known as the primary-key join operator. The primary-key join operator can extract rows from two tables that share the same primary key, generating a new table that contains all attribute values ​​of the common business object.

[0005] Therefore, it is necessary to design an MPC protocol for primary key join operators that can effectively reduce their communication and computational overhead, thereby reducing the communication and computational consumption of joint data analysis. Summary of the Invention

[0006] This specification describes one or more embodiments of a data processing method and apparatus that, by securely sorting unsorted secret-shared table shards according to the same rules, avoids high communication overhead caused by subsequent calls to the complete unintentional sorting operator, thereby effectively reducing communication and computational load.

[0007] According to a first aspect, a data processing method is provided, executed by any target party among multiple parties, comprising: obtaining two table shards corresponding to two data tables, wherein the target table shard corresponding to any target table in the two data tables is one of multiple secret-sharing shards of the target table. For the target table shard, based on its primary key column sub-shards and the corresponding generated key shards, a pseudo-random array shard is determined for recovering the pseudo-random array; a permutation process based on the pseudo-random array is performed on the target table shard to obtain the corresponding sorted table shards. Based on the two sorted table shards corresponding to the two table shards, a join table shard is determined, which is one of multiple secret-sharing shards of the primary key join table corresponding to the two data tables.

[0008] In one embodiment, obtaining two table shards corresponding to two data tables includes: obtaining the two table shards from two data parties other than the target party, wherein each of the two data parties holds partial attributes of multiple business objects to form the two data tables.

[0009] In one embodiment, before determining the pseudo-random array shards based on the primary key column sub-shards and the corresponding generated key shards contained therein, the method further includes: obtaining the key shards by jointly executing a key shard generation protocol with other parties among the multiple parties.

[0010] In one embodiment, performing a permutation process on the target table shard based on the pseudo-random array includes: obtaining a permutation vector, which is determined based on the pseudo-random array before and after sorting; and performing the permutation process on the target table shard using the permutation vector.

[0011] In one specific embodiment, the target table has a designated processing party; wherein, obtaining the permutation vector includes: when the target party is the designated processing party, recovering the pseudo-random array based on the generated pseudo-random array shards and other pseudo-random array shards received from other parties; sorting the pseudo-random array, and determining the permutation vector based on the position mapping relationship of the pseudo-random array before and after sorting.

[0012] In a more specific embodiment, determining the permutation vector includes: determining whether any two pseudo-random numbers in the pseudo-random array are different; and if the determination is yes, determining the permutation vector.

[0013] In a more specific embodiment, obtaining the permutation vector further includes: if the target is not the designated processor, sending the generated pseudo-random array fragments to the designated processor to restore the pseudo-random array, and determining the permutation vector based on the position mapping relationship of the pseudo-random array before and after sorting; and receiving the permutation vector from the designated processor.

[0014] In a more specific embodiment, the two data tables have different designated processing methods.

[0015] In one embodiment, before determining the join table shards, the method further includes: performing a verification protocol for the sorting of table shards jointly with other parties based on two sorted pseudo-random array shards and the two sorted table shards; wherein determining the join table shards includes: determining the join table shards if the output of the verification protocol indicates that the verification has passed.

[0016] In one embodiment, determining the join table shard based on two sorted table shards corresponding to the two table shards includes: vertically concatenating the two sorted table shards to obtain a concatenated table shard; performing an unintentional sort on the concatenated table shard according to the pseudo-random number shards corresponding to each row in the concatenated table shard, wherein the pseudo-random number shards belong to the union of two pseudo-random array shards; and determining the join table shard based on the result of the unintentional sort.

[0017] In one embodiment, before determining the pseudo-random array shards, the method further includes: obtaining operators parsed from a query request initiated by the client; wherein determining the pseudo-random array shards includes: determining the pseudo-random array shards if the operators include a primary key join operator; wherein, after determining the join table shards, the method further includes: determining query result shards based on the join table shards; and sending the query result shards to the client so that the client can recover the query results based on the query result shards and other query result shards.

[0018] According to the second aspect, a data processing method is provided, executed by the cloud, comprising: obtaining two table shards corresponding to two data tables, wherein the target table shard corresponding to any target table in the two data tables is one of multiple secret-sharing shards of the target table; for the target table shard, determining a pseudo-random array shard based on its primary key column sub-shards and the corresponding generated key shards, for recovering the pseudo-random array; performing a permutation process based on the pseudo-random array on the target table shard to obtain the corresponding sorted table shards; and determining a join table shard based on the two sorted table shards corresponding to the two table shards, which is one of multiple secret-sharing shards of the primary key join table corresponding to the two data tables.

[0019] According to a third aspect, a data processing apparatus is provided, the apparatus being integrated into any target party among the plurality of parties, the apparatus comprising: a table shard acquisition unit configured to acquire two table shards corresponding to two data tables, wherein the target table shard corresponding to any target table in the two data tables is one of a plurality of secret-sharing shards of the target table; a random number shard determination unit configured to, for the target table shard, determine a pseudo-random array shard based on its primary key column sub-shards and corresponding generated key shards, for recovering the pseudo-random array; a permutation processing unit configured to perform permutation processing on the target table shard based on the pseudo-random array to obtain a corresponding sorted table shard; and a join table shard determination unit configured to, based on the two sorted table shards corresponding to the two table shards, determine a join table shard, which is one of a plurality of secret-sharing shards of the primary key join table corresponding to the two data tables.

[0020] In one embodiment, the table shard acquisition unit is specifically configured to: acquire the two table shards from two data parties other than the target party, wherein each of the two data parties holds partial attributes of multiple business objects to form the two data tables.

[0021] In one embodiment, the apparatus further includes a key fragmentation generation unit configured to obtain the key fragments by jointly executing a key fragmentation generation protocol with other parties among the plurality of parties.

[0022] In one embodiment, the apparatus further includes a permutation vector acquisition unit configured to acquire a permutation vector, which is determined based on a pseudo-random array before and after sorting; the permutation processing unit is specifically configured to perform the permutation processing on the target table shards using the permutation vector.

[0023] In one specific embodiment, the target table has a designated processing party; wherein, the permutation vector acquisition unit is specifically configured to: when the target party is the designated processing party, recover the pseudo-random array based on the generated pseudo-random array shards and other pseudo-random array shards received from other parties; sort the pseudo-random array, and determine the permutation vector based on the position mapping relationship of the pseudo-random array before and after sorting.

[0024] In a more specific embodiment, the permutation vector acquisition unit is further configured to: determine whether any two pseudo-random numbers in the pseudo-random array are different; if the determination is yes, determine the permutation vector.

[0025] On the other hand, in a more specific embodiment, the permutation vector acquisition unit is specifically configured to: when the target party is not the designated processing party, send the generated pseudo-random array fragments to the designated processing party to restore the pseudo-random array, and determine the permutation vector based on the position mapping relationship of the pseudo-random array before and after sorting; and receive the permutation vector from the designated processing party.

[0026] In a more specific embodiment, the two data tables have different designated processing methods.

[0027] In one embodiment, the apparatus further includes a verification unit configured to jointly execute a verification protocol for the sorting of table shards with other parties based on two sorted pseudo-random array shards and the two sorted table shards; wherein, the join table shard determination unit is specifically configured to determine the join table shards if the output result of the verification protocol indicates that the verification has passed.

[0028] In one embodiment, the join table fragment determination unit is specifically configured as follows: vertically concatenating the two sorted table fragments to obtain a concatenated table fragment; randomly sorting the concatenated table fragment according to the pseudo-random number fragments corresponding to each row in the concatenated table fragment, wherein the pseudo-random number fragments belong to the union of the two pseudo-random array fragments; and determining the join table fragment based on the result of the random sorting.

[0029] In one embodiment, the apparatus further includes an operator acquisition unit configured to acquire operators parsed from a query request initiated by a client; wherein the random number sharding determination unit is specifically configured to: determine the pseudo-random array shards when the operators include a primary key join operator; the apparatus further includes a query processing determination unit configured to determine query result shards based on the join table shards; and send the query result shards to the client so that the client can recover the query results based on the query result shards and other query result shards.

[0030] According to a fourth aspect, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method of the first or second aspect.

[0031] According to a fifth aspect, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method of the first aspect or the second aspect.

[0032] By employing the methods and apparatus disclosed in the embodiments of this specification, and securely sorting unsorted secret-shared table fragments according to the same rules, high communication overhead caused by subsequent calls to the complete unintentional sorting operator can be avoided, thereby effectively reducing communication and computational load. Regarding security, this solution is provably secure under most honest and semi-trusted models, meaning it can resist attackers who control one participant and adhere to the protocol description. Attached Figure Description

[0033] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 This diagram shows an example of using the primary key join operator.

[0035] Figure 2 This diagram illustrates the execution data flow of the primary key connection protocol disclosed in the embodiments of this specification.

[0036] Figure 3 This diagram illustrates a data processing method flow chart disclosed in an embodiment of this specification.

[0037] Figure 4 This diagram illustrates the data processing implementation architecture disclosed in the embodiments of this specification.

[0038] Figure 5 This diagram illustrates the application architecture of the data processing disclosed in the embodiments of this specification in an e-commerce scenario;

[0039] Figure 6 This diagram illustrates the structure of the data processing apparatus disclosed in the embodiments of this specification. Detailed Implementation

[0040] The solution provided in this specification will now be described with reference to the accompanying drawings.

[0041] As mentioned above, MPC technology is an effective solution for joint analysis while protecting data privacy and security. The general execution process of MPC includes the following three steps: 1) Each data party shares its data with all computing parties through a secret sharing scheme. For intuitive description, let's denote all the data of all data parties as x; 2) Each computing party executes a specific protocol to convert the secret sharing value of the original data x (hereinafter referred to as the sharing value, secret sharing fragment, fragment, etc.) into the secret sharing value of the computation result f(x); 3) If necessary, f(x) can be recovered from the secret sharing value, or subsequent calculations can be performed based on the secret sharing value of f(x).

[0042] When the data held by the data provider is in the form of tables (such as tables in a database or Excel spreadsheets), the functions f mentioned above often include primary key join operators among one or more of the operators involved. Many joint analysis tasks require expansion on the newly joined tables. It should be noted that a primary key is one or more fields (or attributes) in a table. The primary key value is used to uniquely identify a record in the table; that is, different rows in the table will not have the same primary key value.

[0043] To facilitate understanding, we will first introduce the basic input and output of the primary key join operator. The input of the primary key join operator includes the secret sharing fragments [L] and [R] of two data tables L and R, where, This represents the primary key value of the i-th row in data table L. These are the other attributes of the i-th row;

[0044] The operator output is the secret sharing fragment [J] of the data join table J, [J] = ([K1,d1,d1′,f1],…,[K t ,d t ,d′ t ,f t Because the size of the join result needs to be hidden, the number of rows in J needs to be set to the maximum possible value, i.e., t = m + n. At this point, J contains invalid rows, i.e., rows not in the actual join result. The attribute f is used to distinguish between valid rows (f = 1) and invalid rows (f = 0). Specifically, the data join table J satisfies the following condition: for any set (j ∈ [n], l ∈ [m]) such that... Then there must exist one and only one i∈[t] such that f i =1, and For f i Rows with a value of 0 are not considered valid because their data is invalid.

[0045] Intuitively, Figure 1This diagram illustrates an example of using the primary key join operator. For example... Figure 1 As shown, the number of rows in both table shards [L] and [R] is 3, that is, n = m = 3; the connected data table shards include two valid rows.

[0046] The above introduces the basic concepts of primary key join operators. Regarding existing security protocols implementing primary key join operators, the inventors observed that when the secretly shared data table is not pre-sorted, the protocol execution requires a complete call to the unintentional sorting operator, resulting in significant communication and computational overhead. Specifically, the communication and computational overhead for unintentionally sorting a table shard [T] with n rows is O(n log n). 2 (n). Furthermore, the inventors discovered that the actual function of calling the unintentional operator is to place rows with the same primary key in the table shard [T] in adjacent positions for subsequent operations, rather than actually requiring them to be arranged alphabetically according to the plaintext of the primary key.

[0047] Therefore, the inventors propose a scheme to first securely and with low overhead sort the unsorted secret-sharing data tables [L] and [R] according to the same rule, and then perform other processing to complete the primary key join. For more details, see [link to relevant documentation]. Figure 2 This illustrates the execution data flow of the primary key connection protocol disclosed in the embodiments of this specification.

[0048] Specifically, in this scheme, a pseudorandom function (PRF) is used to transform the sorting of the secret sharing table shards into the sorting of pseudorandom values. Then, an efficient permutation protocol is used to complete the sorting of the table shards [L] and [R]. In this way, the large amount of overhead caused by using the complete operator of unintentional sorting can be avoided.

[0049] The implementation steps of the above solution are described below with reference to specific embodiments.

[0050] Figure 3 This diagram illustrates a data processing method disclosed in an embodiment of this specification. The method involves multiple computing parties (hereinafter referred to as "multiple parties") that execute a primary key join protocol based on shards of multiple data tables received from multiple data owners. It should be understood that the minimum number of data tables for a primary key join operation is two. For clarity, the following explanation primarily uses two data tables as an example to illustrate the execution of the primary key join protocol; the same logic applies to cases with a larger number of tables.

[0051] Any data party or computing party can be implemented as a device, platform, or cluster of devices with storage and computing capabilities; furthermore, any data party can simultaneously serve as one of multiple computing parties. For example, multiple data parties may consist of two data parties that also function as two computing parties, with a platform providing secure computing services serving as the other computing party. In this case, the above method involves two data parties and three computing parties. On the other hand, any number of computing parties among the multiple computing parties can be implemented in the cloud. In one implementation, one of the multiple computing parties is implemented in the cloud. In another implementation, the multiple computing parties are implemented as multiple computing nodes in the cloud, for example, specifically as multiple computing devices (such as multiple servers or multiple server clusters) in a cloud resource pool.

[0052] Figure 3 The method shown can be applied to any of the above-mentioned computational methods (hereinafter referred to as the target method). For example... Figure 3 As shown, the method includes the following steps:

[0053] Step S310: Obtain two table shards corresponding to the two data tables. The target table shard corresponding to any target table in the two data tables is one of the multiple secret-sharing shards of that target table. Step S320: For the target table shard, determine its corresponding sorted table shard. Specifically, this includes step S321: Based on the primary key column sub-shards and the corresponding generated key shards contained in the target table shard, determine a pseudo-random array shard for recovering the pseudo-random array; step S322: Obtain a permutation vector, which is determined based on the pseudo-random arrays before and after sorting; step S323: Use the permutation vector to perform permutation processing on the target table shard to obtain the corresponding sorted table shard. Step S330: Based on the two sorted table shards corresponding to the two table shards, determine a join table shard, which is one of the multiple secret-sharing shards of the primary key join table corresponding to the two data tables.

[0054] The steps above are explained in detail below:

[0055] Step S310: Obtain the two table shards corresponding to the two data tables.

[0056] Each of the two data tables mentioned above includes partial attributes of multiple business objects, held by two corresponding data parties. In one embodiment, the multiple business objects belong to one or more of the following categories: users, products, events, terminal devices, etc. In a specific embodiment, an event may be a login event, an access event, a complaint event, etc. In a specific embodiment, a terminal device may be a user terminal (such as a smartphone, wearable device) or an IoT terminal (such as a self-service checkout).

[0057] Each data table includes a primary key and other attributes besides the primary key. The field names (or attribute names) corresponding to the primary key in both data tables are the same, for example, both are business object IDs. It can be understood that the main difference between the two data tables lies in the other attributes. In one embodiment, the two data providers are a payment platform and an e-commerce platform, and the aforementioned multiple business objects represent multiple users. In this case, payment data such as payment time and payment amount collected by the payment platform forms the user's payment attributes, while shopping records such as order product names and order product amounts collected by the e-commerce platform form the user's shopping attributes.

[0058] Regarding the relationship between the target party and the two data parties, in one possible case, the target party is integrated into one of the data parties; in another possible case, the target party is independent of both data parties.

[0059] On the other hand, the target table shard corresponding to either of the two data tables (hereinafter referred to as the target table) is one of the multiple table shards obtained by the target data party through secret sharing of the target table.

[0060] To facilitate understanding, a brief introduction to secret sharing technology is provided. The (n,t)-secret sharing scheme includes the following two algorithms: (1) Sharing algorithm SS, which takes a plaintext message x as input and outputs n sharing values ​​[x] = ([x]1, [x]2, ..., [x]). n (2) Recovery algorithm Rec, which takes t shared values ​​as input to recover message x. The (n,t)-secret sharing scheme needs to satisfy correctness and security. Correctness means that any t shared values ​​can recover message x; security means that any t-1 shared values ​​will not leak any information about message x.

[0061] In this step, the target table corresponds to the plaintext message x, the total number of multiple computers corresponds to the number of shared values ​​n, and the secret sharing fragments of the target table correspond to the shared values. In a typical example, the (n,t)-secret sharing scheme can be instantiated as the (3,2)-replicated sharing scheme, which is implemented as follows: (1) SS(x), randomly generate (x1,x2,x3) such that x1+x2+x3=x, and output [x]1=(x1,x2), [x]2=(x2,x3) and [x]3=(x3,x1); (2) Rec([x] i [x] j The output is x = x1 + x2 + x3. The (3,2)-repeated sharing scheme supports highly efficient multiplication operations and is used by a variety of efficient and secure third-party computing protocols.

[0062] For clarity, the two table shards obtained will be denoted as [L] and [R], and the target table shard will be denoted as [Q]. It can be understood that Q = L or R.

[0063] From the above, the target can obtain two table shards [L] and [R] from the two data providers respectively. Then, in step S320, for any target table shard [Q], its corresponding sorted table shard [Q′] is determined. Specifically, this includes executing sub-steps S321, S322, and S323.

[0064] The following is combined Figure 4 This section will introduce the implementation of each sub-step in this process. Figure 4 This diagram illustrates the data processing implementation architecture disclosed in the embodiments of this specification. For visual clarity, it uses the case where both [L] and [R] have 3 rows as an example, and assigns specific values ​​to the primary key and other attributes, for example... Figure 4 The table shows the primary key column K in the partition [L]. L The slice value of the first row is [f].

[0065] In step S321, the sub-shards [K] are based on the primary key columns contained in the target table shard [Q]. Q ], and the corresponding generated key fragments [k], determine the pseudo-random array fragments [S] Q ], used to recover the pseudo-random array S Q .

[0066] In one embodiment, prior to this step, the method further includes: the target party, together with other parties in the multi-party group, executes a key fragmentation generation protocol to obtain the aforementioned key fragment [k], which is one of multiple secret-shared fragments of the random key k. It should be noted that the key fragmentation generation protocol refers to the MPC protocol used to generate fragments of a random key. MPC allows a group to perform joint computation without disclosing the private inputs of any participant. The participants agree on a function to be computed, and then apply the MPC protocol, inputting each person's secret inputs into the protocol, jointly computing the output of the function without leaking their private inputs. The construction of the key fragmentation generation protocol can be implemented using existing technologies and will not be elaborated further. Specifically, the target party can call the implementation operator F. keygen The protocol obtains a secret sharing of a random key k [k].

[0067] It should be noted that the target table includes a primary key column and other attribute columns; correspondingly, the target table shard [T] includes a primary key column sub-shard [K]. Q [and other attribute columns for sub-sharding. For example,] Figure 4 The table shows the primary key sub-fractions [K] L ] = ([f], [e], [a]), and [K R] = ([b],[a],[f]).

[0068] The pseudo-random number partitioning operator F PRF ([k],[X]), its output is ([PRF(k,x1)],…,[PRF(k,x1)],…,[PRF(k,x1)] n Accordingly, in this step, the implementation operator F is called. PRF The protocol uses key fragmentation [k] and primary key sub-fractionation [K]. Q [Input], calculate Forming pseudo-random array shards [S] Q ], where l equals [K Q The number of rows. In specific implementations, PRF can be implemented using block ciphers, such as LowMC. Regarding the implementation of block ciphers, the operator F... PRF This can be achieved using MPC protocols for Boolean circuits, such as the Yao's garbled circuit protocol or the GMW protocol.

[0069] In this step, sub-sharding is performed based on the primary key column [K]. Q [] and key fragments [k], determine pseudo-random array fragments [S] Q ].

[0070] For example, such as Figure 4 As shown, in step S41, the unintentional pseudo-random function operator F is applied. PRF , partition the table into [L] and [R] attributes [K] L ] and [K R Using [S] as input, the corresponding pseudo-random function output is calculated and used as a new attribute. L ] and [S R That is, F PRF ([k],[K L ])→[S L ] and F PRF ([k],[K R ])→[S R ].

[0071] From the above, we can obtain the pseudo-random array slice [S] corresponding to the target table slice [Q]. Q ].

[0072] Then, in step S322, the permutation vector π is obtained. Q It is based on the pseudo-random array S before and after sorting. Q And that's certain.

[0073] It should be noted that the target table Q has a designated processor, which can be understood as one of the aforementioned processors.

[0074] In one embodiment, this step may include: if the target is a specified processor, first sharding based on the pseudo-random array it generates [S] Q [and other pseudo-random array fragments received from other parties for the target table Q, to recover the pseudo-random array S] Q Then, the recovered pseudo-random array S Q Sort the array and determine the permutation vector π based on the mapping relationship between the pseudo-random array positions before and after sorting. Q In one specific embodiment, it is assumed that the pseudo-random array is sharded [S]. Q In the (3,2)-repeated sharing scheme, the target party can recover the plaintext random array S based on its generated pseudo-random array fragments and other pseudo-random array fragments of the corresponding target table received from any other party. Q In one specific embodiment, the recovered pseudo-random array S Q Perform sorting according to preset rules (such as alphabetical ascending or descending order).

[0075] In one specific embodiment, based on the pseudo-random array S before sorting... Q Each random number in the array is placed in the sorted pseudo-random array S. Q′ The position in the middle determines the permutation vector π. T It should be noted that a permutation of order m is a one-to-one mapping with both domain and range {1,…,m}; the permutation π can be characterized by an m-dimensional array (π(1),…,π(m)), and applying this permutation π to the m-dimensional sequence X=(x1,…,x m This yields a new sequence π·X=X′=(x1′,…,x m If X''), then for any i∈{1,…,m}, X'' holds. j =X π(j) For example, such as Figure 4 As shown, in step S42, the pseudo-random array before sorting includes CS, XS, and NS in sequence. CS is still ranked first after the pseudo-random array is sorted, XS is ranked third, and NS is ranked second. Thus, the permutation vector (1,3,2) can be obtained.

[0076] Furthermore, considering that the pseudo-random array corresponds to the primary key column, and that each primary key value in the primary key column is unique, the pseudo-random numbers in the pseudo-random array should also be unique. Therefore, to prevent malicious actions by other parties, the permutation vector π can be determined... Q Previously, the recovered random array S Q Perform verification. In one specific embodiment, determine the pseudo-random array S. Q Are any two pseudo-random numbers in the set both distinct? Furthermore, if yes, determine the permutation vector π.Q and π Q Send to other parties; otherwise, terminate the current process.

[0077] In another embodiment, when the target party is not the designated processor, the target party shards its generated pseudo-random array [S]. Q The permutation vector is sent to the designated processor, which then restores the pseudo-random array and determines it based on the mapping relationship between the positions of the pseudo-random array before and after sorting; subsequently, the permutation vector π is received from the designated processor. Q .

[0078] In one embodiment, the two data tables L and R have different designated processing parties. This prevents one party from simultaneously receiving two pseudo-random array plaintexts, thus avoiding the leakage of rows with the same primary key value. For example, as... Figure 4 As shown, in step S42, the pseudo-random array S L and S R The restoration, sorting, and permutation vector π L and π R The determination of the result is completed by P1 and P2 respectively, with P1 and P2 being two different computational parties. Furthermore, since step S42 is completed locally by P1 and P2, there is no communication overhead; simultaneously, since only plaintext operations are required, the computational overhead is also low.

[0079] From the above, the target side can obtain the permutation vector π for the target table partition [Q]. Q Then, in step S323, the permutation vector π is used... Q The target table partition [Q] is permuted to obtain the corresponding sorted table partition [Q′].

[0080] It should be noted that the permutation operator F shuffle2p The output of (π,[X]) is [π·X]. In this step, the MPC protocol that implements this permutation operator is invoked to partition the target table [Q] according to π. Q Perform a permutation to obtain the permutation result [π] Q ·Q], as a sorted table partition [Q′]. For example, as... Figure 4 As shown, in step S43, the table partitions [L] and [R] are processed according to π. L and π R Perform the substitution process, that is, execute operator F. shuffle2p (π L ,[L]) and F shuffle2p (π R [R]), at this point, the permuted [L′] and [R′] are equivalent to those based on [S] L ] and [S R Sort them in order.

[0081] From the above, we can obtain the sorted table partition [Q′] corresponding to the target table partition [Q]. Accordingly, we can obtain the two sorted table partitions [L′] and [R′] corresponding to the two table partitions [L] and [R].

[0082] Then in step S330, based on the two sorted table shards [L′] and [R′], the join table shard [J] is determined, which is one of the multiple secret-sharing shards of the primary key join table J corresponding to the two data tables L and R.

[0083] It should be noted that a verification step can be added before this step, whereby the target party and all other computing parties jointly verify whether [L′] and [R′] conform to attribute [S]. L ] and [S R The sorting is completed, thereby enhancing the security of this protocol. Specifically, in one embodiment, prior to this step, the method further includes: slicing based on two sorted pseudo-random arrays [S] L′ ] and [S R′ ], and two sorted table shards [L′] and [R′], together with other parties, execute a verification protocol for the sorting of table shards; accordingly, this step includes: if the output of the verification protocol indicates that the verification is successful, determining the join table shard [J].

[0084] On the other hand, existing primary key join protocols incur significant computational overhead because they require calling the complete unintentional sorting operator for unsorted data table shards. In the primary key join protocol disclosed in the embodiments of this specification, table shards [L] and [R] are sorted first, and then unintentional sorting is performed based on the sorted [L′] and [R′], thereby effectively reducing the overall communication and computational overhead.

[0085] Specifically, first, vertical concatenation is performed based on the two sorted table partitions [L′] and [R′] to obtain the concatenated table partition. It should be understood that the number of rows increases after vertical concatenation. Then, based on the pseudo-random number partitions [s] corresponding to each row in the concatenated table partition, the concatenated table partition is randomly sorted. The pseudo-random number partitions [s] belong to the two pseudo-random array partitions [S]. L ] and [S R The union of ]; then, based on the result of the random sorting, the join table sharding is determined [J].

[0086] It should be noted that the aforementioned unintentional sorting can invoke existing protocol implementations of the operator, and the determination of join table fragments based on the unintentional sorting result [J] can also be implemented based on relevant steps in existing primary key join protocols. In a typical implementation, the sort-compare-shuffle (SCS) protocol can be invoked based on the sorted table fragments [L′] and [R′]. During the execution of this protocol, the sorting by primary key K, which was originally included when invoking the unintentional sorting operator, is replaced by sorting by the pseudo-random number s corresponding to each row of the data table. Thus, by executing the SCS protocol and invoking a portion of the unintentional sorting operator instead of the complete operator, join table fragments can be obtained with low communication and low computational overhead [J]. It should be noted that the SCS protocol was proposed by Huang et al. in the paper "private setintersection: are garbled circuits better than custom protocols".

[0087] For ease of understanding, the following is an exemplary description of the complete process of determining the join table partition [J] based on two sorted table partitions [L′] and [R′].

[0088] (1) Add an attribute [s] to both the table partitions [L′] and [R′], corresponding to the pseudo-random number partitions mentioned above.

[0089] (2) Add another attribute [g] to the table partitions [L′] and [R′]. b The value of this attribute is 1 for each row in table shard [R′], and 0 for each row in table shard [L′].

[0090] (3) Vertically concatenate table partitions [L′] and [R′] into table partition [T]; then apply the unintentional sorting operator to sort table partition [T]: Implement a list that first sorts the list in descending order by attribute s, and then sorts it by attribute g. b Arranged in ascending order.

[0091] (4) For each row [T], follow the steps below. i Add attribute f b :

[0092] [T i [f b ]]=[T i [g b ]]⊙([T i-1 [s]] = ? [T i [s]])

[0093] Among them, T i [fb ]、T i [g b ] and T i [s] represents T i The attribute value f in the row b g b and s, T i-1 [s] represents T i-1 The attribute value s in the row; ⊙ represents a logical AND operation, =? determines whether the left and right values ​​are equal. If they are equal, the result is 1; otherwise, it is 0.

[0094] (5) Add the attribute d′ to each row [T] as follows:

[0095] [T i [d′]]=[T i [f b ]]·[T i-1 [d]

[0096] (6) Reorder the rows of table [T]:

[0097] First call operator F rand This yields a secret share of a random permutation π <π>. In F rand During protocol execution, multiple permutations whose result after permutation superposition is equal to the random permutation π can be randomly selected from the set of all permutations of order m (values ​​being the row number of table [T]) as multiple superposition permutation vectors. Then, multiple vectors are selected from these multiple superposition permutation vectors and combined to obtain the random permutation fragment <π>. It should be noted that the definition and properties of permutation superposition include: the set of all permutations of order m constitutes a non-commutative group, which has a binary operation ◇, where the result of any two permutations π and σ is π◇σ=(π(σ(1)),…,π(σ(m))), and any permutation π has an inverse permutation π. -1 , so that π◇π -1 = (1,…,m).

[0098] For example, assuming the number of computational sides is 3, correspondingly, in F rand During the execution of the protocol, three permutations π1, π2, and π3 can be randomly selected such that π1◇π2◇π3=π, thus obtaining three random permutation fragments (π1,π2), (π2,π3), and (π3,π1). The target party can obtain one of these three fragments.

[0099] Then, the operator F is applied. shuffle (<π>,[T]) , the table [T] is permuted according to the random permutation <π> to obtain [π·T], which is used as the join table fragment [J], thus completing the scrambling.

[0100] Based on the above, the join table shard [J] corresponding to the two data table shards [L] and [R] can be determined.

[0101] According to another embodiment, considering that in a database query scenario, the aforementioned multiple computing parties can pre-obtain table shards of two data tables, and then, based on the operator corresponding to the query request sent by the client, the multiple computing parties call the MPC protocol corresponding to the operator to perform calculations, thereby enabling the client to obtain multiple query result shards from multiple computing parties and restore the query results.

[0102] Therefore, in one embodiment, before step S320, the method further includes: obtaining the operator parsed from the query request initiated by the client; correspondingly, step S320 may include: determining the sorted table shard [Q′] corresponding to the target table shard [Q] when the parsed operator includes a primary key join operator; after step S330, the method further includes: determining the query result shard based on the join table shard [J]; sending the query result shard to the client so that the client can recover the query result based on the query result shard and other query result shards received from other parties.

[0103] In summary, by employing the data processing method disclosed in the embodiments of this specification, the unsorted secret-shared table fragments can be securely sorted according to the same rules, thus avoiding high communication overhead caused by subsequent calls to the complete sorting operator, thereby effectively reducing communication and computational load.

[0104] Specifically, in this scheme, (1) the primary key values ​​[K] that need to be sorted are... L ] and [K R Convert [S] to the corresponding pseudo-random function value. L ] and [S R (2) The plaintext attribute S is determined by two participants, P1 and P2. L and S R Sort the data and define the permutation π based on the sorting result. L and π R (3) Apply the substitution protocol again so that tables [L] and [R] complete according to [S] L ] and [S RThe sorting of [L] and [R] is performed. Step (1) has a computation and communication overhead of O(n); Step (2) is completed locally by P1 and P2, incurring no communication overhead, and its computation overhead is also low since it only operates on plaintext; Step (3) has a communication and computation overhead of O(n). Finally, using the primary key connection protocol for the sorted [L] and [R] results in a communication and computation overhead of O(n log n). Therefore, this scheme achieves a communication and computation overhead of only O(n log n) for the unsorted [L] and [R]. Furthermore, the two additional verification operations mentioned above for malicious security also only incur an overhead of O(n).

[0105] In terms of security, this scheme is provably secure under the majority honest and semi-trusted model, meaning it can resist attackers who control one participant and follow the protocol description. If the protocols implementing each operator invoked by this scheme are secure under the majority honest and malicious model, this scheme, after adding the two additional verification operations mentioned above, also possesses provable security under that model.

[0106] The following section will introduce the application of this solution in the context of e-commerce. Figure 5 This diagram illustrates the application architecture of the data processing disclosed in the embodiments of this specification in an e-commerce scenario. This application architecture involves multiple data parties, including merchants, brand owners, and e-commerce platforms, each holding partial attributes of multiple users and / or multiple products. For example, the merchant's data table includes merchant customer service records, product purchase prices, user IDs, etc.; the brand owner's data table includes product costs, product origins, merchant IDs, wholesale quantities, user IDs, etc.; and the e-commerce platform's data table includes user IDs, user genders, ages, mailing addresses, browsing history, product categories, order numbers, etc. Each data party uses the user ID as the primary key of its data table. Furthermore, any data party in this scenario can simultaneously act as one of multiple computation parties.

[0107] like Figure 5 As shown, each calculation method P i The system receives corresponding table shards from multiple data providers. Then, based on two table shards received from two of these data providers (e.g., merchants and brand owners), it jointly executes a primary key join protocol with all other computational providers to obtain a joined table shard. The protocol execution process includes: for each of the two table shards, determining a pseudo-random array shard based on its primary key column sub-shards and the corresponding generated key shards, used to recover the pseudo-random array; next, obtaining a permutation vector, determined based on the pseudo-random array before and after sorting; using the permutation vector to permutate the table shard, obtaining the corresponding sorted table shard; and finally, based on the two sorted table shards corresponding to the two table shards, determining the joined table shard, which is one of multiple secret-sharing shards in the primary key joined table corresponding to the two data tables.

[0108] Furthermore, when the number of data parties mentioned above is greater than 2, the calculation method P is... i After obtaining the aforementioned join table shards, based on these join table shards and table shards received from another data provider (such as an e-commerce platform), the primary key join protocol is executed together with all other computational parties to obtain another join table shard. This process continues until the primary key joins of all table shards are completed. Furthermore, computational party P... i Subsequent analysis and processing can be performed based on the join table shards obtained from the last execution of the primary key join protocol, such as calculating the shards of query results for the primary key join table.

[0109] As a result, secure multi-party computation can be achieved in e-commerce scenarios, enabling data providers such as merchants, brand owners, and e-commerce platforms to protect the privacy of user and product data held by each party.

[0110] Corresponding to the above-described primary key connection method, embodiments of this specification also disclose a primary key connection device. Figure 6 This diagram illustrates the structure of a data processing apparatus disclosed in an embodiment of this specification. The apparatus 600 is integrated into any target party among the plurality of parties. The apparatus 600 includes:

[0111] The table shard acquisition unit 610 is configured to acquire two table shards corresponding to two data tables, wherein the target table shard corresponding to any target table in the two data tables is one of the multiple secret-sharing shards of the target table; the random number shard determination unit 620 is configured to determine a pseudo-random array shard for the target table shard based on its primary key column sub-shards and the corresponding generated key shards, for recovering the pseudo-random array; the permutation processing unit 630 is configured to perform permutation processing on the target table shard based on the pseudo-random array to obtain the corresponding sorted table shards; the join table shard determination unit 640 is configured to determine a join table shard based on the two sorted table shards corresponding to the two table shards, which is one of the multiple secret-sharing shards of the primary key join table corresponding to the two data tables.

[0112] In one embodiment, the table shard acquisition unit 610 is specifically configured to: acquire the two table shards from two data parties other than the target party, wherein each of the two data parties holds partial attributes of multiple business objects to form the two data tables.

[0113] In one embodiment, the apparatus 600 further includes a key fragmentation generation unit 650 configured to obtain the key fragments by jointly executing a key fragmentation generation protocol with other parties among the plurality of parties.

[0114] In one embodiment, the apparatus 600 further includes a permutation vector acquisition unit 660, configured to acquire a permutation vector, which is determined based on a pseudo-random array before and after sorting; the permutation processing unit 630 is specifically configured to perform the permutation processing on the target table shards using the permutation vector.

[0115] In one specific embodiment, the target table has a designated processing party; wherein, the permutation vector acquisition unit 660 is specifically configured to: when the target party is the designated processing party, recover the pseudo-random array based on the generated pseudo-random array shards and other pseudo-random array shards received from other parties; sort the pseudo-random array, and determine the permutation vector based on the position mapping relationship of the pseudo-random array before and after sorting.

[0116] In a more specific embodiment, the permutation vector acquisition unit 660 is further configured to: determine whether any two pseudo-random numbers in the pseudo-random array are different; if the determination is yes, determine the permutation vector.

[0117] On the other hand, in a more specific embodiment, the permutation vector acquisition unit 660 is specifically configured to: when the target party is not the designated processing party, send the generated pseudo-random array fragments to the designated processing party to restore the pseudo-random array, and determine the permutation vector based on the position mapping relationship of the pseudo-random array before and after sorting; and receive the permutation vector from the designated processing party.

[0118] In a more specific embodiment, the two data tables have different designated processing methods.

[0119] In one embodiment, the apparatus further includes a verification unit 670, configured to jointly execute a verification protocol for the sorting of table shards with other parties based on two sorted pseudo-random array shards and the two sorted table shards; wherein, the connection table shard determination unit is specifically configured to determine the connection table shards if the output result of the verification protocol indicates that the verification has passed.

[0120] In one embodiment, the join table fragment determination unit 640 is specifically configured to: vertically concatenate the two sorted table fragments to obtain a concatenated table fragment; perform random sorting on the concatenated table fragment according to the pseudo-random number fragments corresponding to each row in the concatenated table fragment, wherein the pseudo-random number fragments belong to the union of the two pseudo-random array fragments; and determine the join table fragment based on the result of the random sorting.

[0121] In one embodiment, the apparatus further includes an operator acquisition unit 680, configured to acquire operators parsed from a query request initiated by a client; wherein the random number sharding determination unit is specifically configured to: determine the pseudo-random array shards when the operators include a primary key join operator; the apparatus further includes a query processing determination unit 690, configured to determine query result shards based on the join table shards; and send the query result shards to the client so that the client can recover the query results based on the query result shards and other query result shards.

[0122] In summary, by employing the data processing apparatus disclosed in the embodiments of this specification, the unsorted secret-shared table fragments can be securely sorted according to the same rules, thus avoiding high communication overhead caused by subsequent calls to the complete sorting operator, thereby effectively reducing communication and computational load.

[0123] According to another embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed in a computer, causes the computer to perform a combination Figure 3 The method described.

[0124] According to another embodiment, a computing device is also provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements a combination... Figure 3 The method described.

[0125] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in this invention can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium.

[0126] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. A data processing method, executed by any target party among multiple parties, comprising: Obtain two table shards corresponding to two data tables, wherein the target table shard corresponding to any target table in the two data tables is one of the multiple secret-sharing shards of that target table; For the target table shard, based on the primary key column sub-shards it contains and the corresponding generated key shards, a pseudo-random array shard is determined to recover the pseudo-random array; the target table shard is then subjected to a permutation process based on the pseudo-random array to obtain the corresponding sorted table shards. Based on the two sorted table shards corresponding to the two table shards, the join table shard is determined, which is one of the multiple secret-sharing shards of the primary key join table corresponding to the two data tables; The step of performing a permutation process on the target table shard based on the pseudo-random array includes: obtaining a permutation vector and using the permutation vector to perform the permutation process on the target table shard; the permutation vector is determined based on the mapping relationship between the positions of the pseudo-random array before and after sorting, and the pseudo-random array is recovered from the pseudo-random array shard determined by multiple parties.

2. The method according to claim 1, wherein, Retrieve the two table shards corresponding to the two data tables, including: The two table shards are obtained from two data sources, each of which holds partial attributes of multiple business objects, forming the two data tables.

3. The method according to claim 1, wherein, Before determining the pseudo-random array shards based on the primary key column sub-shards and the corresponding generated key shards, the method further includes: The key fragments are obtained by jointly executing a key fragmentation generation protocol with the other parties in the multi-party group.

4. The method according to claim 1, wherein, The target table has a specified processing method; wherein, obtaining the permutation vector includes: If the target party is the designated processor, the pseudo-random array is recovered based on the generated pseudo-random array shards and other pseudo-random array shards received from other parties; The pseudo-random array is sorted, and the permutation vector is determined based on the position mapping relationship of the pseudo-random array before and after sorting.

5. The method according to claim 4, wherein, Obtaining the permutation vector also includes: If the target is not the designated processor, the generated pseudo-random array is fragmented and sent to the designated processor to restore the pseudo-random array, and the permutation vector is determined based on the position mapping relationship of the pseudo-random array before and after sorting. Receive the permutation vector from the designated processor.

6. The method according to claim 4, wherein, The two data tables have different designated processing methods.

7. The method according to claim 1, wherein, Before determining the join table fragments, the method further includes: Based on the two sorted pseudo-random array shards and the two sorted table shards, a verification protocol for the sorting of table shards is jointly executed with other parties. Determining the join table shards includes: If the output of the verification protocol indicates that the verification has passed, the join table is determined to be fragmented.

8. The method according to claim 1, wherein, Based on the two sorted table partitions corresponding to the two table partitions, the join table partitions are determined, including: Based on the two sorted table fragments, vertical concatenation is performed to obtain the concatenated table fragments; Based on the pseudo-random number fragments corresponding to each row in the splicing table fragments, the splicing table fragments are randomly sorted, and the pseudo-random number fragments belong to the union of two pseudo-random array fragments; Based on the result of the unintentional sorting, the join table shards are determined.

9. The method according to claim 1, wherein, Before determining the pseudo-random array shards, the method further includes: obtaining the operators parsed from the query request initiated by the client; The process of determining the pseudo-random array shards includes: determining the pseudo-random array shards when the operator includes a primary key join operator; After determining the join table shards, the method further includes: determining query result shards based on the join table shards; and sending the query result shards to the client so that the client can restore the query results based on the query result shards and other query result shards.

10. A data processing method, executed by the cloud, comprising: Obtain two table shards corresponding to two data tables, wherein the target table shard corresponding to any target table in the two data tables is one of the multiple secret-sharing shards of that target table; For the target table shard, based on the primary key column sub-shards it contains and the corresponding generated key shards, a pseudo-random array shard is determined to recover the pseudo-random array; the target table shard is then subjected to a permutation process based on the pseudo-random array to obtain the corresponding sorted table shards. Based on the two sorted table shards corresponding to the two table shards, the join table shard is determined, which is one of the multiple secret-sharing shards of the primary key join table corresponding to the two data tables; The step of performing a permutation process on the target table shard based on the pseudo-random array includes: obtaining a permutation vector and using the permutation vector to perform the permutation process on the target table shard; the permutation vector is determined based on the mapping relationship between the positions of the pseudo-random array before and after sorting, and the pseudo-random array is recovered from the pseudo-random array shard determined by multiple parties.

11. A data processing apparatus, said apparatus being integrated into any target party among multiple parties, said apparatus comprising: The table shard acquisition unit is configured to acquire two table shards corresponding to two data tables, wherein the target table shard corresponding to any target table in the two data tables is one of the multiple secret-sharing shards of the target table; The random number sharding determination unit is configured to determine the pseudo-random array shards based on the primary key column sub-shards and the corresponding generated key shards contained in the target table shards, for the purpose of recovering the pseudo-random array; The permutation processing unit is configured to perform permutation processing on the target table partition based on the pseudo-random array to obtain the corresponding sorted table partition; The permutation vector acquisition unit is configured to acquire a permutation vector; the permutation vector is determined based on the mapping relationship between the positions of the pseudo-random array before and after sorting, and the pseudo-random array is recovered from the fragmentation of the pseudo-random array determined by multiple parties; The permutation processing unit is specifically configured to perform the permutation processing on the target table partition using the permutation vector. The join table shard determination unit is configured to determine the join table shard based on two sorted table shards corresponding to the two table shards, which is one of the multiple secret-sharing shards of the primary key join table corresponding to the two data tables.

Images