Super-threshold data set intersection method and system for secure information processing

Abstract

The present application belongs to the technical field of information security, and particularly relates to a super-threshold data set intersection method and system for secure information processing. The method comprises the following steps: S1, each participant performs encoding and decoding of an oblivious key-value store (OKVS) with other participants; S2, each participant constructs a hash table according to the decoding result and its own elements; and S3, the constructed hash table is sent to a designated party, and the designated party collects information in the hash tables of all participants, and finally obtains a super-threshold intersection result by using a secret sharing scheme. The present application has the characteristics of reducing hardware and operation and maintenance costs, enhancing data security, and improving computing efficiency, so as to meet the industrial control demand of short-time response.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of information security, and particularly relates to a super-threshold data set intersection method and system for secure information processing. BACKGROUND

[0002] MPSI (Multi-party Private Set Intersection) is one of the core technologies of secure multi-party computation (MPC), which allows multiple participants to jointly compute the intersection of these sets without revealing their private data sets, and does not reveal any additional information except the intersection result. Its research significance lies in solving the problem of data island. MPSI is mostly based on cryptographic primitives (such as oblivious transfer OT, oblivious pseudo-random function OPRF, homomorphic encryption, etc.) to build schemes, but compared with two-party PSI, it needs to deal with more complex challenges such as communication round explosion and malicious collusion attacks, and needs to balance efficiency and security. In real-world applications, MPSI is a key infrastructure supporting data collaboration in high-sensitive fields, and plays an irreplaceable role in promoting the safe circulation of data elements.

[0003] Super-threshold private intersection technology is a variant of multi-party private intersection: for a specific threshold k, it allows n participants to jointly compute elements that appear in at least k participant sets (k < n), and only the specified party knows the result. During the interaction, the specified party will not expose any other participant's elements except the super-threshold elements. Except for the specified party knowing the final super-threshold result, no participant will know any information about the other participant's set elements. This type of super-threshold protocol has many application scenarios in life.

[0004] Although the super-threshold private set intersection protocol meets the requirements of multi-party privacy protection in design, it has certain limitations in efficiency and security due to its reliance on homomorphic encryption technology and untrusted third parties.

[0005] In the research of existing super-threshold PSI technology, the homomorphic operation problem and the combinatorial explosion problem constitute a significant technical bottleneck. Traditional schemes such as the polynomial intersection method based on Paillier encryption require all participants to participate in distributed decryption, polynomial reconstruction, and encryption operations throughout the process, resulting in a linear increase in computing and communication load with the number of participants, forming a serious system scalability bottleneck. This mandatory real-time collaboration not only introduces high latency, but also increases the risk of protocol interruption due to node instability in dynamic network environments.

[0006] Secondly, the existing threshold PSI technology also has a combinatorial explosion problem. Unlike the traditional combinatorial explosion problem (which needs to perform Compared to the independent intersection operation, the method used in the d-and-over MPSI protocol improves efficiency by algebraically expressing threshold conditions and performing aggregation calculations, transforming the traditional exponential complexity of combinatorial enumeration into linear operations. However, it requires repeated execution of homomorphic encryption, polynomial reconstruction, or Bloom filter (BF) aggregation for each operation, resulting in high computational complexity.

[0007] Furthermore, while existing unbalanced d-and-over MPSI protocols improve efficiency, they rely on an untrusted third party, D. While D cannot decrypt the data, it can reveal the set size (via the length of the BF array) and the number of participants. In practical applications, this could potentially lead to the inference of sensitive information such as business scale. In addition, each participant in this scheme can only obtain elements from their own set that appear at least d times. This limits its applicability to scenarios requiring individual responses (such as savings institutions freezing high-risk accounts) but not to scenarios requiring macro-level analysis, such as government agencies coordinating pandemic tracking.

[0008] Therefore, it is of great importance to design a method and system for finding intersections of super-threshold data sets for secure information processing that can reduce hardware and maintenance costs, enhance data security, and improve computing efficiency to meet the needs of industrial control with short response times. Summary of the Invention

[0009] This invention aims to overcome the problems of existing super-threshold multi-set intersection techniques, such as single point of failure, easy data leakage, and large computational load, which make them unsuitable for high-latency industrial scenarios. It provides a super-threshold data set intersection method and system for secure information processing that can reduce hardware and maintenance costs, enhance data security, and improve computational efficiency to meet the needs of industrial control with short response times.

[0010] To achieve the above-mentioned objectives, the present invention adopts the following technical solution:

[0011] A method for finding the intersection of data sets exceeding the threshold for security information processing includes the following steps;

[0012] S1, each participant performs unintentional key-value pair storage OKVS encoding and decoding with other participants;

[0013] S2, each participant constructs a hash table based on the decoding result and its own elements;

[0014] S3 sends the constructed hash table to the designated party, which then aggregates the information from all participating parties' hash tables and uses a secret sharing scheme to finally obtain the super-threshold intersection result.

[0015] Preferably, step S1 includes the following steps:

[0016] S11, Settings It is a calculation participant, Participants The set, Represents the i-th participant The collection of possessions The j-th element in Represents a set Size;

[0017] S12, each participant OKVS encoding is performed using custom elements, as follows:

[0018] Use your own elements As the key, construct a (k-1)th order polynomial. And calculate a polynomial value. :

[0019] ;

[0020] ;

[0021] in, Generated by the pseudo-random number generator PRG, i.e. ;

[0022] Ultimately, each participating user manages their own element. Each constructs a key-value pair ;

[0023] S13, A data structure is obtained by performing OKVS encoding on the constructed key-value pairs. ,then Will Send to all other participants;

[0024] S14, Use your own elements Other participants The data structure sent Perform OKVS decoding, and record the decoding result as... : , ( yes In addition to Identifiers of all other participants, because To decode the data structures sent by all other participants besides oneself, therefore The range of values ​​is the range from 1 to n excluding i.

[0025] Preferably, step S2 includes the following steps:

[0026] S21, Settings Construct a hash table of length B. ,in , The load factor, representing the hash table, is a pre-defined constant used to control and balance the space efficiency and time efficiency of the hash table. ;

[0027] S22, Choose a hash function that is the same as the number of participants, n. This is used to ensure that each element corresponds to n positions;

[0028] S23, Insert the data corresponding to the element into the hash table. The specific interpolation and construction methods are as follows:

[0029] Participants calculate , corresponding to n position indices in the hash table, used for storing and Relevant data;

[0030] For each element Initialize an empty index set of size n. ,Will Add them in ascending order. middle;

[0031] Will be with each element The relevant data is stored in Specifically:

[0032] Each Data pairs are stored in the corresponding middle, Hash table The One bucket:

[0033] ;

[0034] Similarly, Save to middle, Hash table The One bucket:

[0035] .

[0036] Preferably, step S3 includes the following steps:

[0037] S31, Settings To obtain the final result, Each Send to ;

[0038] S32, Use your own hash table One bucket The data in the same bucket as the other n-1 hash tables The data in the middle is used as secret fragments to recover the secret value s; where, ;

[0039] S33, verify whether the polynomial value corresponding to the secret value is the polynomial value used for recovery. If it is, it means that the secret value is a super-threshold intersection. If it is not, it means that the secret value is not a super-threshold intersection.

[0040] Preferably, step S32 includes the following steps:

[0041] S321, Initialize a set Y;

[0042] S322, the secret recovery process is as follows:

[0043] Choose a large prime number p such that p > n and p > n. ;

[0044] Using k label points (in yes All hash tables owned For data pairs stored in the same bucket, Lagrange interpolation is performed by first calculating the Lagrange basis functions. (i=1,2,…,n);

[0045] ;

[0046] The final recovered polynomial is:

[0047] ;

[0048] When x=0, .

[0049] Preferably, step S33 includes the following steps:

[0050] S331, Verification process:

[0051] For k label points First calculate k Check each calculated Is it equal to If they are equal, then s is a super-threshold intersection and s is added to set Y; if they are not equal, then s is not a super-threshold intersection and no further operation is performed.

[0052] This invention also provides a system for finding intersections of data sets exceeding a threshold for security information processing, comprising:

[0053] The OKVS encoding and decoding module is used to enable each participant to perform unintentional key-value pair storage OKVS encoding and decoding with other participants;

[0054] The hash table construction module enables each participant to construct a hash table based on the decoding result and its own elements;

[0055] The aggregation and recovery module is used to send the constructed hash table to a designated party, which then aggregates the information from all participating parties' hash tables and uses a secret sharing scheme to finally obtain the super-threshold intersection result.

[0056] Compared with existing technologies, the advantages of this invention are: (1) This invention abandons homomorphic encryption operations and instead uses distributed hashing to directly perform set operations and OKVS encoding and decoding in the hash space, avoiding expensive homomorphic operations. Homomorphic encryption requires high-performance hardware support (such as GPU / ASIC), and the use of homomorphic technology can rely on general-purpose servers, which greatly reduces hardware and maintenance costs; (2) This invention eliminates the dependence on untrusted third parties (UTP), enabling all parties to cooperate equally and avoiding the risk of data centralization. Traditional UTP-dependent schemes need to centralize data to a third party (such as cloud service providers), which poses a single point of leakage risk. This invention ensures that the data of each party is always kept locally through distributed computing, and only exchanges encrypted shares or intermediate results. The security and confidentiality of data information are enhanced, and the original data is not easily leaked; (3) This invention combines hash tables with mapping order transformation, transforming the threshold condition of "appearing in at least t sets" into the problem of recovering secret fragments in the hash bucket, avoiding the exponential complexity of traditional combination enumeration (such as (4) This invention enhances the applicability of high-latency scenarios because higher computational complexity may be difficult to meet the real-time data requirements in practical applications. After adopting a more lightweight computational scheme, the efficiency will be greatly improved, meeting the industrial control requirements of shorter response time. Attached Figure Description

[0057] Figure 1 This is a schematic diagram of a method for finding the intersection of data sets exceeding the threshold used in security information processing in this invention. Detailed Implementation

[0058] To more clearly illustrate the embodiments of the present invention, specific implementation methods will be described below with reference to the accompanying drawings. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings and other implementation methods can be obtained based on these drawings without any creative effort.

[0059] This invention provides a method for finding the intersection of data sets exceeding a threshold for security information processing, comprising the following steps;

[0060] 1. Each participant performs unintentional key-value pair storage with other participants using OKVS encoding and decoding (this step is to check whether an element is in the other participants' sets; only if the element is in the other party's set will the corresponding secret fragment be obtained, otherwise a random value is obtained. However, due to the randomness of OKVS decoding, the participants themselves cannot distinguish between them, which reflects the security of the scheme, as it is impossible to infer the information of other participants from the decoding result).

[0061] 2. Each participant constructs a hash table based on the decoding result and its own elements;

[0062] 3. Send the constructed hash table to the designated party, which then aggregates the information from all participating parties' hash tables and uses a secret sharing scheme to finally obtain the super-threshold intersection result.

[0063] The method of this invention involves n parties, and its objective is to obtain elements that exist in at least k sets of parties among the n parties. The final result is obtained by a designated party. The following describes the process in conjunction with... Figure 1 The detailed implementation steps of the method of the present invention are as follows:

[0064] Step 1:

[0065] 1-1, Setting It is a calculation participant, Participants The set, Represents the i-th participant The collection of possessions The j-th element in Represents a set Size:

[0066] , , ;

[0067] 1-2, each participant OKVS encoding is performed using custom elements, as follows:

[0068] Use your own elements As the key, construct a (k-1)th order polynomial. And calculate a polynomial value. :

[0069] ;

[0070] ;

[0071] in, Generated by the pseudo-random number generator PRG, i.e. ;

[0072] Ultimately, each participating user manages their own element. Each constructs a key-value pair ;

[0073] 1-3, A data structure is obtained by performing OKVS encoding on the constructed key-value pairs. ,then Will Send to all other participants;

[0074] 1-4, Use your own elements Other participants The data structure sent Perform OKVS decoding, and record the decoding result as... : , . ( yes In addition to Identifiers of all other participants, because To decode the data structures sent by all other participants besides oneself, therefore The range of values ​​is the range from 1 to n excluding i.

[0075] Based on OKVS's random decryption property, the result of OKVS decoding is indistinguishable from a random number. This means an attacker cannot deduce which keys were encoded into OKVS by observing the decode output. Therefore, the decoding user cannot distinguish whether the decoded value is a random value or the key-value pair.

[0076] if Then you will get the corresponding key-value pair. Otherwise, a random number will be obtained.

[0077] Step 2:

[0078] 2-1, Setting Construct a hash table of length B. ,in , The load factor, representing the hash table, is a pre-defined constant used to control and balance the space efficiency and time efficiency of the hash table. ;

[0079] 2-2, Choose a hash function that is the same as the number of participants, n. This is used to ensure that each element corresponds to n positions;

[0080] 2-3, Insert the data corresponding to the element into the hash table. The specific interpolation and construction methods are as follows:

[0081] Participants calculate , corresponding to n position indices in the hash table, used for storing and Relevant data;

[0082] For each element Initialize an empty index set of size n. ,Will Add them in ascending order. middle;

[0083] Will be with each element The relevant data is stored in Specifically:

[0084] Each Save to the corresponding middle, Hash table The One bucket:

[0085] ;

[0086] Will Save to middle:

[0087] ;

[0088] For example, suppose there are three parties involved. , and . Having element a, and None.

[0089] For an element 'a', the corresponding hash table The three buckets, the first one stores (This value is determined by) (Calculated directly), the second and third positions store the decoded data respectively. and The resulting values ​​are two random numbers (assuming a is neither in the range of 'a' nor 'a'). In the set, and not in (Within the set). Similarly, each element has three mapping locations: one storing its own calculated value, and two storing the OKVS decoding results. No data is stored in buckets not mapped in the hash table.

[0090] right In other words, the second position corresponding to each element x stores the decoded data. The result The third position stores The first position stores the decoding. The result .right For example, the third position corresponding to each element y stores the decoding. The result The first position stores the decoding. The result The second position stores .

[0091] Step 3:

[0092] 3-1, Setting To obtain the final result, Each Send to ;

[0093] 3-2, Use your own hash table One bucket The data in the same bucket as the other n-1 hash tables The data in the middle is used as secret fragments to recover the secret value s; where, ;

[0094] For each of the n hash tables A bucket may store multiple values; each time, a label point is selected from the bucket. , ) To perform polynomial interpolation, we can avoid the exponential complexity of combined enumeration in traditional methods. (Number of calculations), because this method does not find a subset of size t among n elements; it maps to a hash table using a hash function. Not every element in the same bucket contains an element, so it is not... Meanwhile, this polynomial recovery process avoids the previous homomorphic encryption and decryption operations, reducing computational complexity.

[0095] Step 3-2 specifically includes the following steps:

[0096] Initialize a set Y;

[0097] The secret recovery process is as follows:

[0098] Choose a large prime number p such that p > n and p > n. (Used for interpolation);

[0099] Using k label points ( yes All hash tables owned For data pairs stored in the same bucket, Lagrange interpolation is performed by first calculating the Lagrange basis functions. , i=1,2,…,n:

[0100] ;

[0101] The final recovered polynomial is:

[0102] ;

[0103] When x=0, .

[0104] 3-3. Verify whether the polynomial value corresponding to the secret value is the polynomial value used for recovery. If it is, it means that the secret value is a super-threshold intersection. If it is not, it means that the secret value is not a super-threshold intersection.

[0105] Step 3-3 specifically includes the following steps:

[0106] Verification process:

[0107] For k label points First calculate k Check each calculated Is it equal to If they are equal, then s is a super-threshold intersection and s is added to set Y; if they are not equal, then s is not a super-threshold intersection and no further operation is performed.

[0108] In addition, the present invention also provides a system for finding intersections of data sets exceeding thresholds for security information processing, comprising:

[0109] The OKVS encoding and decoding module is used to enable each participant to perform unintentional key-value pair storage OKVS encoding and decoding with other participants;

[0110] The hash table construction module enables each participant to construct a hash table based on the decoding result and its own elements;

[0111] The aggregation and recovery module is used to send the constructed hash table to a designated party, which then aggregates the information from all participating parties' hash tables and uses a secret sharing scheme to finally obtain the super-threshold intersection result.

[0112] Based on the technical solution of this invention, the following case scenario illustrates the implementation process of this invention in practical applications. The specific application implementation scheme is as follows:

[0113] In collaborative analysis of rare pathogenic genes across hospitals, multiple hospitals want to discover frequently mutated genes (e.g., at least k hospitals have reported them), but patient genomic data is prohibited from being shared in plaintext. The final results are obtained by a designated hospital, and other hospitals cannot access the original data from others.

[0114] The implementation plan is as follows: Each hospital inputs a set of mutated gene loci as input to the threshold protocol. Through interaction among the participants: each participating hospital uses its own gene loci data to generate multiple related key-value pairs, and performs unintentional key-value pair encoding and decoding with other hospitals. Each party constructs a hash table based on the decoding result and its own elements, and sends the constructed hash table to the designated hospital, which then aggregates the gene loci information in all hospital hash tables. At the same time, a secret sharing scheme is used to finally obtain the threshold intersection result, that is, the designated hospital obtains a list of gene loci that appear in the gene loci sets of at least k hospitals and the total number of mutations.

[0115] Suppose five large cancer hospitals across the country want to collaborate on a study of the distribution of breast cancer susceptibility gene loci in the population, but are unwilling to directly share patient genetic information. Each hospital has whole-exome sequencing data from approximately 1,000-3,000 breast cancer patients. They want to identify susceptibility gene loci that appear in the databases of at least three of the hospitals. Taking the rs123456 locus as an example, "rs" is an abbreviation for "Reference SNP," which comes from the NCBI dbSNP database (a database specifically storing single nucleotide polymorphisms and other short sequence variations).

[0116] Assuming the hospital If a patient carries a mutation at this site, then the hospital... An element in a susceptibility gene locus database is =123456, generate PRG using a pseudo-random number generation function ( If ) = 11, then the two corresponding numbers 1 and 1 serve as the coefficients of the polynomial, thus becoming Construct polynomial So each elements The corresponding key-value pair is ( ).

[0117] In this way, each hospital uses its own key-value pairs for each element to perform OKVS encoding and decoding with other hospitals as described above, constructs a hash table, and finally aggregates the results. The resulting intersection of values ​​exceeding the threshold is as follows:

[0118] Table 1. Data table of intersection results exceeding the threshold

[0119]

[0120] Table 1 presents the results and their clinical significance for breast cancer research.

[0121] Therefore, the threshold-based protocol of this invention provides a practical and efficient solution for the secure sharing of gene data among multiple hospitals, and is expected to accelerate medical research progress while protecting information security. With the continuous development of computing technology and the ongoing accumulation of gene data, this technology will play an increasingly important role in precision medicine and population genetics research.

[0122] This invention proposes an innovative method for unbalanced over-threshold PSI that does not rely on third parties or homomorphic encryption: it abandons homomorphic encryption and implements set operations directly within the hash space using a distributed hash table, significantly reducing hardware and maintenance costs; it is completely decentralized, requiring no third-party intervention, which reduces the risk of data leakage caused by untrusted third parties; it adopts a point-to-point topology to ensure node dynamism, thus enhancing data security; and through multi-bucket mapping and storage order transformation of the hash table, it avoids the problem of computational complexity explosion, greatly improving computational efficiency and meeting the industrial control requirements of short-time response.

[0123] Threshold calculation protocols for specific scenarios (such as OT-MP-PSI) are of great significance in multi-party data collaboration. They achieve the ideal state of "data usable but not visible" through cryptographic technology, becoming the cornerstone of data protection in sensitive areas such as medical research, risk control, and public safety. For example, multiple hospitals can collaborate to statistically analyze the characteristics of prevalent diseases across institutions, revealing only those diseases jointly diagnosed by at least k institutions. Other participating institutions, except for designated parties, cannot access the original patient data of other hospitals, ensuring the comprehensiveness of medical research while eliminating the risk of patient data leakage. Similarly, a savings institution consortium can use this protocol to identify suspicious accounts marked by at least k institutions, outputting only a list of risky accounts exceeding the threshold. Individual savings institutions cannot access customer account information from other banks, greatly enhancing the collaborative efficiency and compliance of risk control. Its importance lies in reconstructing trust among multiple parties through technological means—transforming "data silos" scattered among various parties into jointly valuable data that can be securely computed without fully disclosing the original data. At the same time, by designating a single point of output for a specific party, it balances the controllability of the result with the confidentiality of the process, providing a secure and practical solution for the cross-domain circulation of highly sensitive data.

[0124] The above description is merely a detailed explanation of preferred embodiments and principles of the present invention. For those skilled in the art, there may be changes in specific implementation methods based on the ideas provided by the present invention, and these changes should also be considered within the scope of protection of the present invention.

Claims

1. A method for super-threshold data set intersection for secure information processing, characterized in that, Comprising the following steps: S1, each participant performs oblivious key-value pair storage OKVS encoding and decoding with other participants; S2, each participant constructs a hash table according to the decoding result and its own elements; S3, the constructed hash table is sent to the designated party, and the designated party summarizes the information in all participant hash tables, and finally obtains the super-threshold intersection result by using a secret sharing scheme; Step S1 comprises the following steps: S11, set is a set of participants, is a participant in the set, denotes the jth element in the set owned by the ith participant , and denotes the size of the set . S12, each participant OKVS encoding with own elements, as follows: by Construct a (k-1)th order polynomial for the key and according to Calculate the value of a polynomial : ； ； wherein, generated by a pseudo-random number generator PRG; Each participating user constructs a key-value pair for each element of his own ; S13, A data structure is obtained by performing OKVS encoding on the constructed key-value pairs. ,then Will Send to all other participants; S14, with own elements and other participants sent data structure , OKVS decoding, decoding results recorded as : , ; wherein, is an identifier of all participants except ; and the value range of i is the range left from 1 to n except i. Step S2 comprises the following steps: S21, set constructing a hash table with a length of B wherein , representing a load factor of the hash table, is a pre-set constant for controlling and balancing the space efficiency and time efficiency of the hash table; ; S22, Select and participate in the same number of hash functions n , to ensure that each element corresponds to n positions; S23, Inserting data corresponding to the element into the hash table The specific interpolation and construction method is as follows: Participants Computing corresponding to n position indexes in the hash table, for storing and related data; for each element initialize an empty index set of size n , add to in ascending order ; Data related to each element is stored in , in particular: each of data pairs into , denotes the th bucket of a hash table; th bucket; To data pairs into , denotes the hash table the first bucket; Step S3 comprises the following steps: S31, set Obtaining the final result, then Respectively Send to ; S32, with own hash table one bucket data in same bucket in other n-1 hash tables data in same bucket in other n-1 hash tables as secret shards to recover secret value s; wherein, ; S33, verifying whether the polynomial value corresponding to the secret value is the polynomial value used for recovery, if yes, it means that the secret value is a super-threshold intersection, if not, it means that the secret value is not a super-threshold intersection; Step S32 comprises the following steps: S321, Initialize a set Y; S322, the secret recovery process is as follows: Select a large prime number p, satisfying p > n, and p > 2n ; Using k landmark points Performing Lagrange interpolation, is All hash tables owned by For pairs of stored data in the same bucket, first compute the Lagrange basis functions , i = 1, 2,..., n: ； The final recovered polynomial is: ； When x = 0, , is one data in a data pair in the hash table; Step S33 comprises the following steps: S331, verification process: For k labeled points , first calculate k , check whether each is equal to , if equal, s is a super-threshold intersection, and s is added to the set Y; if not equal, it means that s is not a super-threshold intersection, and there is no subsequent operation.

Images