A Blockchain-Based Streaming Private Set Intersection Method and System

Abstract

The present invention discloses a blockchain-based streaming private set intersection method and system. First, the private set intersection initiator sends a private set intersection task application to the data party. After receiving the request, the data party starts parallel stream computing to generate private computing results. During the stream computing process, checkpoints are stored through a two-phase commit mechanism and the zero-knowledge proofs of the computing process are uploaded to the blockchain, enabling the stream computing cluster to quickly resume the previous computing state after a downtime restart and verifying the previous computing results. Moreover, the present invention adopts a streaming private set intersection method, uses parallel streams for private set intersection computing, ensures high computing efficiency and high data real-time performance, and at the same time uploads the zero-knowledge proofs of the computing process of stream computing to the blockchain, making the overall computing process verifiable. Thus, a method and system for private set intersection with high efficiency, high real-time performance, and high credibility are formed, providing great value for blockchain-based private computing.

Description

Technical Field

[0001] The present invention relates to the fields of privacy computing and blockchain, and particularly to a blockchain-based streaming private set intersection method and system. Background Art

[0002] As a new type of production factor, the value and importance of data are becoming increasingly prominent in the information-based contemporary society. How to solve the problem of the secure circulation of data elements is one of the main difficulties in stimulating the value of data elements at present. Privacy computing is an important method to realize the circulation of data and the in-depth mining of data value on the premise of protecting the rights and interests and personal privacy of data owners. Private set intersection is a commonly used privacy computing technology, which can obtain the intersection of the data held by both parties without any information leakage by the participating parties. In the implementation of private set intersection, a large number of full-database calculations are often required, which will lead to problems of the speed and efficiency of private set intersection, and cannot guarantee the real-time nature of the intersecting data, making it difficult to meet some private set intersection application scenarios that require computational efficiency and data real-time nature. Summary of the Invention

[0003] Aiming at the deficiencies of the prior art, the problem to be solved by the present invention is how to accelerate the computational efficiency of private set intersection and how to ensure the real-time nature of private set intersection data. To solve this problem, the present invention proposes a blockchain-based streaming private set intersection method and system.

[0004] The object of the present invention is achieved through the following technical solutions:

[0005] A blockchain-based streaming private set intersection method, characterized by including the following steps:

[0006] (1) The data party receives a private set intersection task request from the initiating party. The request includes the node information of the initiating party and the required private data, and the data party and the initiating party establish communication;

[0007] (2) After the data party accepts the task request, the scheduler in the stream computing cluster schedules the stream computing nodes and sets parallel streams;

[0008] (3) The data party performs corresponding encryption processing on the private data according to different private set intersection methods; different private set intersection methods are methods such as key exchange, oblivious transfer, oblivious pseudorandom function, and programmable oblivious random function;

[0009] (4) In the communication, the encrypted result of the data is processed in a streaming manner, and the data arrives in a stream and performs private set intersection calculation in a parallel stream manner;

[0010] (5) The data party inserts barriers in the data stream through the scheduler to divide checkpoints;

[0011] (6) When the operator that calculates the data stream reads a barrier, it will generate and save the state snapshot of this operator.

[0012] (7) After all operators have generated state snapshots, the scheduler will call back to notify all operators that the state snapshot of this checkpoint has been generated, and generate a zero-knowledge proof of the calculation process of this checkpoint and upload it to the chain.

[0013] (8) Obtain the private intersection result, and continuously update the private intersection result as new data arrives.

[0014] (9) End the private intersection task, close the corresponding data stream, and release the corresponding data stream computing node.

[0015] Further, the step (7) includes the following sub-steps:

[0016] (2.1) After the stream computing cluster fails and recovers, it will read the persisted state snapshot, verify the corresponding zero-knowledge proof on the chain to verify the correctness of the snapshot calculation process, and restore to the corresponding computing state.

[0017] (2.2) When the stream computing cluster resumes normal operation, it requests data from the data source according to the data processing status, and continues to process the subsequent data starting from the restored checkpoint state.

[0018] Further, in the step (5), dividing the checkpoint is to divide the data before the barrier into the previous checkpoint, and the data after the barrier into the next checkpoint.

[0019] Further, the step (9) is specifically that after the user ends the private intersection task operation, it will close the data sending stream, and notify the data party to close the data receiving stream, and release the corresponding stream computing node after the remaining data is processed.

[0020] A streaming private intersection system applied to a blockchain, the streaming private intersection module performs parallel stream processing on the private intersection calculation, regularly generates state snapshots by setting checkpoints, and generates a zero-knowledge proof of the corresponding calculation process and uploads it to the chain; it includes the following modules:

[0021] The private intersection task module is responsible for selecting the data parties for private intersection and sending task requests.

[0022] The streaming private intersection module, after receiving the private intersection task request, establishes communication with the initiator, performs private comparison in the parallel stream to obtain the intersection of private data, and regularly uploads the zero-knowledge proof of the intermediate calculation state to the chain.

[0023] The calculation state recovery and verification module is responsible for the recovery of the stream computing cluster in case of failure or downtime, enabling the system to quickly resume the latest calculation state and perform a correctness check on the previous calculation results.

[0024] The beneficial effects of the present invention are as follows:

[0025] The system of the present invention improves the efficiency of private set intersection through the method of distributed stream computing and supports the parallel execution of private set intersection calculations. Relying on the characteristics of stream computing, there is no need to wait until all private data is received before starting the private set intersection operation. At the same time, new data during the private set intersection calculation can also participate in the calculation, ensuring the timeliness of the private set intersection data. The fault tolerance of the system is improved through the checkpoint two-phase commit mechanism.

[0026] After the checkpoint is completed, the system will generate a zero-knowledge proof for the corresponding calculation process and upload it to the blockchain, making the calculation process of the checkpoint verifiable and ensuring the credibility of the calculation under the premise of privacy. Description of the Drawings

[0027] The present invention will be further described below with reference to the drawings.

[0028] Figure 1 is a flowchart of the blockchain-based stream privacy intersection method described in the embodiments of the present invention;

[0029] Figure 2 is a flowchart of the blockchain-based stream privacy intersection fault recovery and verification method described in the embodiments of the present invention;

[0030] Figure 3 is the basic system architecture diagram of the blockchain-based stream privacy intersection described in the embodiments of the present invention. Detailed Embodiments

[0031] The present invention will be further described in detail below with reference to the drawings and specific embodiments.

[0032] As Figure 1 and Figure 2 shown, a blockchain-based stream privacy intersection system according to an embodiment of the present invention includes a private set intersection task module, a stream privacy intersection module, and a calculation state recovery and verification module, wherein:

[0033] S1: The user initiates a task using the private set intersection task module 1. This module is part of the private computing node. Each data party will deploy a private computing node. It can be considered that each private computing node represents a data party. The user selects the data parties and data sources of the task to participate in through the private set intersection task module and sends a task request.

[0034] S2: After the data party receives the information of the private set intersection task in the private set intersection task module 1, it determines whether to accept the task request according to the information of the initiator and the data source to be provided, etc. If it accepts the task, it allocates computing resources and sets up parallel streams through the scheduler of the local stream computing cluster, and establishes a connection communication with the initiator node to receive the data from the sender in the form of a stream, without having to receive all the data at once.

[0035] S3: The initiator obtains the response from the data party through the private set intersection task module 1. If it passes, it starts to let the scheduler of the local stream computing cluster allocate computing resources and set up parallel streams, starts to encrypt the local data according to the implementation method of the private set intersection, and sends the encrypted result in the form of a stream, without having to send all the data at once. If new data is generated during the calculation process, it will also be sent through the stream.

[0036] S4: The streaming private set intersection module 2 of the data party will add a barrier to the received data stream through the scheduler of the stream computing cluster. The barrier will not affect the processing of the data in the stream, but only be used to divide the checkpoints. The data before the barrier belongs to the previous checkpoint, and the data after the barrier belongs to the next checkpoint. The streaming private set intersection module 2 will process the received encrypted data in a parallel stream manner and obtain the private set intersection set through the method of private comparison. As the received data stream continuously receives data for processing, the streaming private set intersection module 2 will continuously update the result of the private set intersection.

[0037] S5: When the operator in the stream computing task in the streaming private set intersection module 2 reads the barrier, it will generate and save the state snapshot of this operator, and then send the barrier to the next operator. When all operators have generated their respective state snapshots, a pre-commit will be performed. After the pre-commit is successful, the scheduler of the cluster will notify each operator that this round of checkpoint has been completed. At this time, a real commit will be performed and a zero-knowledge proof of the calculation process of this checkpoint will be generated and uploaded to the chain; that is, the zero-knowledge proof of the calculation process will be saved on the blockchain, and the zero-knowledge proof uploaded at this time will be obtained from the blockchain during subsequent verification.

[0038] S6: In the case of a failure and downtime of the stream computing cluster, the computing state recovery and verification module 3 will, after the cluster resumes normal operation, read the latest saved state snapshot, obtain the corresponding zero-knowledge proof from the blockchain, and verify the correctness of the calculation of this state snapshot. If the verification passes, it will restore the stream computing cluster to the state snapshot of the corresponding checkpoint, and request data from the sender again according to the data position processed at this time state, and resume the normal data processing flow. If the verification fails, it will read the previous snapshot and repeat the above steps.

[0039] After the Privacy Intersection Task Module 1 monitors that the user has ended the privacy intersection task operation, it will close the data sending stream and notify the Streaming Privacy Intersection Module 2 of the data party to close the data receiving stream. After the remaining data is processed, the corresponding stream computing node will be released.

[0040] As Figure 3 shown, the streaming privacy intersection module of the present invention performs parallel stream processing on privacy intersection calculations, regularly generates status snapshots by setting checkpoints, and generates zero-knowledge proofs of the corresponding calculation process and uploads them to the chain; it includes the following modules:

[0041] The Privacy Intersection Task Module is responsible for selecting the data party for privacy intersection and sending task requests;

[0042] The Streaming Privacy Intersection Module, after the data party receives the privacy intersection task request, establishes communication with the initiator, performs privacy comparison in the parallel stream to obtain the privacy data intersection, and regularly uploads the zero-knowledge proof of the intermediate calculation state to the chain;

[0043] The Calculation State Recovery and Verification Module is responsible for the recovery of stream computing cluster failures or outages, enabling the system to quickly recover to the latest calculation state and perform correctness verification on the previous calculation results.

[0044] This embodiment is only used to illustrate the technical solution of the present invention, rather than imposing any formal restrictions on the invention. It is still possible to modify the technical solution recorded in the foregoing embodiment, or perform equivalent replacement on some or all of its technical features. These modifications or replacements do not cause the essence of the corresponding technical solution to deviate from the scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A blockchain-based streaming private set intersection method, characterized in that, It includes the following steps: (1) The data party receives a privacy intersection task request from the initiating party. The request includes the node information of the initiating party and the privacy data required. The data party and the initiating party establish communication; (2) After the data party accepts the task request, the scheduler in the stream computing cluster schedules the stream computing nodes and sets up parallel streams; (3) The data party performs corresponding encryption processing on the privacy data according to different privacy intersection methods; different privacy intersection methods are key exchange, oblivious transfer, oblivious pseudorandom function, and programmable oblivious random function methods; (4) In communication, the encrypted result of the data is processed in a streaming manner. The data arrives in a stream and privacy intersection calculation is performed in a parallel stream manner; (5) The data party inserts barriers into the data stream through the scheduler to divide checkpoints; (6) When the operator that calculates the data stream reads the barrier, it generates and saves the state snapshot of this operator; (7) After all operators have generated state snapshots, the scheduler will callback to notify all operators that the state snapshot of this checkpoint has been generated, and generate a zero-knowledge proof of the calculation process of this checkpoint and upload it to the chain; (8) Obtain the privacy intersection result, and continuously update the privacy intersection result as new data arrives; (9) End the privacy intersection task, close the corresponding data stream, and release the corresponding data stream computing nodes.

2. The method for applying the blockchain-based streaming private intersection according to claim 1, wherein The step (7) includes the following sub-steps: (2.1) After the stream computing cluster fails and recovers, it reads the persisted state snapshot, verifies the corresponding zero-knowledge proof on the chain to verify the correctness of the snapshot calculation process, and restores to the corresponding calculation state; (2.2) When the stream computing cluster resumes normal operation, it requests data from the data source according to the data processing status, and continues the subsequent data processing starting from the restored checkpoint state.

3. The method for applying a blockchain-based streaming private intersection according to claim 1, wherein In the step (5), dividing the checkpoint means dividing the data before the barrier into the previous checkpoint and the data after the barrier into the next checkpoint.

4. The method for applying the blockchain-based streaming private set intersection according to claim 1, characterized in that The step (9) is specifically that after the user ends the privacy intersection task operation, it closes the data sending stream, notifies the data party to close the data receiving stream, and releases the corresponding stream computing nodes after the remaining data is processed.

5. A streaming private set intersection system applied to the streaming private set intersection method based on blockchain according to any one of claims 1-4, characterized in that, The streaming privacy intersection performs parallel stream processing on the privacy intersection calculation, regularly generates state snapshots by setting checkpoints, and generates a zero-knowledge proof of the corresponding calculation process and uploads it to the chain; it includes the following modules: Privacy intersection task module, responsible for selecting the privacy intersection data party and sending task requests; Streaming privacy intersection module, after the data party receives the privacy intersection task request, it establishes communication with the initiating party, performs privacy comparison in a parallel stream to obtain the privacy data intersection, and regularly uploads the zero-knowledge proof of the intermediate calculation state to the chain; Calculation state recovery and verification module, responsible for the recovery of the stream computing cluster failure or downtime, enabling the system to quickly restore to the latest calculation state and making a correctness verification of the previous calculation results.

Images