Streaming data black box real-time verification method and system based on on-chain and off-chain mixed storage
Through the combination of hybrid storage on-chain and off-chain storage and smart contracts, an adaptive trapped hash tree structure is built, which solves the data loss and debunk problems in streaming data outsourcing storage, real-time verification and privacy protection of streaming data are realized. Improves the credibility and security of verification.
Patent Information
- Application Number
- CN202410989615.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-11-29
AI Technical Summary
In the outsourcing storage scenario of streaming data, there are security issues such as data loss, corruption and user denial. Traditional methods cannot effectively ensure the real-time verification and credibility of streaming data, especially in terms of limited scalability and privacy protection of blockchains. There are challenges.
Using a streaming data black box real-time verification method based on chain-on-chain hybrid storage, the trapped hash tree is constructed through the chameleon hash function, adaptively expand, and fair audits are conducted with blockchain smart contracts, using zero-knowledge proof and zk -SNARKs algorithm hides verification information to ensure data integrity and privacy protection.
Realize instant verification of convective data, improve verification credibility in outsourcing scenarios, prevent cloud server spoofing and user denial, ensure the integrity and correctness of data in the sharing process, reduce the cost and overhead of blockchain, and apply For big data privacy protection and secure storage.
Smart Images

Figure CN119051828A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of cross-blockchain technology, and in particular to a black box real-time verification method and system for streaming data based on on-chain and off-chain hybrid storage. Background Art
[0002] In recent years, with the rapid development of the Internet of Things and the popularization of 5G networks, everything is connected, network edge devices have increased dramatically, and the data generated has grown exponentially. The rapid development of large-scale complex networks such as P2P networks, cloud computing, and sensor networks has also posed new challenges to data security. The data that needs to be processed is not only static data of known size, but also continuous, infinite, fast, and time-varying streams (such as real-time monitoring, online video, ambient temperature monitoring, stock trading, etc.). Limited user resources cannot store and process such a large number of streaming data, and outsourcing streaming data to third-party storage, management, analysis, and computing applications has gradually entered the public eye. Compared with traditional internal storage infrastructure, it has higher scalability and lower management costs, attracting more and more users to choose to store various types of data on cloud storage servers. In traditional cloud-based data storage and sharing solutions, IoT devices upload the collected data to cloud servers for storage, which solves the problem of limited data storage capacity of IoT devices and facilitates users to share data. However, outsourcing streaming data also brings additional security and privacy issues. In traditional outsourcing data, the security of data depends on the cloud server, but in an open network environment, no node is absolutely trustworthy. If the cloud server is hacked by an attacker, once the root node is destroyed, all leaf data information cannot be verified, which is a single point of failure. Users deny the cloud server, etc. In the above case, it is particularly important to introduce a trusted third party to conduct fair audits. Therefore, how to improve the credibility of outsourced streaming data integrity verification has become a research hotspot.
[0003] With the development of P2P network technology, the decentralized distributed storage structure, blockchain, is widely used in finance, the Internet of Things, and file management due to its decentralized and tamper-proof characteristics. It is considered a promising solution for secure data storage and retrieval. Blockchain can be regarded as a distributed ledger, which is jointly maintained by untrusted nodes in the network. With the emergence and application of smart contract functions, blockchain technology has gradually transformed from a simple transaction record ledger to an effective and reliable data trusted storage carrier, which can realize trusted transactions digitally without a third party. These transactions are traceable and irreversible. While effectively solving the problems of service fairness and storage correctness, due to the inherent transparency and openness of blockchain, the construction of data storage platforms based on blockchain is still seriously hindered by potential privacy and compromised functional issues. Although blockchain has obvious advantages, data needs to be replicated throughout the network, its scalability is limited, and the overhead problem cannot be ignored in the design of data outsourcing storage solutions. As a data storage carrier, the overhead cost and data privacy protection of blockchain are both issues that need to be considered.
[0004] Stream data is often continuous and unpredictable. Traditional authenticatable data structures cannot be directly applied to stream data. It is necessary to design a dedicated authenticatable data structure for stream data, which not only supports real-time verification of stream data, but also ensures the verifiability of stream sequences. With the continuous breakthroughs and developments in the integrity verification technology of stream data, the main methods for constructing verifiable data structures for stream data are Merkle hash trees, chameleon authentication trees, and password accumulators. However, the above methods may have problems such as untrustworthy third-party audit nodes and user denial. Summary of the invention
[0005] To this end, the present invention provides a streaming data black box real-time verification method and system based on on-chain and off-chain hybrid storage to solve security issues such as data loss, damage, and data repudiation that may occur in existing streaming data outsourcing storage scenarios.
[0006] According to the design scheme provided by the present invention, on the one hand, a black box real-time verification method for streaming data based on on-chain and off-chain hybrid storage is provided, comprising:
[0007] The data owner uses the chameleon hash function to construct a trapdoor hash tree for the flow data in the outsourcing scenario, uploads the flow data related information and authentication path to the cloud server, and uploads the flow data verification evidence to the blockchain. The trapdoor hash tree is used to represent the authenticatable data structure corresponding to the flow data in the outsourcing scenario of the data owner and is adaptively expanded according to the change of the flow data scale; the flow data verification evidence is obtained based on the subtree root information fixed at each layer of the trapdoor hash tree, the data owner ID and the root node upload timestamp;
[0008] The data accessor who obtains data access rights obtains the authentication-related data of the queried content by initiating a query request to the cloud server, and restores the subtree root and corresponding verification evidence to the queried content based on the authentication-related data and sends them to the blockchain. The authentication-related data includes the encrypted data block corresponding to the query request, authentication information, data owner, and data generation time;
[0009] The blockchain uses preset smart contract functions to audit the subtree root and verification evidence to which the query content belongs, and feeds back the audit results to the data accessor.
[0010] As a black box real-time verification method for streaming data based on on-chain and off-chain hybrid storage of the present invention, further, the data owner uses the chameleon hash function to construct a trapdoor hash tree for streaming data in the outsourcing scenario, including:
[0011] The data owner encrypts the current stream data to obtain the encrypted ciphertext, and generates a data block based on the encrypted timestamp, data owner ID and encrypted ciphertext;
[0012] A hash tree is established with the data blocks corresponding to each flow data in the outsourcing service scenario as leaf nodes. When the flow data scale meets the specified conditions, the tree structure is expanded by increasing the tree structure depth, and the original tree structure is changed into the left subtree of the newly generated root node, so as to use the expanded tree structure to adapt to the current flow data scale.
[0013] As a black box real-time verification method for streaming data based on on-chain and off-chain hybrid storage of the present invention, further, when the streaming data scale meets the specified conditions, it includes: after inserting a specified number of data blocks into the hash tree, when the hash tree structure forms a full binary tree structure, the specified number is 2 i , i is the number of subtree root nodes.
[0014] As a black box real-time verification method for streaming data based on on-chain and off-chain hybrid storage of the present invention, further, uploading streaming data verification evidence to the blockchain includes:
[0015] The data owner sends a data initialization request to the blockchain and cloud server, and creates a default zero value, which is used to calculate the root hash value of each layer of subtree when the stored data is empty;
[0016] After receiving the initialization request, the blockchain creates a smart contract account in the state tree, and obtains the initial authentication evidence through the local zero-knowledge proof of the data owner when the data is empty, and stores the contract code of the relevant operation in the specified field, so as to perform fair and public audit operations by reading the specified field information during the audit phase;
[0017] When the data owner's stream data meets the conditions, verification evidence is generated based on the smart contract and using the set of generated subtree root nodes, and the verification evidence is uploaded to the blockchain for storage through consensus.
[0018] As a black box real-time verification method for streaming data based on on-chain and off-chain hybrid storage of the present invention, further, the verification evidence is generated using the set of generated subtree root nodes, including:
[0019] The root set of each layer of the subtree of the stream data adaptive trapdoor hash tree is used as private information, and the digital signature and local time are attached to the private information to generate auxiliary information;
[0020] The hash value of the auxiliary information is calculated using random numbers, and the hash value is electronically signed using a private key. Verification evidence is obtained based on the electronic signature, auxiliary information, and the key generated by the zero-knowledge proof.
[0021] As a black box real-time verification method for streaming data based on on-chain and off-chain hybrid storage of the present invention, further, uploading streaming data verification evidence to the blockchain also includes:
[0022] The bottom leaf nodes in the hash tree generate verification evidence with the root node of the smallest subtree in which they are located. And every time a new subtree root node is generated in the adaptive expansion of the hash tree, the root node set regenerates verification evidence through the zero-knowledge proof algorithm and uploads it to the blockchain.
[0023] As a black box real-time verification method for streaming data based on on-chain and off-chain hybrid storage of the present invention, further, the blockchain uses a preset smart contract function to audit the subtree root and verification evidence to which the query content belongs, including:
[0024] The verification evidence is verified using the data owner's public key and the specified verification key. After the verification is passed, the integrity of the stream data is verified by comparing the zero-knowledge proof, calculation results and hash values of the data owner and the cloud server.
[0025] On the other hand, the present invention also provides a streaming data black box real-time verification system based on on-chain and off-chain hybrid storage, comprising: a data owner, a cloud storage, a blockchain and a data accessor, wherein:
[0026] The data owner uses the chameleon hash function to construct a trapdoor hash tree for the flow data in the outsourcing scenario, uploads the flow data related information and the authentication path to the cloud server, and uploads the flow data verification evidence to the blockchain. The trapdoor hash tree is used to represent the authenticatable data structure corresponding to the flow data in the outsourcing scenario of the data owner and is adaptively expanded according to the change of the flow data scale; the flow data verification evidence is obtained based on the subtree root information fixed at each layer of the trapdoor hash tree, the data owner ID and the root node upload timestamp;
[0027] The cloud server is used to store and maintain information related to the stream data and respond to data query requests from data visitors;
[0028] The blockchain, as a third party that performs fair audits, stores dynamic root node verification evidence of streaming data, and decentralized smart contracts perform verification in fair audits;
[0029] The data accessor obtains the authentication-related data of the queried content by obtaining data access rights and initiating a query request to the cloud server. Based on the authentication-related data, the subtree root to which the queried content belongs and the corresponding verification evidence are restored and sent to the blockchain. The authentication-related data includes the encrypted data block corresponding to the query request, authentication information, data owner and data generation time.
[0030] Beneficial effects of the present invention:
[0031] In the outsourcing storage verification of big data stream, the present invention aims at the problems of unpredictable scale, high real-time verification, and delay sensitivity. The real-time verification structure of on-chain and off-chain hybrid storage is used. The complete stream data is stored off-chain based on an adaptive trapdoor hash tree. The authentication root node is stored on-chain as the verification information of the query data. With the dynamic growth and change of the big data stream, it is constructed in real time, and the data stream is verified immediately. Based on the blockchain structure, the on-chain and off-chain hybrid storage architecture is constructed, and the complete authenticatable data structure is stored off-chain. The on-chain smart contract triggers the execution of fair result verification, prevents cloud server deception and user denial, ensures the credibility of data verification results, and improves the credibility of verification nodes in the stream data outsourcing scenario. In addition, the on-chain black box authentication is used to solve the problem of verification data privacy leakage in third-party audits under the cloud platform, hide the verification information on the chain, effectively maintain the security of the on-chain authentication data, and enable the auditor to perform black box verification without leaking any data privacy, so as to achieve the integrity and correctness of the data in the sharing process, protect the interests of all parties of the data, and have good application prospects in the field of big data privacy protection and secure storage. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 This is a schematic diagram of the real-time black box verification process of streaming data based on on-chain and off-chain hybrid storage in the embodiment;
[0033] Figure 2 Schematic diagram of adaptive trapdoor hash tree structure at different scales in the embodiment;
[0034] Figure 3 This is a schematic diagram of the stream data trapdoor hash tree authentication data structure in the embodiment;
[0035] Figure 4 This is a schematic diagram of the complete on-chain authentication information storage and verification process in the embodiment. DETAILED DESCRIPTION
[0036] In order to make the purpose, technical solutions and advantages of the present invention clearer and more understandable, the present invention is further described in detail below in conjunction with the accompanying drawings and technical solutions.
[0037] Blockchain is often designed to effectively resolve the role of third-party trust delegation due to its decentralized and tamper-proof characteristics. Blockchain itself can be used as a database to ensure that data is not tampered with. Its service process is calculated by smart contracts to ensure the openness and fairness of audit results. Due to the limitation of on-chain storage space, the consensus and verification costs of running the system on the blockchain will increase with the increase in demand, making it impossible for data audit solutions on the blockchain to operate sustainably due to the huge on-chain costs. In response to security issues such as data loss, corruption, and user denial that may occur in streaming data outsourcing storage scenarios, the embodiments of the present invention cleverly combine verifiable data structures, blockchains, and zero-knowledge proofs to provide a streaming data black box real-time verification method based on on-chain and off-chain hybrid storage, including:
[0038] The data owner uses the chameleon hash function to construct a trapdoor hash tree for the flow data in the outsourcing scenario, uploads the flow data related information and authentication path to the cloud server, and uploads the flow data verification evidence to the blockchain. The trapdoor hash tree is used to represent the authenticatable data structure corresponding to the flow data in the outsourcing scenario of the data owner and is adaptively expanded according to the change of the flow data scale; the flow data verification evidence is obtained based on the subtree root information fixed at each layer of the trapdoor hash tree, the data owner ID and the root node upload timestamp;
[0039] The data accessor who obtains data access rights obtains the authentication-related data of the queried content by initiating a query request to the cloud server, and restores the subtree root and corresponding verification evidence to the queried content based on the authentication-related data and sends them to the blockchain. The authentication-related data includes the encrypted data block corresponding to the query request, authentication information, data owner, and data generation time;
[0040] The blockchain uses preset smart contract functions to audit the subtree root and verification evidence to which the query content belongs, and feeds back the audit results to the data accessor.
[0041] Among them, the trapdoor hash function is also called the chameleon hash function. It is a special hash function with the anti-collision property of the hash function. For the person holding the trapdoor, there is an algorithm that outputs a random number r' that matches m', so that Th(m,r)=Th(m',r'), (m,r) and (m',r') are called a collision pair. The trapdoor hash function is collision-resistant. If the trapdoor is not known, it is computationally infeasible to find two different pairs of (m,r) and (m',r') so that Th(m,r)=Th(m',r').
[0042] Specifically, it can be set as a trapdoor hash function composed of the following polynomial time algorithm (TrapGen, Th, Col): TrapGen (1 λ )→(tpk,tsk): Initial key generation algorithm, input security parameter 1 λ The trapdoor key tsk and the hash key tpk are generated.
[0043] Th(tpk,m,r)→Th(m): trapdoor hash value calculation algorithm, input trapdoor hash function public key, message m∈{0,1} in ,The random value r is used to calculate the corresponding trapdoor hash value through the hash key.
[0044] Col(tsk,m,r,m')→r': trapdoor hash collision calculation algorithm, input the private key and message m of the one-way trapdoor hash function, the random number r and the new information m' that needs to be matched. The algorithm outputs a random number r' such that
[0045] Th(tpk,m,r)=Th(tpk,m',r').
[0046] In view of the problems of untrustworthy third-party audit nodes and user denial in the verifiable outsourced data structure of streaming data, the C2BR-VDS scheme is proposed in the embodiment of this case. The blockchain and cloud server jointly maintain the verifiable data structure of outsourced streaming data efficiently, store the complete verifiable data structure off-chain, and the smart contract performs the verification of the data query structure fairly on the chain. The complete authentication data structure is based on the adaptive trapdoor hash tree structure. Combined with the data characteristics of streaming data that are verified immediately, the chain-on-chain hybrid storage authentication data structure is designed. The authentication scale does not need to be fixed in advance, and it can be adaptively expanded according to the change of data flow scale. The root node is stored on the blockchain as verification information, and the verification is performed by the smart contract. It not only solves the real-time verification problem of large data streams, but also prevents user denial, reduces the cost of blockchain, and effectively improves the fairness of verification results in the streaming data outsourcing scenario. In addition, in order to protect the privacy of on-chain data and prevent users from snooping on the verification information and denying it to the cloud server, black box verification can be implemented based on the zk-SNARKs algorithm combined with smart contracts to hide the valid verification information on the chain. During the execution process, without providing any valid information to the blockchain node, the verifier believes that a certain assertion is correct. In blockchain data sharing audit, the blockchain smart contract stream data authentication scheme based on zero-knowledge proof achieves the effect of black-box verification of stream data without exposing the on-chain data. Figure 1 As shown in the figure, the C2BR-VDS solution can divide the flow data storage, query and verification into 6 main steps:
[0047] a: Data storage stage: ① The data owner constructs the corresponding trapdoor hash tree for the data m locally, uploads the complete data and its authentication path to the cloud server, and ② uploads the generated fixed subtree root values of each layer to the blockchain at the same time.
[0048] b: Data query phase: ③ After the data accessor obtains data access rights, he initiates a query request to the cloud server. ④ The cloud server returns the query result R and the corresponding evidence authentication path auth through range query.
[0049] c: Data authentication phase: ⑤ The data accessor calculates the root value locally based on the information returned by the cloud server and sends it to the blockchain, ⑥ The blockchain's smart contract performs the audit and returns the audit result True / False to the data accessor.
[0050] In this embodiment of the case, the data owner uses the chameleon hash function to construct a trapdoor hash tree for the downstream data in the outsourcing scenario, which can be designed to include the following content:
[0051] The data owner encrypts the current stream data to obtain the encrypted ciphertext, and generates a data block based on the encrypted timestamp, data owner ID and encrypted ciphertext;
[0052] A hash tree is established with the data blocks corresponding to each flow data in the outsourcing service scenario as leaf nodes. When the flow data scale meets the specified conditions, the tree structure is expanded by increasing the tree structure depth, and the original tree structure is changed into the left subtree of the newly generated root node, so as to use the expanded tree structure to adapt to the current flow data scale.
[0053] Due to the unpredictable characteristics of streaming data, based on the trapdoor hash tree structure, in order to adapt to the real-time requirements and unpredictable data scale in streaming data outsourcing scenarios, a hybrid storage structure on-chain and off-chain is constructed based on the adaptive trapdoor hash tree. The verification information is stored on the chain, and the complete authenticatable data structure is stored in the off-chain cloud server. Based on the adaptive trapdoor hash tree structure, it is adaptively expanded according to the changes in the scale of the data stream, and the big data stream is read, written and verified in the streaming data outsourcing service, which can be better applied to the big data stream scenario under the cloud platform. When the underlying leaf node is inserted and updated, there is always a unique trapdoor hash node corresponding to it in the authentication tree. Among them, the leaf nodes all use the root node of the smallest subtree in which they are located as the verification information, which is recorded on the blockchain, and at the same time, a shorter authentication path is generated, and the problem of single point failure is avoided. It solves the problems of high real-time data stream verification, delay sensitivity, and unpredictable data volume.
[0054] During the adaptive expansion of the off-chain fully authenticatable data structure, a new root node will be generated each time the current structure cannot meet the data scale, and stored on the blockchain as verification information. The original structure becomes the left subtree of the newly generated root node, the depth increases by 1, and the amount of data that can be supported is expanded to twice the original amount. Figure 2 These are the structures of authentication trees at three different scales. The authentication tree structures at different scales are consistent. The smaller authentication tree is the left subtree of the larger authentication tree. Since the amount of data supported by each authentication tree is doubled after expansion, the subsequent expansion operation will not be particularly frequent.
[0055] like Figure 3 As shown, insert every 2 i After data blocks, the hash tree structure will form a full binary tree structure. In order to continue inserting new data blocks, the adaptive trapdoor hash tree will adaptively expand one layer upward, and the original structure will become a subtree of the newly generated root node, which is used as the verification root node for the right subtree to insert data block m. Each time a new subtree root node Root is generated i (0<i≤n), the root node set is re-uploaded to the blockchain as verification evidence through the zero-knowledge proof algorithm.
[0056] Among them, uploading the stream data verification evidence to the blockchain can be designed to include:
[0057] The data owner sends a data initialization request to the blockchain and cloud server, and creates a default zero value, which is used to calculate the root hash value of each layer of subtree when the stored data is empty;
[0058] After receiving the initialization request, the blockchain creates a smart contract account in the state tree, and obtains the initial authentication evidence through the local zero-knowledge proof of the data owner when the data is empty, and stores the contract code of the relevant operation in the specified field, so as to perform fair and public audit operations by reading the specified field information during the audit phase;
[0059] When the data owner's stream data meets the conditions, verification evidence is generated based on the smart contract and using the set of generated subtree root nodes, and the verification evidence is uploaded to the blockchain for storage through consensus.
[0060] Specifically, generating verification evidence using the generated set of subtree root nodes may include:
[0061] The root set of each layer of the subtree of the stream data adaptive trapdoor hash tree is used as private information, and the digital signature and local time are attached to the private information to generate auxiliary information;
[0062] The hash value of the auxiliary information is calculated using random numbers, and the hash value is electronically signed using a private key. Verification evidence is obtained based on the electronic signature, auxiliary information, and the key generated by the zero-knowledge proof.
[0063] The chain stores verification information and hides the valid data on the chain based on the zk-SNARKS algorithm, realizing the black box verification of the smart contract on the chain. The complete chain authentication information storage and verification process is as follows: Figure 4 As shown in the figure, the verification information on the chain is the root node {C} = {Root} of each layer of the subtree, the user ID of the data owner, and the root node upload timestamp, which are generated by the circuit design to generate the verification proof Proof and uploaded to the blockchain. When the user queries and verifies, the verification root node corresponding to the queried data is calculated locally, and after the same circuit operation, it is delivered to the smart contract for verification. In this process, the user does not know whether the verification information calculated locally is valid, which prevents the user from denying it.
[0064] The authenticatable data structure of the C2BR-VDS solution is real-time and black-box, which can be summarized as follows:
[0065] Real-time: Based on the adaptive trapdoor hash tree, an on-chain and off-chain hybrid storage structure is constructed, which can be adaptively expanded according to the changes in the scale of data flow, and the reading, writing and verification of large data streams can be realized in the streaming data outsourcing service, which can be better applied to large data stream scenarios under the cloud platform.
[0066] Black box: Based on the zk-SNARKs algorithm, the verification information on the chain is hidden, effectively maintaining the security of the authentication data on the chain. The auditor can perform black box verification without leaking any data privacy, achieve consistency and availability of data in the sharing process, and protect the interests of all parties involved in the data.
[0067] The black-box real-time verification scheme for streaming data running on a hybrid on-chain and off-chain storage architecture consists of the following tuple of polynomial time algorithms:
[0068] B-BRVSD=(Source Append ; Server Append ; B.C. Append ;
[0069] GenProof Server ;GenProof User ;Verify Contact )
[0070] Source Append (m,st)→(auth send,st′,c) is the data source data addition algorithm, which is run by the data source, that is, the data owner, to encrypt each stream data l to obtain l′, and generate a data block m=(l′,T,ID u ), and calculate the corresponding authentication path information and update the state vector st, then encrypt the data block m′, the authentication path information auth send ,Insert location information c, data generation time T and data owner ID and upload them to the cloud storage server.
[0071] After acquiring the data, the data source generates the structural update information of the authentication tree. First, according to the scale value of the current authentication tree (the amount of data that can be accommodated is represented by the parameter capacity) and the number of leaf nodes c added to the authentication tree, it is determined whether the authentication tree should be expanded. It can be divided into the following cases: if the capacity of the current tree is full (c = capacity), the authentication tree is expanded, the depth value is increased by d←d+1, the capacity of the authentication tree becomes twice the original capacity = capacity*2, and a new root node new root←R is generated d , the original root node is the left child R of the new root node d.leftchild ←R (d-1) , add the new root node to Root, randomly select x (d-1,1) ,r (d-1,1) , calculate the trapdoor hash value v of the right child of the newly generated root node (d-1,1) ←Th(x (d-1,1) ,r (d-1,1) ), we can get the hash value R of the newly generated root node d =H(R d-1 ||v (d-1,1) ), and change the state of st: st'←st, and add the calculated trapdoor hash value to the authentication path auth. The next operation is the same as when the tree capacity is not full (c<capacity), find the bottom-level puppet node (the trapdoor hash node without children) from st, generate a subtree with this node as the root node, and after inserting the data, update the value of the corresponding trapdoor hash node along the leaf from bottom to top, and add the calculated trapdoor hash value to the authentication path auth of the data.
[0072] Server Append (auth,c)→SNode new Add an algorithm to the cloud server data and run it on the cloud server. The cloud server receives the authentication path information auth send,After obtaining the data information and the data insertion position information, the authentication tree SNode is updated. After finding the corresponding insertion position, the corresponding path node is updated from bottom to top. According to the current data insertion position information, the data insertion can be completed through binary search, and the missing trapdoor nodes can be calculated along the way.
[0073] BC_Server Append ( EK C ,Root) adds an algorithm to the blockchain data and runs on the blockchain, ZKP_KeyGen(1 λ ,C)→(EK C ,VK C ) Input the security parameter λ and circuit C in the computing task and calculate the key pair (EK C ,VK C ),EK C Used to generate zero-knowledge proof, VK C Used to verify zero-knowledge proofs.
[0074] The data owner sends a data initialization request to the blockchain and cloud server, specifying the creation of a default zero value Zero, where Zero is used to calculate the root hash value of each layer of subtrees when the stored data is empty. After receiving the request, the blockchain initializes the data authentication storage and first creates a smart contract account in the state tree. When the data is empty, the initial authentication evidence is calculated through the data owner's local zero-knowledge proof, and the contract code of the relevant operation is stored in the Code field. Later, in the fair audit phase, the smart contract reads this field information to perform fair and public audit operations.
[0075] The blockchain triggers the smart contract and stores the generated subtree root node set as authentication information through consensus in the blockchain. The specific process is as follows:
[0076] When the data owner's stream data meets the conditions, the root set of each layer of the stream data adaptive trapdoor hash tree is used as the private information Root, and its own digital signature and local time are attached to the private data as auxiliary information δ, which is submitted to zk-SNARKs to generate a zero-knowledge proof.
[0077] Root=root1,root2,,root n
[0078] δ=(Root,T,ID u )
[0079] Random number r, calculate the hash value of r and auxiliary information δ: H(δ,r).
[0080] The data owner generates a certified electronic signature Sign through the private key:
[0081]
[0082] The data owner builds circuit C: Circuit input common parameters PK1, PK2, ..., PK n And private data Root, the output result R and hash value h verify the authenticity and availability of the data.
[0083] (<Root,r> )=
[0084] ( <root1,root2,...,root n ,r>)→(R,h)
[0085] Generate the key EK by zero-knowledge proof C ,The auxiliary information δ of the data owner, the output result R and hash value h generated by the circuit C, and the digital signature together generate the authentication information π of the zero-knowledge proof. It is stored on the blockchain as authentication information.
[0086] BC Append (EK C ,δ,σ a )→π
[0087] GenProof Server (SNode,i)→(m',auth') is the data challenge algorithm. After receiving the query information i from the data accessor, the cloud server returns the data information m' at position i, and also returns the corresponding authentication information auth', the data generation time T and the data owner ID.
[0088] GenProof User (auth')→(R',h') is a data integrity authentication generation algorithm, which is run by the data accessor. When the data accessor receives the query result and authentication path information from the cloud server, it performs calculations locally and restores the subtree root Root to which the data belongs through the path information. j , and the verification evidence message of the corresponding data, and send it to the blockchain to trigger the smart contract for verification.
[0089] Verify Contact (VK C ,π,R′,h′,σ a )→(true / flase) is a data verification algorithm, which is executed by the blockchain smart contract. The verification evidence δ′ returned by the query algorithm is used to verify the integrity of the data information at the i-th position and the integrity of the sequence information.
[0090] The smart contract first verifies the signature of the data owner using the public key, and then verifies the zero-knowledge proof using the verification key of zk-snark. After verification, the smart contract automatically compares the data owner's zero-knowledge proof π, calculation result R, hash value h, the cloud service organization's zero-knowledge proof π′, calculation result R′, hash value h'. If the verification result is correct, output 1; otherwise output 0.
[0091] Furthermore, based on the above method, the present invention also provides a streaming data black box real-time verification system based on on-chain and off-chain hybrid storage, comprising: a data owner, a cloud storage, a blockchain and a data accessor, wherein:
[0092] The data owner uses the chameleon hash function to construct a trapdoor hash tree for the flow data in the outsourcing scenario, uploads the flow data related information and the authentication path to the cloud server, and uploads the flow data verification evidence to the blockchain. The trapdoor hash tree is used to represent the authenticatable data structure corresponding to the flow data in the outsourcing scenario of the data owner and is adaptively expanded according to the change of the flow data scale; the flow data verification evidence is obtained based on the subtree root information fixed at each layer of the trapdoor hash tree, the data owner ID and the root node upload timestamp;
[0093] The cloud server is used to store and maintain information related to the stream data and respond to data query requests from data visitors;
[0094] The blockchain, as a third party that performs fair audits, stores dynamic root node verification evidence of streaming data, and decentralized smart contracts perform verification in fair audits;
[0095] The data accessor obtains the authentication-related data of the queried content by obtaining data access rights and initiating a query request to the cloud server. Based on the authentication-related data, the subtree root to which the queried content belongs and the corresponding verification evidence are restored and sent to the blockchain. The authentication-related data includes the encrypted data block corresponding to the query request, authentication information, data owner and data generation time.
[0096] In the embodiment of this case, a decentralized blockchain structure is introduced as a fair third-party audit in the application scenario of outsourced storage of streaming data. A hybrid storage authenticatable data structure on-chain and off-chain is constructed based on a dynamic trapdoor hash tree. It is constructed in real time with the dynamic growth and change of the big data stream, and there is no need to specify the depth at the initial construction, thus realizing the instant verification of the data stream. The verification is performed by a smart contract with a decentralized structure, which improves the credibility of the verification node in the streaming data outsourcing scenario. At the same time, a zero-knowledge proof algorithm is introduced in the data integrity authentication process, which controls the complexity of on-chain authentication within logarithmic time, conceals relevant authentication information, effectively prevents malicious users from denying, and improves the security of the hybrid storage structure on-chain and off-chain.
[0097] Unless otherwise specifically stated, the relative steps, numerical expressions and values of the components and steps set forth in these embodiments do not limit the scope of the present invention.
[0098] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
[0099] The units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person of ordinary skill in the art may use different methods to implement the described functions for each specific application, but such implementation is not considered to be beyond the scope of the present invention.
[0100] Those skilled in the art will appreciate that all or part of the steps in the above method can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk or an optical disk. Optionally, all or part of the steps in the above embodiment can also be implemented using one or more integrated circuits, and accordingly, each module / unit in the above embodiment can be implemented in the form of hardware or in the form of software function modules. The present invention is not limited to any specific form of combination of hardware and software.
[0101] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims
1. A black box real-time verification method for streaming data based on on-chain and off-chain hybrid storage, characterized in that: Include: The data owner uses the chameleon hash function to construct a trapdoor hash tree for the flow data in the outsourcing scenario, uploads the flow data related information and authentication path to the cloud server, and uploads the flow data verification evidence to the blockchain. The trapdoor hash tree is used to represent the authenticatable data structure corresponding to the flow data in the outsourcing scenario of the data owner and is adaptively expanded according to the change of the flow data scale; the flow data verification evidence is obtained based on the subtree root information fixed at each layer of the trapdoor hash tree, the data owner ID and the root node upload timestamp; The data accessor who obtains data access rights obtains the authentication-related data of the queried content by initiating a query request to the cloud server, and restores the subtree root and corresponding verification evidence to the queried content based on the authentication-related data and sends them to the blockchain. The authentication-related data includes the encrypted data block corresponding to the query request, authentication information, data owner, and data generation time; The blockchain uses preset smart contract functions to audit the subtree root and verification evidence to which the query content belongs, and feeds back the audit results to the data accessor.
2. The black box real-time verification method for streaming data based on on-chain and off-chain hybrid storage according to claim 1 is characterized in that: The data owner uses the Chameleon hash function to construct a trapdoor hash tree for the downstream data in the outsourcing scenario, including: The data owner encrypts the current stream data to obtain the encrypted ciphertext, and generates a data block based on the encrypted timestamp, data owner ID and encrypted ciphertext; A hash tree is established with the data blocks corresponding to each flow data in the outsourcing service scenario as leaf nodes. When the flow data scale meets the specified conditions, the tree structure is expanded by increasing the tree structure depth, and the original tree structure is changed into the left subtree of the newly generated root node, so as to use the expanded tree structure to adapt to the current flow data scale.
3. The black box real-time verification method for streaming data based on on-chain and off-chain hybrid storage according to claim 2 is characterized in that: When the stream data scale meets the specified conditions, including: after inserting the specified number of data blocks into the hash tree, the hash tree structure forms a full binary tree structure, the specified number is 2 i , i is the number of subtree root nodes.
4. The black box real-time verification method for streaming data based on on-chain and off-chain hybrid storage according to claim 1 is characterized in that: Upload stream data verification evidence to the blockchain, including: The data owner sends a data initialization request to the blockchain and cloud server, and creates a default zero value, which is used to calculate the root hash value of each layer of subtree when the stored data is empty; After receiving the initialization request, the blockchain creates a smart contract account in the state tree, and obtains the initial authentication evidence through the local zero-knowledge proof of the data owner when the data is empty, and stores the contract code of the relevant operation in the specified field, so as to perform fair and public audit operations by reading the specified field information during the audit phase; When the data owner's stream data meets the conditions, verification evidence is generated based on the smart contract and using the set of generated subtree root nodes, and the verification evidence is uploaded to the blockchain for storage through consensus.
5. The black box real-time verification method for streaming data based on on-chain and off-chain hybrid storage according to claim 4 is characterized in that: The verification evidence is generated using the set of generated subtree root nodes, including: The root set of each layer of the subtree of the stream data adaptive trapdoor hash tree is used as private information, and the digital signature and local time are attached to the private information to generate auxiliary information; The hash value of the auxiliary information is calculated using random numbers, and the hash value is electronically signed using a private key. Verification evidence is obtained based on the electronic signature, auxiliary information, and the key generated by the zero-knowledge proof.
6. The black box real-time verification method for streaming data based on on-chain and off-chain hybrid storage according to claim 1 is characterized in that: Upload the stream data verification evidence to the blockchain, which also includes: The bottom leaf nodes in the hash tree generate verification evidence with the root node of the smallest subtree in which they are located. And every time a new subtree root node is generated in the adaptive expansion of the hash tree, the root node set regenerates verification evidence through the zero-knowledge proof algorithm and uploads it to the blockchain.
7. The black box real-time verification method for streaming data based on on-chain and off-chain hybrid storage according to claim 1 is characterized in that: The blockchain uses the preset smart contract function to audit the subtree root and verification evidence of the query content, including: The verification evidence is verified using the data owner's public key and the specified verification key. After the verification is passed, the integrity of the stream data is verified by comparing the zero-knowledge proof, calculation results and hash values of the data owner and the cloud server.
8. A streaming data black box real-time verification system based on on-chain and off-chain hybrid storage, characterized in that: Includes: data owners, cloud storage, blockchain and data accessors, among which, The data owner uses the chameleon hash function to construct a trapdoor hash tree for the flow data in the outsourcing scenario, uploads the flow data related information and the authentication path to the cloud server, and uploads the flow data verification evidence to the blockchain. The trapdoor hash tree is used to represent the authenticatable data structure corresponding to the flow data in the outsourcing scenario of the data owner and is adaptively expanded according to the change of the flow data scale; the flow data verification evidence is obtained based on the subtree root information fixed at each layer of the trapdoor hash tree, the data owner ID and the root node upload timestamp; The cloud server is used to store and maintain information related to the stream data and respond to data query requests from data visitors; The blockchain, as a third party that performs fair audits, stores dynamic root node verification evidence of streaming data, and decentralized smart contracts perform verification in fair audits; The data accessor obtains the authentication-related data of the queried content by obtaining data access rights and initiating a query request to the cloud server. Based on the authentication-related data, the subtree root to which the queried content belongs and the corresponding verification evidence are restored and sent to the blockchain. The authentication-related data includes the encrypted data block corresponding to the query request, authentication information, data owner and data generation time.
9. An electronic device, characterized in that: include: at least one processor, and a memory coupled to the at least one processor; The memory stores a computer program, and the computer program can be executed by the at least one processor to implement the method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 7 can be implemented.
Citation Information
Cited By
Expandable block chain identity authentication method and system for distributed resource aggregation scene
CN119675935A
