A method, system, device and medium for dynamic verification of non-structural data integrity

Abstract

The present application discloses a method, system, device and medium for dynamic verification of the integrity of non-structured data, which is applied to the data owner in the blockchain to obtain a set of original non-structured data; based on the intrinsic relationship of the original non-structured data, a data copy of the original non-structured data is generated, and the data copy is transmitted to the data storage in the blockchain, so that the data storage generates metadata of the data copy and uploads it to the chain; the version update data of the data copy is transmitted to the data storage in the blockchain, so that the data storage generates version update information and uploads it to the chain; based on the data copy and the version update data, challenge data of the original non-structured data is generated and uploaded to the chain, so that the data storage obtains the challenge data and generates a data integrity certificate based on the data copy and uploads it to the chain, thereby enabling the blockchain to judge the data integrity based on the metadata, challenge data and data integrity certificate of the data copy and upload it to the chain. The dynamic verification of the integrity of non-structured data is realized, and the applicability is good.

Description

Technical Field

[0001] The present application relates to the technical field of data integrity verification, and more specifically, to a method, system, device and medium for dynamic verification of the integrity of unstructured data. Background Art

[0002] Big data technology is developing rapidly and becoming increasingly integrated into people's lives. To ensure the security of big data storage, data integrity verification is required to determine whether the data has been tampered with or damaged.

[0003] Data integrity verification technology typically uses a challenge-response mechanism to determine data integrity. Data owners store data in a data storage system. Before storage, the data is signed using technologies such as BLS (Boneh-Lynn-Shacham) signatures and generates challenge metadata. The data owner selects appropriate data from the challenge metadata to generate a challenge and sends the challenge to the data storage system. Based on the challenge, the data storage system generates data integrity evidence using the stored data. Data integrity auditing organizations evaluate the evidence generated by the data storage system to determine whether the data is consistent with the original data.

[0004] For example, in the scheme of distributed data integrity verification system based on GlusterFS, 1. Key generation: Generate a pair of public and private keys x and pk. 2. Data label generation: The data owner divides the data M into n blocks, M = (m1, m2, ..., mn), and generates l blocks of challenges C = (c1, c2, ..., cl), concatenates the data block mi with the challenge, calculates the summary of the concatenated data block using the hash function, and signs the summary of the concatenated data block using the private key.<i,ci> The challenge is sent to the data storer, who uses the stored data block and ci to perform hash calculation to obtain evidence and verify the integrity of the data through the BLS signature algorithm.

[0005] However, the above methods only consider verifying the integrity of individual data points. Existing data integrity verification methods operate by dividing data M into n blocks, failing to consider the unstructured nature of data in big data environments. Furthermore, these data integrity verification techniques only address the data owner, preventing data corruption or tampering during storage in the data storage system. However, they fail to consider data modifications by the data owner to conceal potential issues. This makes existing data integrity verification methods less applicable.

[0006] In summary, how to improve the applicability of data integrity verification is an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0007] The purpose of this application is to provide a method for dynamic verification of the integrity of unstructured data, which can, to a certain extent, solve the technical problem of how to improve the applicability of data integrity verification. This application also provides a system, device, and computer-readable storage medium for dynamic verification of the integrity of unstructured data.

[0008] In order to achieve the above objectives, this application provides the following technical solutions:

[0009] A dynamic verification method for the integrity of unstructured data, applied to data owners in the blockchain, includes:

[0010] Get a set of raw unstructured data;

[0011] Generate a data copy of the original unstructured data based on the intrinsic relationship of the original unstructured data, and transmit the data copy to the data storage in the blockchain, so that the data storage generates metadata for the data copy and uploads it to the blockchain; transmit version update data of the data copy to the data storage, so that the data storage generates version update information and uploads it to the blockchain;

[0012] Based on the data copy and the version update data, challenge data for the original unstructured data is generated and uploaded to the chain, so that the data storer obtains the challenge data and generates a data integrity certificate based on the data copy and uploads it to the chain, thereby enabling the blockchain to perform data integrity judgment based on the metadata of the data copy, the challenge data and the data integrity certificate and upload it to the chain.

[0013] Preferably, the step of generating challenge data of the original unstructured data based on the data copy and the version update data and uploading the challenge data to the chain includes:

[0014] obtaining a composite data domain of the data copy;

[0015] generating challenge metadata based on the composite data domain and the version update data;

[0016] The challenge data is generated based on the challenge metadata, metadata identifier, timestamp and data storage credential and uploaded to the chain.

[0017] Preferably, generating the challenge metadata based on the composite data domain and the version update data includes:

[0018] Constructing a data-domain-based hybrid Merkle DAG structure for the composite data domain;

[0019] Construct a new Merkle DAG structure based on the version update data;

[0020] Connecting the hybrid Merkle DAG structure and the new Merkle DAG structure through a Merkle binary tree to obtain a new version data structure;

[0021] Obtaining root node information in the new version data structure;

[0022] Calculating a hash value of the root node information of the composite data domain;

[0023] Generate a public-private key pair;

[0024] Sign the hash value based on the private key to obtain a signature value;

[0025] The challenge metadata is generated based on the signature value, the root node information, the public key, and the version number.

[0026] Preferably, the obtaining of the composite data domain of the data copy includes:

[0027] Dividing the data in the data replica into blocks to obtain replica data blocks;

[0028] Obtain random sequences and splicing strategies;

[0029] splicing the random sequence and the replica data block based on the splicing strategy to obtain the composite data domain;

[0030] Generating the challenge metadata based on the signature value, the root node information, the public key, and the version number includes:

[0031] The challenge metadata is generated based on the signature value, the random sequence, the splicing strategy, the root node information, the public key, and the version number.

[0032] Preferably, generating a data copy of the original unstructured data based on the intrinsic relationship of the original unstructured data includes:

[0033] Defining the intrinsic relationship of the original unstructured data and constructing an original data domain based on the intrinsic relationship;

[0034] A symmetric encryption key is obtained, and the data in the original data domain is encrypting based on the symmetric encryption key to obtain the data copy.

[0035] Preferably, uploading data to the blockchain includes:

[0036] Upload data to the blockchain based on the preset metadata structure;

[0037] Among them, the metadata structure includes ObjectType, Timestamp, Node_block, Data1, and Data2; the ObjectType identifies the type of metadata, including on-chain data, challenge data, proof data, and audit data; the Timestamp identifies the unique identifier of the metadata; the Node_block identifies the unique identifier of a batch of the original unstructured data to be verified; and the Data1 and Data2 represent the content of the metadata.

[0038] Preferably, after generating the challenge data of the original unstructured data based on the data copy and uploading it to the chain, the method further includes:

[0039] The data integrity proof is audited based on the identification of the original unstructured data and / or the type of metadata.

[0040] A dynamic verification system for the integrity of unstructured data, applied to data owners in the blockchain, including:

[0041] Unstructured data acquisition module, used to acquire a set of original unstructured data;

[0042] A data copy generation module is configured to generate a data copy of the original unstructured data based on the intrinsic relationship of the original unstructured data, and transmit the data copy to a data storage provider in the blockchain so that the data storage provider generates metadata for the data copy and uploads it to the blockchain; and transmit version update data of the data copy to the data storage provider so that the data storage provider generates version update information and uploads it to the blockchain;

[0043] The challenge data generation module is used to generate challenge data for the original unstructured data based on the data copy and the version update data and upload it to the chain, so that the data storer can obtain the challenge data and generate a data integrity certificate based on the data copy and upload it to the chain, thereby enabling the blockchain to perform data integrity judgment based on the metadata of the data copy, the challenge data and the data integrity certificate and upload it to the chain.

[0044] An electronic device, comprising:

[0045] memory for storing computer programs;

[0046] A processor is configured to implement the steps of any of the above-mentioned methods for dynamic verification of the integrity of non-structural data when executing the computer program.

[0047] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of any of the above-mentioned methods for dynamic verification of the integrity of unstructured data.

[0048] The present application provides a method for dynamic verification of the integrity of unstructured data, which is applied to the data owner in the blockchain to obtain a set of original unstructured data; based on the intrinsic relationship of the original unstructured data, a data copy of the original unstructured data is generated, and the data copy is transmitted to the data store in the blockchain, so that the data store generates metadata for the data copy and puts it on the chain; the version update data of the data copy is transmitted to the data store in the blockchain, so that the data store generates version update information and puts it on the chain; based on the data copy and the version update data, challenge data of the original unstructured data is generated and put on the chain, so that the data store generates data integrity proof based on the data copy after obtaining the challenge data and puts it on the chain, thereby making the blockchain perform data integrity judgment based on the metadata, challenge data and data integrity proof of the data copy and put it on the chain. In the present application, a data copy of the original unstructured data can be generated based on the intrinsic relationship of the original unstructured data, and challenge data of the original unstructured data can be generated based on the data copy and the version update data of the data copy, thereby realizing dynamic verification of the integrity of the unstructured data. In addition, during the whole process, the metadata, version update data, data integrity proof and data integrity judgment result of the data copy are put on the chain, which is convenient for subsequent tracing and review of the integrity verification process of the original unstructured data, and has good applicability. The non-structural data integrity dynamic verification system, electronic device and computer-readable storage medium provided in this application also solve the corresponding technical problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0050] Figure 1 A first flow chart of a method for dynamic integrity verification of unstructured data provided in an embodiment of the present application;

[0051] Figure 2 A second flow chart of a method for dynamic integrity verification of non-structural data provided in an embodiment of the present application;

[0052] Figure 3 A flowchart for generating challenge metadata based on composite data domains and version update data in this application;

[0053] Figure 4 A structural diagram of a non-structural data integrity dynamic verification system provided in an embodiment of the present application;

[0054] Figure 5Schematic diagram of the hardware structure of the electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0056] See also Figure 1 , Figure 1 This is a first flow chart of a method for dynamic integrity verification of non-structural data provided in an embodiment of the present application.

[0057] The present invention provides a method for dynamic verification of the integrity of unstructured data, which is applied to the data owner in the blockchain and may include the following steps:

[0058] Step S101: Acquire a set of original unstructured data.

[0059] In practical applications, data owners need to deploy data in a blockchain, such as a consortium chain, and obtain a set of raw unstructured data. It should be noted that unstructured data in this application refers to data with an irregular or incomplete data structure, no predefined data model, and data that is not conveniently represented by a two-dimensional database logical table. This includes office documents of all formats, text, images, XML (Extensible Markup Language), HTML (Hypertext Markup Language), various reports, images, and audio / video information. For example, a set of raw unstructured data can be government data including business data, public opinion and social data, environmental data, and decentralized public data, and business data includes statistical tables, report documents, meeting videos, and other data. In addition, according to specific needs, the blockchain can also be equipped with a contract management node, a smart contract module, a blockchain ledger module, a consensus module, etc., and the blockchain ledger module includes a state database module and a chain ledger module, etc. Among them, the contract management node is used to add the used contracts to the smart contract module, the blockchain ledger module is used to save data, and the consensus module is used to reach consensus on the data on the chain, etc. This application does not make specific limitations here.

[0060] Step S102: Generate a data copy of the original unstructured data based on the intrinsic relationship of the original unstructured data, and transmit the data copy to the data storage in the blockchain, so that the data storage generates metadata for the data copy and uploads it to the chain; transmit version update data of the data copy to the data storage in the blockchain, so that the data storage generates version update information and uploads it to the chain.

[0061] In actual applications, after obtaining a set of original unstructured data, the data owner can generate a data copy of the original unstructured data based on the intrinsic relationship of the original unstructured data, and transmit the data copy to the data store in the blockchain, such as transmitting the data copy to the data store through an off-chain secure channel, so that the integrity verification of the original unstructured data can be completed based on the data copy later. The data store also needs to be deployed in the blockchain. In this way, after receiving the data copy, the data store can generate metadata for the data copy and upload it to the chain to record in the blockchain that the data store has received the data copy. In addition, when the data owner wants to update the data version, it is necessary to transmit the data node identifier that needs to be modified and the new node data to the data store, that is, it is necessary to transmit the version update data of the data copy to the data store so that the data store generates the version update information and uploads it to the chain. It should be noted that after receiving the data copy or version update data, the data store can store the data copy or version update data in its off-chain storage facility, etc. This application does not make specific restrictions here.

[0062] Step S103: Generate challenge data for the original unstructured data based on the data copy and version update data and upload it to the chain, so that the data storer can obtain the challenge data and generate a data integrity certificate based on the data copy and upload it to the chain, thereby enabling the blockchain to make a data integrity judgment based on the metadata of the data copy, the challenge data and the data integrity certificate and upload it to the chain.

[0063] In practical applications, after the data owner generates a data copy and transmits it to the data storage, they can then generate challenge data for the original unstructured data based on the data copy and version update data and upload it to the blockchain to record the challenge data. After viewing the challenge data on the blockchain, the data storage deployed on the blockchain will obtain the challenge data, generate a data integrity proof based on the data copy, and upload it to the blockchain. The blockchain then makes a data integrity judgment based on the metadata, challenge data, and data integrity proof of the data copy and uploads it to the blockchain, completing dynamic verification of the integrity of the original unstructured data. It should be noted that in the process of making data integrity judgments based on the metadata, challenge data, and data integrity proof of the data copy, the blockchain can use an integrity judgment contract to make data integrity judgments based on the metadata, challenge data, and data integrity proof of the data copy.

[0064] In specific application scenarios, during the dynamic verification of the integrity of original unstructured data, when the data is uploaded to the chain, the data can be uploaded to the chain based on the metadata upload contract, and the data can be uploaded to the chain based on the preset metadata structure through the metadata upload contract; wherein, in order to ensure the accuracy of the data uploaded to the chain, the metadata structure may include ObjectType, Timestamp, Node_block, Data1, and Data2; ObjectType identifies the type of metadata, including uploaded data, challenge data, proof data, and audit data; Timestamp identifies the unique identifier of the metadata; Node_block identifies the unique identifier of a batch of original unstructured data to be verified; Data1 and Data2 represent the content of the metadata. It should be noted that the specific content of uploading data to the chain based on the preset metadata structure through the metadata upload contract can be determined according to the type of data required to be uploaded to the chain. For example, in the process of uploading a data copy to the chain based on the preset metadata structure through the metadata upload contract, the uplinked data may include the metadata identifier data_up, timestamp, storage data name, data storage certificate, etc.; in the process of uploading challenge data to the chain based on the preset metadata structure through the metadata upload contract, the uplinked data may include the metadata identifier challenge, timestamp, challenge metadata, data storage certificate, etc.; in the process of uploading data integrity proof to the chain based on the preset metadata structure through the metadata upload contract, the uplinked data may include the metadata identifier prove, timestamp, data integrity proof, timestamp corresponding to the challenge data, etc.; in the process of uploading data integrity judgment results to the chain based on the preset metadata structure through the metadata upload contract, the uplinked data may include the metadata identifier audit, timestamp, judgment result, timestamp corresponding to the evidence data, etc.

[0065] In specific application scenarios, after generating challenge data for the original unstructured data based on the data copy and version update data and uploading it to the chain, the data integrity proof can also be audited. For example, the data integrity proof can be audited based on the identifier and / or metadata type of the original unstructured data, or a certain data type in the uploaded data copy can be indexed and audited. It should be noted that during the data audit process, corresponding data audit operations can be performed based on the audit history index contract deployed in the blockchain, and this application does not make specific limitations here.

[0066] The present application provides a method for dynamic verification of the integrity of unstructured data, which is applied to the data owner in the blockchain to obtain a set of original unstructured data; based on the intrinsic relationship of the original unstructured data, a data copy of the original unstructured data is generated, and the data copy is transmitted to the data store in the blockchain, so that the data store generates metadata for the data copy and puts it on the chain; the version update data of the data copy is transmitted to the data store in the blockchain, so that the data store generates version update information and puts it on the chain; based on the data copy and the version update data, challenge data of the original unstructured data is generated and put on the chain, so that the data store generates data integrity proof based on the data copy after obtaining the challenge data and puts it on the chain, thereby making the blockchain perform data integrity judgment based on the metadata, challenge data and data integrity proof of the data copy and put it on the chain. In the present application, a data copy of the original unstructured data can be generated based on the intrinsic relationship of the original unstructured data, and challenge data of the original unstructured data can be generated based on the data copy and the version update data of the data copy, thereby realizing dynamic verification of the integrity of the unstructured data. In addition, during the whole process, the metadata, version update data, data integrity proof and data integrity judgment result of the data copy are put on the chain, which is convenient for subsequent tracing and review of the integrity verification process of the original unstructured data, and has good applicability.

[0067] See also Figure 2 , Figure 2 This is a second flow chart of a method for dynamic integrity verification of non-structural data provided in an embodiment of the present application.

[0068] The present invention provides a method for dynamic verification of the integrity of unstructured data, which is applied to the data owner in the blockchain and may include the following steps:

[0069] Step S201: Acquire a set of original unstructured data.

[0070] Step S202: Generate a data copy of the original unstructured data based on the intrinsic relationship of the original unstructured data, and transmit the data copy to the data storage in the blockchain, so that the data storage generates metadata for the data copy and uploads it to the chain; transmit version update data of the data copy to the data storage in the blockchain, so that the data storage generates version update information and uploads it to the chain.

[0071] In practical applications, in the process of generating a data copy of the original unstructured data based on the intrinsic relationship of the original unstructured data, the intrinsic relationship of the original unstructured data can be defined, and the original data domain can be constructed based on the intrinsic relationship; a symmetric encryption key can be obtained, and the data in the original data domain can be encrypted based on the symmetric encryption key to obtain a data copy.

[0072] It should be noted that the definition of the data domain in this application is as follows: for a batch of unstructured data, the batch of unstructured data contains multiple data, the internal relationship of the batch of unstructured data is defined, and the data domain is constructed based on this relationship. The data domain defined here is a representation of the internal relationship of the unstructured data, and the data domain contains sub-data domains and multiple unstructured data. In addition, when generating a data copy, the data owner can use its own off-chain computing facility module to encrypt the data in the original data domain based on a symmetric encryption key to obtain a data copy, and can use its own off-chain storage facility module to temporarily store the data copy, etc. This application does not make specific restrictions on this.

[0073] Step S203: Acquire the composite data domain of the data copy.

[0074] Step S204: Generate challenge metadata based on the composite data domain and version update data.

[0075] In actual applications, in the process of generating challenge data of the original unstructured data based on the data copy and version update data and uploading it to the chain, the corresponding challenge data can be generated based on the composite data domain of the data copy. Specifically, the composite data domain of the data copy can be obtained first; then the challenge metadata can be generated based on the composite data domain and version update data; so that the challenge data can be subsequently generated based on the challenge metadata, metadata identifier, timestamp and data storage credentials.

[0076] It should be noted that in the process of obtaining the composite data domain of the data copy, the composite data domain of the data copy generated previously during the integrity verification process of the original unstructured data can be obtained, or the composite data domain of the data copy can be generated in real time, etc. This application does not make specific limitations here.

[0077] In a specific application scenario, in the process of generating a composite data domain of a data copy in real time, the data in the data copy can be divided into blocks to obtain replica data blocks; a random sequence and a splicing strategy are obtained, wherein the splicing strategy is a strategy that specifies the splicing method of different replica data blocks of the data copy with the random sequence, and the splicing method refers to the position of the random sequence in the replica data block; the random sequence and the replica data block are spliced ​​based on the splicing strategy to obtain a composite data domain.

[0078] Step S205: Generate challenge data based on the challenge metadata, metadata identifier, timestamp and data storage credentials and upload it to the chain, so that the data storage owner can obtain the challenge data and generate a data integrity certificate based on the data copy and upload it to the chain, thereby enabling the blockchain to make a data integrity judgment based on the metadata of the data copy, the challenge data and the data integrity certificate and upload it to the chain.

[0079] See also Figure 3 , Figure 3Flowchart for generating challenge metadata based on composite data domain and version update data in this application.

[0080] The present invention provides a method for dynamic verification of the integrity of unstructured data, which is applied to a data owner in a blockchain. The process of generating challenge metadata based on composite data domains and version update data may include the following steps:

[0081] Step S301: Construct a data domain-based hybrid Merkle DAG structure for the composite data domain.

[0082] Step S302: Construct a new Merkle DAG structure based on the version update data.

[0083] Step S303: Connect the hybrid Merkle DAG structure and the new Merkle DAG structure through a Merkle binary tree to obtain a new version data structure.

[0084] Step S304: Obtain the root node information in the new version data structure.

[0085] Step S305: Calculate the hash value of the root node information of the composite data domain.

[0086] Step S306: Generate a public-private key pair.

[0087] Step S307: Sign the hash value based on the private key to obtain a signature value.

[0088] Step S308: Generate challenge metadata based on the signature value, root node information, public key, and version number.

[0089] In practical applications, when a data owner needs to update a data structure, they can use the version update data to generate different versions of challenge metadata. The version update data includes the identifiers of the data nodes to be modified and the new node data. A new Merkle DAG structure is constructed based on the version update data. The composite data domain and the new Merkle DAG structure are connected via a Merkle binary tree to obtain a new version of the data structure. Challenge metadata is then generated based on the new version of the data structure. Specifically, in the process of generating challenge metadata based on a composite data domain, a hybrid Merkle DAG structure based on the data domain can be constructed for the composite data domain; a new Merkle DAG structure can be constructed based on the version update data; the hybrid Merkle DAG structure and the new Merkle DAG structure can be connected via a Merkle binary tree to obtain a new version of the data structure; the root node information in the new version of the data structure can be obtained; a hash value of the root node information of the composite data domain can be calculated; a public-private key pair can be generated; the hash value can be signed using the private key to obtain a signature value; and challenge metadata can be generated based on the signature value, root node information, public key, and version number.

[0090] In specific application scenarios, in the process of generating challenge metadata based on signature value, root node information, public key, and version number, challenge metadata can be generated based on signature value, random sequence, splicing strategy, root node information, public key, version number, etc. This application does not make specific limitations here.

[0091] It should be noted that the hybrid Merkle DAG structure based on the data domain can be used to structure the data domain. The acquisition of the hybrid Merkle DAG structure based on the data domain requires two steps: constructing the MerkleDAG file structure based on the data domain and obtaining the Merkle DAG node node. id Information; and the hybrid Merkle DAG structure based on the data domain contains data domain nodes and data nodes, and the data domain nodes contain node identification information node id , right pointer information, child pointer, the data node contains node identification information node id , right pointer information; and in the process of obtaining the root node information in the hybrid Merkle DAG structure, the root node information can be determined based on the Merkle tree. For example, a multi-branch balanced Merkle tree can be constructed for the data in the data domain, and the Merkle root can be obtained and used as the node id , the data domain node in the data domain obtains the node of the lower layer through the child node id Information, and use the information obtained to calculate the node of the data domain node id . In addition, in the process of generating challenge data for the original unstructured data based on the data copy, the data owner can generate challenge data for the original unstructured data based on the data copy based on its own off-chain computing facility module, and can use its own off-chain storage facility module to store the corresponding data, etc. This application does not make specific limitations here.

[0092] See also Figure 4 , Figure 4 A structural diagram of a non-structural data integrity dynamic verification system provided in an embodiment of the present application.

[0093] The present application provides a dynamic verification system for the integrity of unstructured data, which is applied to data owners in a blockchain and may include:

[0094] The unstructured data acquisition module 101 is used to acquire a set of original unstructured data;

[0095] The data copy generation module 102 is used to generate a data copy of the original unstructured data based on the inherent relationship of the original unstructured data, and transmit the data copy to the data storage in the blockchain so that the data storage generates metadata for the data copy and uploads it to the blockchain; transmit version update data of the data copy to the data storage so that the data storage generates version update information and uploads it to the blockchain;

[0096] The challenge data generation module 103 is used to generate challenge data of the original unstructured data based on the data copy and version update data and upload it to the chain, so that the data storer can obtain the challenge data and generate a data integrity certificate based on the data copy and upload it to the chain, thereby enabling the blockchain to make a data integrity judgment based on the metadata of the data copy, the challenge data and the data integrity certificate and upload it to the chain.

[0097] The present application provides a dynamic verification system for the integrity of unstructured data, which is applied to data owners in a blockchain. The challenge data generation module may include:

[0098] A composite data domain acquisition unit, used for acquiring the composite data domain of the data copy;

[0099] a challenge metadata generation unit, configured to generate challenge metadata based on the composite data domain and the version update data;

[0100] The challenge data generation unit is used to generate challenge data based on the challenge metadata, metadata identifier, timestamp and data storage credentials and upload it to the chain.

[0101] An embodiment of the present application provides a dynamic verification system for the integrity of unstructured data, which is applied to data owners in a blockchain. The challenge metadata generation unit is specifically used to: construct a hybrid Merkle DAG structure based on the data domain for a composite data domain; construct a new Merkle DAG structure based on version update data; connect the hybrid Merkle DAG structure and the new Merkle DAG structure through a Merkle binary tree to obtain a new version data structure; obtain root node information in the new version data structure; calculate a hash value of the root node information of the composite data domain; generate a public-private key pair; sign the hash value based on the private key to obtain a signature value; and generate challenge metadata based on the signature value, root node information, public key, and version number.

[0102] An embodiment of the present application provides a dynamic verification system for the integrity of unstructured data, which is applied to data owners in a blockchain. The composite data domain acquisition unit is specifically configured to: divide the data in the data replica into blocks to obtain replica data blocks; obtain a random sequence and a splicing strategy; and splice the random sequence and the replica data blocks based on the splicing strategy to obtain a composite data domain;

[0103] The challenge metadata generation unit is specifically used to generate challenge metadata based on the signature value, random sequence, splicing strategy, root node information, public key, and version number.

[0104] The embodiment of the present application provides a dynamic verification system for the integrity of unstructured data, which is applied to data owners in a blockchain. The data copy generation module includes:

[0105] A definition unit is used to define the internal relationship of the original unstructured data and construct the original data domain based on the internal relationship;

[0106] The key acquisition unit is used to obtain a symmetric encryption key, encrypt the data in the original data domain based on the symmetric encryption key, and obtain a data copy.

[0107] An embodiment of the present application provides a dynamic verification system for the integrity of unstructured data, which is applied to data owners in a blockchain and may include: an on-chain module for uploading data to the chain based on a preset metadata structure; wherein the metadata structure includes ObjectType, Timestamp, Node_block, Data1, and Data2; ObjectType identifies the type of metadata, including on-chain data, challenge data, proof data, and audit data; Timestamp identifies the unique identifier of the metadata; Node_block identifies the unique identifier of a batch of original unstructured data to be verified; Data1 and Data2 represent the content of the metadata.

[0108] The embodiment of the present application provides a dynamic verification system for the integrity of unstructured data, which is applied to data owners in a blockchain and may also include:

[0109] The audit module is used to audit the data integrity proof based on the type of the identifier and / or metadata of the original unstructured data after the challenge data generation module generates the challenge data of the original unstructured data based on the data copy and uploads it to the chain.

[0110] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiment of the present invention, the embodiment of the present invention further provides an electronic device, Figure 5 FIG. 1 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Figure 5 As shown, the electronic equipment includes:

[0111] Communication interface 1, capable of exchanging information with other devices such as network devices;

[0112] The processor 2 is connected to the communication interface 1 to implement information exchange with other devices and is used to execute the dynamic verification method for the integrity of unstructured data of an application provided by one or more of the above technical solutions when running a computer program. The computer program is stored in the memory 3.

[0113] Of course, in actual application, the various components in the electronic device are coupled together through the bus system 4. It can be understood that the bus system 4 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 4 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 5 Various buses are labeled as bus system 4.

[0114] The memory 3 in the embodiment of the present invention is used to store various types of data to support the operation of the electronic device. Examples of such data include: any computer program used to operate on the electronic device.

[0115] It is understood that the memory 3 can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a magnetic disk memory or a magnetic tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM).The memory 2 described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0116] The method disclosed in the above embodiment of the present invention can be applied to processor 2 or implemented by processor 2. Processor 2 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by the hardware integrated logic circuit in processor 2 or by instructions in software form. The above processor 2 can be a general-purpose processor, a DSP, or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. Processor 2 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiment of the present invention. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in memory 3. Processor 2 reads the program in memory 3 and completes the steps of the above method in combination with its hardware.

[0117] When the processor 2 executes the program, the corresponding processes in the various methods of the embodiments of the present invention are implemented, which will not be described here for the sake of brevity.

[0118] In an exemplary embodiment, the present invention further provides a storage medium, namely, a computer storage medium, specifically, a computer-readable storage medium, such as a memory 3 storing a computer program. The computer program can be executed by a processor 2 to perform the steps of the aforementioned method. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface mount storage, optical disk, or CD-ROM.

[0119] In the several embodiments provided in this application, it should be understood that the disclosed devices, terminals and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0120] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0121] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0122] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: mobile storage devices, ROM, RAM, disks or optical disks, etc. Various media that can store program codes.

[0123] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling an electronic device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0124] It should also be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0125] The above description of the disclosed embodiments will enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for dynamic integrity verification of non-structural data, characterized in that: Data owners in blockchain include: Get a set of raw unstructured data; Generate a data copy of the original unstructured data based on the intrinsic relationship of the original unstructured data, and transmit the data copy to the data store in the blockchain, so that the data store generates metadata of the data copy and uploads it to the blockchain; transmit version update data of the data copy to the data store, so that the data store generates version update information and uploads it to the blockchain; Generate challenge data for the original unstructured data based on the data copy and the version update data and upload the data to the chain, so that the data storage owner can obtain the challenge data and generate a data integrity certificate based on the data copy and upload the data to the chain, and then the blockchain can make a data integrity judgment based on the metadata of the data copy, the challenge data and the data integrity certificate and upload the data to the chain; The step of generating a data copy of the original unstructured data based on the intrinsic relationship of the original unstructured data includes: Defining the internal relationship of the original unstructured data, and constructing an original data domain based on the internal relationship; A symmetric encryption key is obtained, and the data in the original data domain is encrypting based on the symmetric encryption key to obtain the data copy.

2. The method according to claim 1, characterized in that The step of generating challenge data for the original unstructured data based on the data copy and the version update data and uploading the challenge data to the chain includes: obtaining a composite data domain of the data copy; generating challenge metadata based on the composite data domain and the version update data; The challenge data is generated based on the challenge metadata, metadata identifier, timestamp and data storage credential and uploaded to the chain.

3. The method according to claim 2, characterized in that The generating challenge metadata based on the composite data domain and the version update data includes: Constructing a data domain-based hybrid Merkle DAG structure for the composite data domain; Construct a new Merkle DAG structure based on the version update data; Connecting the hybrid Merkle DAG structure and the new Merkle DAG structure through a Merkle binary tree to obtain a new version data structure; Obtaining root node information in the new version data structure; Calculating a hash value of the root node information of the composite data domain; Generate a public-private key pair; Sign the hash value based on the private key to obtain a signature value; The challenge metadata is generated based on the signature value, the root node information, the public key, and the version number.

4. The method according to claim 3, characterized in that The composite data domain for obtaining the data copy includes: Dividing the data in the data copy into blocks to obtain copy data blocks; Obtain random sequences and splicing strategies; splicing the random sequence and the replica data block based on the splicing strategy to obtain the composite data domain; The generating the challenge metadata based on the signature value, the root node information, the public key, and the version number includes: The challenge metadata is generated based on the signature value, the random sequence, the splicing strategy, the root node information, the public key, and the version number.

5. The method according to claim 1, characterized in that Uploading data to the blockchain includes: Upload data to the chain based on the preset metadata structure; Among them, the metadata structure includes ObjectType, Timestamp, Node_block, Data1, and Data2; the ObjectType identifies the type of metadata, including on-chain data, challenge data, proof data, and audit data; the Timestamp identifies the unique identifier of the metadata; the Node_block identifies the unique identifier of a batch of the original unstructured data to be verified; and the Data1 and Data2 represent the content of the metadata.

6. The method according to claim 1, characterized in that After generating the challenge data of the original unstructured data based on the data copy and uploading it to the chain, the method further includes: The data integrity proof is audited based on the identification of the original unstructured data and / or the type of metadata.

7. A dynamic verification system for the integrity of unstructured data, characterized in that: Data owners in blockchain include: An unstructured data acquisition module is used to acquire a set of original unstructured data; A data copy generation module, used to generate a data copy of the original unstructured data based on the intrinsic relationship of the original unstructured data, and transmit the data copy to the data storage in the blockchain, so that the data storage generates metadata of the data copy and uploads it to the blockchain; transmit version update data of the data copy to the data storage, so that the data storage generates version update information and uploads it to the blockchain; A challenge data generation module, used to generate challenge data of the original unstructured data based on the data copy and the version update data and upload the challenge data to the chain, so that the data storage owner can obtain the challenge data and generate a data integrity certificate based on the data copy and upload the challenge data to the chain, thereby enabling the blockchain to make a data integrity judgment based on the metadata of the data copy, the challenge data and the data integrity certificate and upload the challenge data to the chain; Wherein, the data copy generation module includes: A definition unit, used for defining the internal relationship of the original unstructured data and constructing an original data domain based on the internal relationship; The key acquisition unit is used to acquire a symmetric encryption key, and encrypt the data in the original data domain based on the symmetric encryption key to obtain the data copy.

8. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the method for dynamic integrity verification of non-structural data as claimed in any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for dynamic integrity verification of non-structured data as claimed in any one of claims 1 to 6 are implemented.

Images