A big data dynamic updating method based on nested multi-branch path tree

Abstract

The application discloses a big data dynamic updating method based on nested multi-branch path tree; is used for solving the problem that a user manages a large number of data files in a cloud storage server and dynamically updates the large number of data files; the nested multi-branch path tree is divided into an external multi-branch path tree and an internal multi-branch path tree; by using the nested multi-branch path tree as a storage structure, the user stores root hash values of a plurality of files managed by the user on leaf nodes of the external multi-branch path tree, and finds corresponding files through the root hash values; the internal multi-branch path tree stores data blocks of file segmentation, and dynamic updating operations on the data blocks and dynamic updating operations on the files are realized; compared with the prior art, the application can store a plurality of files to realize big data storage, and improve dynamic updating efficiency.

Description

Technical Field

[0001] This invention belongs to the field of information security, specifically a method for dynamic updating of big data based on nested multi-branch path trees. Background Technology

[0002] With the advent of the big data era and the explosive growth of data, users are storing more and more data on cloud storage servers. As the amount of data increases, the operation of dynamically updating data becomes slow. Traditional methods for dynamically updating data initially used linked lists. However, linked lists have relatively low search efficiency. For files stored on cloud servers, search operations far outweigh additions and deletions. Therefore, the use of Merkle hash trees to store files gradually emerged. Currently, the mainstream data structure for dynamic updates is still the Merkle hash tree. However, while the Merkle hash tree can effectively perform dynamic updates, its fixed branch count of 2 means that as the data volume increases, the height of the Merkle hash tree grows, reducing the efficiency of dynamic updates. Furthermore, it only supports storing one file at a time. Later, multi-branch path trees were introduced based on the Merkle hash tree. These trees have multi-branch characteristics, reducing the tree height, but they can only store one file at a time, making them unsuitable for users managing multiple files. Summary of the Invention

[0003] To address the challenge of managing and dynamically updating large amounts of data files on cloud storage servers, this invention proposes a dynamic big data update method based on nested multi-branch path trees. The nested multi-branch path tree consists of an outer multi-branch path tree and an inner multi-branch path tree. Using this nested multi-branch path tree as the storage structure, the user stores the root hash values ​​of multiple managed files in the leaf nodes of the outer multi-branch path tree and locates the corresponding file using the root hash value. The inner multi-branch path tree stores data blocks segmented from the file, enabling dynamic updates to both data blocks and the file itself. Compared to Merkle hash trees, which can only store a single file, this method can store multiple files, achieving large data storage and improving dynamic update efficiency.

[0004] To address the aforementioned technical problems, this invention proposes a dynamic big data update method based on nested multi-branch path trees. This method uses a nested multi-branch path tree as the storage structure to store multiple files, thereby achieving large data storage and improving dynamic update efficiency. The method includes the following steps:

[0005] Step 1, Establish a system model: The system model includes users, cloud storage servers, third-party auditing institutions, and private key generators; the users store multiple files they own in the cloud storage servers;

[0006] Step 2, Initialize system parameters: The user uses the private key generator to set system parameters, thereby generating system parameters and the user's public-private key pair (pk, sk); the system parameters are: Let G, G T Let G be two multiplicative cyclic groups of the same prime order p, and g be a generator of G; let e: G × G → G T Let H be a bilinear mapping; H: {0, 1} * →G is a secure hash function;

[0007] Step 3, calculate data block labels and file labels: The user has n files {F1, F2, ..., F...} i}(1≤i≤n), first, for each file F i Divide into m data blocks B, i.e., {F i}={B i1 B i2 B ij}(1≤i≤n,1≤j≤m); then use the private key to encrypt all data blocks {B ij}, generate encrypted data blocks Finally, calculate the data block label set {σ} for all data blocks. ij} and the set of file tags for all big data files F {σ i}, where, for file F i The data block label for the j-th data block is:

[0008]

[0009] In equation (1), H(B) ij ) represents file F i The unique identifier of the j-th data block, and j auxiliary variables u are randomly selected. j ∈G(1≤j≤m);

[0010] Calculate file tags: File F i The file tags are:

[0011]

[0012] In equation (2), H(F) i ) represents the unique identifier of the file, and j auxiliary variables u are randomly selected. j ∈G(1≤j≤m);

[0013] Step 4: Construct a nested multi-branch path tree and upload files: A nested multi-branch path tree consists of an outer multi-branch path tree and multiple inner multi-branch path trees. A leaf node of an inner multi-branch path tree stores the hash value of a data block. Each inner multi-branch path tree represents a file. The hash value of the root node of each inner multi-branch path tree serves as the leaf node of the outer multi-branch path tree, thus constructing the nested multi-branch path tree. After the user stores multiple files into a nested multi-branch path tree data structure, the user will use a private key to sign the hash values ​​of the root nodes of the outer and inner multi-branch path trees, and then send them along with the entire tree structure to the cloud storage server. After that, the user will delete their locally stored files.

[0014] Step 5, Integrity Verification: Before making a dynamic update, the user needs to verify the integrity of the data in the cloud storage server. The user sends an integrity verification request to a third-party auditing agency. After receiving the user's integrity verification request, the third-party auditing agency sends a challenge message to the cloud storage server. After receiving the challenge message from the third-party auditing agency, the cloud storage server generates integrity evidence. The third-party auditing agency verifies the integrity evidence. If the verification passes, it proves that the data stored on the cloud storage server is complete and returns TRUE; otherwise, it proves that the data stored on the cloud storage server is incomplete and returns FALSE.

[0015] Step 6, Dynamic Update: When a user modifies, inserts, or deletes data stored on the cloud server, dynamic operations are performed. Since the data structure used is a nested multi-branch path tree structure, there are two types of dynamic update schemes: dynamic update operations on data blocks in the internal multi-branch path tree and dynamic update operations on files in the external multi-branch path tree.

[0016] Furthermore, in the big data dynamic update method described in this invention:

[0017] Step 4 includes the following:

[0018] 4-1) Calculate the root hash value and signature of the internal multi-branch path tree: Construct the internal multi-branch path tree using the hash values ​​of the data blocks, and calculate the root hash value H(interMBT) of the internal multi-branch path tree for the i-th file. i ), and generate the corresponding signature sig(H(interMBT) i ))=H(interMBT i ) sk This is used to check whether the internal multi-branch path tree is complete.

[0019] 4-2) Calculate the root hash value and signature of the external multi-branch path tree: Construct the external multi-branch path tree using the hash value of the internal multi-branch path tree, calculate the root hash value H(R) of the external multi-branch path tree, and generate the corresponding signature sig(H(R))=H(R). sk It is also used to check whether the entire nested multi-branch path tree is complete;

[0020] 4-3) Upload the file; the user uploads the encrypted data block obtained in step three. Data block label set {σ ij}, file tag set {σ i} and the root hash value H(interMBT) of the internal multi-branch path tree obtained in step four. i ) and the corresponding signature sig(H(interMBT) i The root hash value H(R) of the external multi-branch path tree and the corresponding signature sig(H(R)) are uploaded to the cloud storage server. After the cloud storage server returns the storage response, the user deletes the local storage data.

[0021] Step 5 includes the following:

[0022] 5-1) TPA issues a challenge message: The user sends an integrity verification request to the TPA, and the TPA randomly selects a set L = {s1, s2, ..., sn} consisting of c data blocks from the set [1, n]. c TPA randomly selects c auxiliary variables v l ∈G, 1≤l≤c;

[0023] Finally, TPA sends a challenge message Q = {L, v} to CSP. l (1≤l≤c);

[0024] 5-2) CSP generates integrity evidence: After receiving the challenge information Q from TPA, CSP first finds the node corresponding to the challenged file, and represents the auxiliary information on the authentication path from that node to the root node as {∩ l (1≤l≤c);

[0025] CSP calculates σ and μ on the received data, that is:

[0026]

[0027]

[0028] Next, CSP will prove P = {σ, μ{H(F)} l ), ∩ l The entire sig(H(R)) is returned to the TPA, which then processes the response data.

[0029] 5-3) TPA verifies integrity evidence: After receiving the proof P sent by the CSP, the TPA verifies the integrity evidence. l ), ∩ l Generate a new root node value R' and then verify whether the equation holds true:

[0030] e(sig(H(R)),g)=e((H(R')),g sk )

[0031] If the equation is false, return FALSE; otherwise, verify the integrity of the requested data block based on the challenge information Q and the proof P; determine whether the equation is true:

[0032]

[0033] If the equation is true, it means the data is intact and has not been corrupted, and TRUE is returned; otherwise, the data is corrupted, and FALSE is returned.

[0034] Step 6 includes:

[0035] 6-1) Data block modification operation: The user wants to modify the θ-th data block in the α-th internal multi-branch path tree to... Users generate new labels for new data blocks, namely:

[0036]

[0037] The user updates the information in step 4-3), and σ α，θ′ Add to the data block label set {σ ij} and recalculate the encrypted data block. File tag set {σ i The root hash value H(interMBT) of the internal multi-branch path tree. i ) and the corresponding signature sig(H(interMBT) i The root hash value H(R) and the corresponding signature sig(H(R)) of the external multi-branch path tree are uploaded to the cloud storage server for update operations.

[0038] 6-2) Data block insertion operation: Suppose the user intends to insert a new data block before the (θ+1)th data block in the αth internal multi-branch path tree. The user generates new labels for the data blocks to be inserted, namely:

[0039]

[0040] The user updates the information in step 4-3), and adds the data block label set {σ}. ij} in σα，θ Replace with σ α，β And recalculate the encrypted data blocks. File tag set {σ i The root hash value H(interMBT) of the internal multi-branch path tree. i ) and the corresponding signature sig(H(interMBT) i The root hash value H(R) and the corresponding signature sig(H(R)) of the external multi-branch path tree are uploaded to the cloud storage server for update operations.

[0041] 6-3) Data block deletion operation: Suppose the user wants to delete the data block at position θ in the α-th internal multi-branch path tree; the user first labels the data block to be deleted with σ. α，θ Perform the deletion; then, set the hash value corresponding to the tag to null;

[0042] The user updates the information in step 4-3), and adds the data block label set {σ}. ij} in σ α，θ Replace with null and recalculate the encrypted data block. File tag set {σ i The root hash value H(interMBT) of the internal multi-branch path tree. i ) and the corresponding signature sig(H(interMBT) i The root hash value H(R) and the corresponding signature sig(H(R)) of the external multi-branch path tree are uploaded to the cloud storage server for update operations.

[0043] 6-4) File modification operations: Assume the α-th internal multi-branch path tree interMBT α Change to interMBT' α ;

[0044] 6-5) File insertion operation: Insert one external multi-branch path tree before another, assuming the user is in the αth interMBT. α Insert new interMBT before β ;

[0045] 6-6) File deletion operation: Deleting an external multi-branch path tree is similar to inserting it; it refers to deleting the external multi-branch path tree at a specific location. After receiving the update request message, the cloud storage server performs the deletion operation; first, it deletes the interMBT. α And H(interMBT) α The root node R' is set to null, and then a new root node R' is generated. Otherwise, the deletion process is similar to the modification and insertion operations.

[0046] Furthermore, step 6-4) includes:

[0047] (1) File chunking: The user first chunks the interMBT' α Divide the message into m blocks and re-encrypt the blocks to obtain the desired message. Regenerate interMBT α New tag:

[0048]

[0049] (2) User sends update request: The user sends an update request to the third-party auditing firm, which then sends the update request to the cloud storage server; Update request: Where M represents the modification operation, and α represents the modification of the αth internal multi-branch path tree;

[0050] (3) The cloud storage server generates an update certificate; after receiving the update request, the cloud storage server replaces the data in the original α-th file. Replace with The file's tag σ α Replace with σ α‘ Recalculate the hash value of the α-th file, H(interMBT) α Replace ) with H(interMBT' α The cloud storage server returns an update proof V, and regenerates a new root R'. update =(sig(H(R)),R′,H(interMBT α Send it to a third-party auditing firm;

[0051] (4) A third-party auditing firm performs updated verification; the third-party auditing firm uses H(interMBT) α ) and ∩ α Generate the root node R of the original nested multi-branch path tree. Verify the integrity of the data before the cloud storage server performs the data update. If the verification fails, output FALSE; otherwise, the third-party auditing agency continues the verification. The third-party auditing agency uses H(interMBT') α ) and ∩ α Generate the root node R”' of the updated nested multi-branch path tree and compare it with R' to verify whether the cloud storage server has correctly executed the update operation. If they match, output TRUE; otherwise, output FALSE. If the final result is TRUE, the user signs the recalculated root node of the nested multi-branch path tree with sig(H(R′)) and sends it to the cloud storage server for update. After the update operation is executed, delete the user's local data.

[0052] Step 6-5) includes:

[0053] (1) File chunking: The user first chunks the interMBT file. β Divide the message into m blocks and re-encrypt the blocks to obtain the desired message. Regenerate new file tags:

[0054]

[0055] (2) User sends update request: The user sends an update request to the third-party auditing firm, which then sends the update request to the cloud storage server; Update request:

[0056]

[0057] Where I represents the insertion operation, and α indicates that the insertion is performed after the αth internal multi-branch path tree;

[0058] (3) The cloud storage server generates an update certificate; storage Add σ to the file's tag set. β Recalculate the hash value H(interMBT) of the new file. β And regenerate a new root R'; record the auxiliary path ∩ from the new file to the root node. β The cloud storage server returned an update certificate V. update =(sig(H(R)), R′, ∩ β Send it to a third-party auditing firm;

[0059] (4) A third-party auditing firm performs updated verification; the third-party auditing firm uses H(interMBT) α ) and ∩ α Generate the root node R of the original nested multi-branch path tree. Verify the integrity of the data before the cloud storage server performs the data update. If the verification fails, output FALSE; otherwise, the third-party auditing agency continues the verification. The third-party auditing agency uses H(interMBT) β 'and ∩ β Generate the root node R”' of the updated nested multi-branch path tree and compare it with R' to verify whether the cloud storage server has correctly executed the update operation. If they match, output TRUE; otherwise, output FALSE. If the final result is TRUE, the user signs the recalculated root node of the nested multi-branch path tree with sig(H(R′)) and sends it to the cloud storage server for update. After performing the insertion operation, delete the user's local data.

[0060] Compared with the prior art, the beneficial effects of the present invention are:

[0061] This invention uses a nested multi-branch path tree storage structure, allowing each leaf node of the outer multi-branch path tree to store a file. Compared to traditional Merkle hash trees and multi-branch path trees, which can only store one file, this scheme increases the number of files that can be stored, thus enabling large data storage. In addition, compared to traditional Merkle hash trees, the tree depth is reduced when storing files of the same size, improving the efficiency of dynamic updates.

[0062] This invention enables dynamic updates to both files and data blocks. Compared to traditional Merkle hash trees and multi-branch path trees, which can only dynamically update data blocks, this invention supports not only dynamic updates to data blocks but also dynamic updates to files. Attached Figure Description

[0063] Figure 1 This is a nested multi-branch path tree structure diagram constructed in the method of this invention;

[0064] Figure 2 This is a system flowchart of the method of the present invention;

[0065] Figure 3 This is a schematic diagram of the data block modification operation in the dynamic update operation of this invention;

[0066] Figure 4 This is a schematic diagram of the data block insertion operation in the dynamic update operation of this invention;

[0067] Figure 5 This is a schematic diagram of the data block deletion operation in the dynamic update operation of this invention;

[0068] Figure 6 This is a schematic diagram of file modification operations in the dynamic update operation of this invention;

[0069] Figure 7 This is a schematic diagram of the file insertion operation in the dynamic update operation of this invention;

[0070] Figure 8 This is a schematic diagram of the file deletion operation in the dynamic update operation of this invention. Detailed Implementation

[0071] The process of the big data dynamic update method based on nested multi-branch path trees proposed in this invention is as follows: Figure 2As shown, there are six steps: establishing a system model, initializing system parameters, calculating data block labels and file labels, constructing a nested multi-branch path tree and uploading files, integrity verification, and dynamic updates. In the step of constructing a nested multi-branch path tree and uploading files, each inner multi-branch path tree stores one file, and the hash value of the entire inner multi-branch path tree is stored in the root node of the outer multi-branch path tree. Using this nested pattern, the storage capacity of data files can be doubled, realizing big data storage. The dynamic update section introduces the dynamic update operations of data blocks and files.

[0072] The following details the specific steps:

[0073] Step 1: Establish a system model

[0074] The system model of this solution mainly consists of four entities: user, cloud storage server (CSP), third-party auditing agency (TPA), and private key generator (PKG).

[0075] A Cloud Storage Server (CSP) is an entity that provides cloud storage services; a Third-Party Auditor (TPA) receives user requests and initiates data integrity verification challenges to the CSP on behalf of the user. After receiving evidence from the CSP, the TPA performs integrity verification. If the verification fails, the TPA immediately notifies the user and then takes a series of penalty actions; a Private Key Generator (PKG) is an honest and trustworthy entity responsible for generating public parameters and creating secure and trustworthy keys for the user, reducing the user's related complex calculations.

[0076] Step 2: Initialize system parameters

[0077] The private key generator (PKG) sets system parameters and generates public and private key pairs for system parameters and users.

[0078] 2-1) Generate system parameters: Let G, G T Let G be two multiplicative cyclic groups of the same prime order p, and g be a generator of G; let e: G × G → G T Let H be a bilinear mapping; H: {0, 1} * →G is a secure hash function, Z p It represents a set of integers.

[0079] 2-2) Generate the user's public-private key pair: Based on the user's identity information, the PKG generates a corresponding public-private key pair (pk, sk) for the user, where pk = g sk And transmit it to the user through a secure channel.

[0080] Step 3: Calculate data block labels and file labels.

[0081] Suppose there are n files F in a user management project. Each file needs to be divided into m data blocks, then the data blocks are encrypted, and finally the data block tags and file tags are calculated.

[0082] 3-1) Large data file splitting into data blocks: The user divides each large data file into m data blocks B, and the final split file is represented as: {F i}={B i1 B i2 B ij (1≤i≤n,1≤m).

[0083] 3-2) Encrypting Data Blocks: After performing the above file splitting operation, the user encrypts the data blocks using their private key, generating the encrypted file as follows:

[0084] 3-3) Calculate data block labels: The label of the j-th data block under the i-th large data file is:

[0085]

[0086] Among them, H(B) ij ) represents a unique identifier for the data block, and j auxiliary variables u are randomly selected. j ∈G.

[0087] 3-4) Calculate file tags: For the i-th file F stored in the internal multi-branch path tree i Calculate file tags:

[0088]

[0089] Among them, H(F) i ) represents the unique identifier of the file, and j auxiliary variables u are randomly selected. j ∈G, 1≤j≤m.

[0090] Step 4: Construct a nested multi-branch path tree and upload the file.

[0091] The main idea behind constructing a nested multi-branch path tree is to store a data block in the leaf node of an inner multi-branch path tree, store a file in another inner multi-branch path tree, and finally store multiple files in an outer multi-branch path tree. The inner and outer multi-branch path trees together constitute a nested multi-branch path tree. After a user stores their multiple files into a nested multi-branch path tree data structure, the user will use their private key to sign the hash values ​​of the root nodes of the outer and inner multi-branch path trees, and then send the hash values ​​along with the entire tree structure to the CSP before deleting their locally stored files.

[0092] Nested multi-branch path tree structure, such as Figure 1 As shown, the user owns three files F1, F2, and F3, each consisting of three data blocks B1, B2, and B3. All files owned by the user are stored as a nested multi-branch path tree. The dotted box represents the inner multi-branch path tree, and the rest represent the outer multi-branch path tree. The hash value of file F1 is calculated as: H(F1) = H(H(B1) + H(B2) + H(B3)), and so on. The hash value of file F2 is: H(F2) = H(H(B1) + H(B2) + H(B3)), and the hash value of file F3 is: H(F3) = H(H(B1) + H(B2) + H(B3)). The root hash value is: H(R) = H(H(F1) + H(F2) + H(F3)).

[0093] 4-1) Calculate the hash value and signature of the root node of the internal multi-branch path tree: The user obtains the corresponding parameters from the PKG private key generator, and calculates the hash value H(interMBT) of the i-th internal multi-branch path tree based on the internal multi-branch path tree stored at the i-th position. i ), and generate the corresponding signature sig(H(interMBT) i ), used to check whether the internal multi-branch path tree is complete.

[0094] 4-2) Calculate the hash value and signature of the root node of the external multi-branch path tree: For storage of large amounts of data, the root node of the external multi-branch path tree stores the hash values ​​of multiple internal multi-branch path trees. Calculate the hash value H(R) = H(outerMBT) of the root node R of the external multi-branch path tree. i And generate the corresponding signature sig(H(R)) = sigH(outerMBT) i ))=H(outerMBT i ) sk It is also used to check whether the entire nested multi-branch path tree is complete.

[0095] 4-3) The user thus obtains a set of encrypted files and tags; including a collection of the entire encrypted data block file. The set of labels for the data block {σ ij}, the set of tags for a file composed of data blocks {σ i}, Internal multi-branch path tree signature sig(H(interMBT) i )) and external multi-branch path tree signature sigH(outerMBT) i The user uploads the encrypted file and tag set to the CSP, and then deletes the local storage data after the CSP returns a storage response.

[0096] Step 5, Integrity Verification

[0097] When a user wants to verify the integrity of data in a cloud storage server, the user sends an integrity verification request to the TPA. After receiving the user's integrity verification request, the TPA sends a challenge message Q to the CSP. After receiving the challenge message Q from the TPA, the CSP generates integrity evidence P. The TPA verifies the integrity evidence P. If the verification passes, it proves that the data stored on the CSP is complete and returns TRUE. Otherwise, it proves that the data stored on the CSP is incomplete and returns FALSE.

[0098] 5-1) TPA issues a challenge message:

[0099] The user sends an integrity verification request to the TPA. The TPA randomly selects a set L = {s1, s2, ..., sn} consisting of c data blocks from the set [1, n]. c TPA randomly selects c auxiliary variables v l ∈G, 1≤l≤c.

[0100] Finally, TPA sends a challenge message Q = {L, vl} (1 ≤ l ≤ c) to CSP.

[0101] 5-2) CSP generates integrity evidence:

[0102] After receiving the challenge information Q from the TPA, the CSP first locates the node corresponding to the challenged file and represents the auxiliary information on the authentication path from that node to the root node as {∩ l (1≤l≤c).

[0103] CSP calculates σ and μ on the received data, that is:

[0104]

[0105]

[0106] Next, CSP will prove P = {σ, μ, {H(F)} l ), ∩ l}, sig(H(R))} is returned to TPA, and TPA processes the response data.

[0107] 5-3) TPA verifies integrity evidence: After receiving the proof P sent by the CSP, the TPA verifies the integrity evidence. l ), ∩ l Generate a new root node value R′ and then verify whether the equation holds true:

[0108] e(sig(H(R)),g)=e((H(R')),g sk )

[0109] If the equation is false, return FALSE; otherwise, verify the integrity of the requested data block based on the challenge information Q and the proof P; determine whether the equation is true:

[0110]

[0111] If the equation is true, it means the data is intact and has not been corrupted, and TRUE is returned; otherwise, the data is corrupted, and FALSE is returned.

[0112] Step Six: Dynamic Update Operation

[0113] When users modify, insert, or delete data stored on the cloud server, dynamic operations are performed. Because the data structure used is a nested multi-branch path tree structure, there are two types of dynamic update schemes: dynamic update operations on data blocks in the internal multi-branch path tree and dynamic update operations on files in the external multi-branch path tree. The types of dynamic update operations include: modify, insert, and delete, which will be represented by M, I, and D respectively below. Each file in a big data file consists of multiple data blocks. The update operations performed by users on data blocks will affect the operation of the entire file. Therefore, in the following introduction, we will first introduce the dynamic update operations on data blocks, and then introduce the dynamic update operations on files.

[0114] 6-1) Data Block Modification Operation: A data block modification operation refers to replacing a data block in one internal multi-branch path tree with a new data block in another internal multi-branch path tree. Suppose the user wants to modify the θ-th data block in the α-th internal multi-branch path tree to...

[0115] First, the user generates new data blocks and new labels, namely:

[0116]

[0117] Users update the big data files and tags corresponding to the data blocks. After the big data files and tags are updated, the root hash values ​​and corresponding signatures of the internal and external multi-branch path trees are recalculated. Finally, users upload the root hash values ​​and signatures to the CSP for update operations.

[0118] The nested multi-branch path tree after modifying the data block is as follows: Figure 3 As shown, data block B1 of file F2 has been modified.

[0119] 6-2) Data Block Insertion Operation: Data block insertion refers to inserting a new data block of an internal multi-branch path tree before a data block in the existing internal multi-branch path tree; assuming the user intends to insert a new data block before the (θ+1)th data block in the αth internal multi-branch path tree.

[0120] First, the user generates new labels for the data block to be inserted, namely:

[0121]

[0122] Then, the user updates the big data file corresponding to the data block and the tags corresponding to the big data file. After the big data file and tags are updated, the root hash value and corresponding signature of the internal multi-branch path tree and the external multi-branch path tree are recalculated.

[0123] Finally, the user uploads the root hash value and signature, which are then submitted to the CSP for update.

[0124] Nested multi-branch path tree after inserting data blocks, such as Figure 4 As shown, data block B4 is inserted before data block B1 in file F2.

[0125] 6-3) Deletion of data blocks: The deletion operation is simpler than the insertion and modification operations. It does not require recalculation of the data block label. It refers to deleting any specified data block in the α-th internal multi-branch path tree. Suppose the user wants to delete the data block at position θ in the α-th internal multi-branch path tree.

[0126] The user first labels the data block to be deleted as σ. α，θ The process involves deleting the data block, setting the hash value corresponding to the tag to null, updating the file and file tag corresponding to the data block, recalculating the root hash value and corresponding signature of the internal and external multi-branch path trees, and then uploading it to the CSP for updating.

[0127] Nested multi-branch path tree after deleting data blocks, such as Figure 5 As shown, data block B1 of file F2 has been deleted.

[0128] 6-4) File modification operations: Assume the α-th internal multi-branch path tree interMBT α Change to interMBT' α .

[0129] (1) File chunking: The user first chunks the interMBT' α Divide the message into m blocks and re-encrypt the blocks to obtain the desired message. Regenerate interMBT α New tag:

[0130]

[0131] (2) User sends update request: The user sends an update request to the TPA, and the TPA sends the update request to the CSP; Update request: Where M represents the modification operation, and α represents the modification of the α-th internal multi-branch path tree.

[0132] (3) CSP generates update proof: After receiving the update request, CSP replaces the data in the original α-th file. Replace with The file's tag σ α Replace with σ α‘ Recalculate the hash value of the α-th file, H(interMBT) α Replace ) with H(interMBT' α ), and regenerate a new root R'; CSP returns an updated proof V update =(sig(H(R)),R′,H(interMBT α Send it to TPA.

[0133] (4) TPA performs update verification: TPA uses H(interMBT) α ) and ∩ α Generate the root node R of the original nested multi-branch path tree, verify the integrity of the data before CSP execution and update it. If the verification fails, output FALSE; otherwise, TPA continues the verification. TPA uses H(interMBT' α ) and ∩ α Generate the root node R”' of the updated nested multi-branch path tree and compare it with R' to verify whether CSP has correctly executed the update operation. If they match, output TRUE; otherwise, output FALSE. If the final result is TRUE, the user signs the recalculated root node of the nested multi-branch path tree with sig(H(R′)) and sends it to the cloud storage server for update. After the update operation is executed, delete the user's local data.

[0134] (5) Modify the nested multi-branch path tree of the file, such as Figure 6 As shown, file F2 has been modified.

[0135] 6-5) File insertion operation: Insert one external multi-branch path tree before another, assuming the user inserts the file in the αth interMBT. α Insert new interMBT before β .

[0136] (1) File chunking: The user first chunks the interMBT file. β Divide the message into m blocks and re-encrypt the blocks to obtain the desired message. Regenerate new file tags:

[0137]

[0138] (2) User sends update request: The user sends an update request to the TPA, and the TPA sends the update request to the CSP; Update request:

[0139]

[0140] Where I represents the insertion operation, and α indicates that the insertion is performed after the αth internal multi-branch path tree.

[0141] (3) CSP generates update proof: storage Add σ to the file's tag set. β Recalculate the hash value H(interMBT) of the new file. β ), and regenerate a new root R'; record the auxiliary path ∩ from the new file to the root node. β CSP returns an updated proof V update =(sig(H(R)), R′, ∩ β Send it to TPA.

[0142] (4) TPA performs update verification: TPA uses H(interMBT) α ) and ∩ α Generate the root node R of the original nested multi-branch path tree, verify the integrity of the data before CSP execution and update it. If the verification fails, output FALSE; otherwise, TPA continues the verification. TPA uses H(interMBT) β ') and ∩ β Generate the root node R”' of the updated nested multi-branch path tree and compare it with R' to verify whether CSP has correctly executed the update operation. If they match, output TRUE; otherwise, output FALSE. If the final result is TRUE, the user signs the recalculated root node of the nested multi-branch path tree with sig(H(R′)) and sends it to the cloud storage server for update. After performing the insertion operation, delete the user's local data.

[0143] (5) Inserting a nested multi-branch path tree of a file, such as Figure 7 As shown, file F4 was inserted before file F3.

[0144] 6-6) File deletion operation: Deletion of the external multi-branch path tree is similar to the insertion operation; it refers to deleting the external multi-branch path tree at a specific location. After receiving the update request message, the CSP performs the deletion operation; first, the interMBT is deleted. α And H(interMBT) α The root node R' is set to null, and then a new root node R' is generated. Otherwise, the deletion process is similar to the modification and insertion operations.

[0145] Deleting nested multi-branch path trees of files, such as Figure 8 As shown, file F2 has been deleted.

[0146] This invention can be applied to users managing large amounts of data files in cloud storage servers and performing dynamic update operations on these large amounts of data files.

[0147] Although the present invention has been described above in conjunction with the accompanying drawings, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many improvements and changes under the guidance of the present invention without departing from the spirit of the present invention, and these improvements and changes are all within the protection scope of the present invention.

Claims

1. A method for dynamic updating of large data based on nested multi-branch path trees, characterized in that, Using a nested multi-branch path tree as the storage structure to store multiple files enables large data storage and improves dynamic update efficiency. This method includes the following steps: Step 1, Establish a system model The system model includes users, cloud storage servers, third-party auditing agencies, and private key generators; the users store multiple files they own in the cloud storage servers; Step 2, Initialize system parameters Users use a private key generator to set system parameters, thereby generating a public-private key pair (pk, sk) between the system parameters and the user; The system parameters are: Let G, G T Let G be two multiplicative cyclic groups of the same prime order p, and g be a generator of G; let e: G × G → G T It is a bilinear mapping; H:{0,1} * →G is a secure hash function; Step 3: Calculate data block labels and file labels. The user has n files {F1, F2, ..., F...} i }(1≤i≤n), first, for each file F i Divide into m data blocks B, i.e., {F i }={B i1 B i2 ,...,B ij }(1≤i≤n,1≤j≤m); then use the private key to encrypt all data blocks {B ij }, generate encrypted data blocks Finally, calculate the data block label set {σ} for all data blocks. ij } and the set of file tags for all big data files F {σ i }, where, for file F i The data block label for the j-th data block is: In equation (1), H(B) ij ) represents file F i The unique identifier of the j-th data block, and j auxiliary variables u are randomly selected. j ∈G(1≤j≤m); Calculate file tags: File F i The file tags are: In equation (2), H(F) i ) represents the unique identifier of the file, and j auxiliary variables u are randomly selected. j ∈G(1≤j≤m); Step 4: Construct a nested multi-branch path tree and upload the file. A nested multi-branch path tree consists of an outer multi-branch path tree and multiple inner multi-branch path trees. A leaf node in an inner multi-branch path tree stores the hash value of a data block. Each inner multi-branch path tree represents a file. The hash value of the root node of each inner multi-branch path tree serves as a leaf node in the outer multi-branch path tree, thus constructing the nested multi-branch path tree. When a user stores multiple files in a nested multi-branch path tree data structure, the user signs the hash values ​​of the outer and inner multi-branch path trees' roots with their private key, and then sends this signature along with the entire tree structure to the cloud storage server before deleting their locally stored files. Step 5, Integrity Verification Before making dynamic updates, users need to verify the integrity of the data on the cloud storage server. Users send an integrity verification request to a third-party auditing firm. After receiving the user's integrity verification request, the third-party auditing firm sends a challenge message to the cloud storage server. After receiving the challenge message from the third-party auditing firm, the cloud storage server generates integrity evidence. The third-party auditing firm verifies the integrity evidence. If the verification passes, it proves that the data stored on the cloud storage server is complete and returns TRUE; otherwise, it proves that the data stored on the cloud storage server is incomplete and returns FALSE. Step 6, Dynamic Update When a user modifies, inserts, or deletes data stored on a cloud server, dynamic operations are performed. Because the data structure used is a nested multi-branch path tree structure, there are two types of dynamic update schemes: dynamic update operations on data blocks in the internal multi-branch path tree and dynamic update operations on files in the external multi-branch path tree.

2. The big data dynamic update method according to claim 1, characterized in that, Step 4 includes the following: 4-1) Calculate the root hash value and signature of the internal multi-branch path tree: Construct the internal multi-branch path tree using the hash values ​​of the data blocks, and calculate the root hash value H(interMBT) of the internal multi-branch path tree for the i-th file. i ), and generate the corresponding signature sig(H(interMBT) i ))=H(interMBT i ) sk This is used to check whether the internal multi-branch path tree is complete. 4-2) Calculate the root hash value and signature of the external multi-branch path tree: Construct the external multi-branch path tree using the hash value of the internal multi-branch path tree, calculate the root hash value H(R) of the external multi-branch path tree, and generate the corresponding signature sig(H(R))=H(R). sk It is also used to check whether the entire nested multi-branch path tree is complete; 4-3) Upload the file; the user uploads the encrypted data block obtained in step three. Data block label set {σ ij }, file tag set {σ i } and the root hash value H(interMBT) of the internal multi-branch path tree obtained in step four. i ) and the corresponding signature sig(H(interMBT) i The root hash value H(R) of the external multi-branch path tree and the corresponding signature sig(H(R)) are uploaded to the cloud storage server. After the cloud storage server returns the storage response, the user deletes the local storage data.

3. The big data dynamic update method according to claim 1, characterized in that, Step 5 includes the following: 5-1) TPA issues a challenge message: The user sends an integrity verification request to the TPA, and the TPA randomly selects a set L = {s1, s2, ..., sn} consisting of c data blocks from the set [1, n]. c TPA randomly selects c auxiliary variables v l ∈G, 1≤l≤c; Finally, TPA sends a challenge message Q = {L, v} to CSP. l (1≤l≤c); 5-2) CSP generates integrity evidence: After receiving the challenge information Q from TPA, CSP first finds the node corresponding to the challenged file, and represents the auxiliary information on the authentication path from that node to the root node as {∩ l (1≤l≤c); CSP calculates σ and μ on the received data, that is: Next, CSP will prove P = {σ, μ, {H(F)} l ), ∩ l The entire sig(H(R)) is returned to the TPA, and the TPA processes the response data. 5-3) TPA verifies integrity evidence: After receiving the proof P sent by the CSP, the TPA verifies the integrity evidence. l ),∩ l Generate a new root node value R' and then verify whether the equation holds true: e(sig(H(R)),g)=e((H(R')),g sk ) If the equation is false, return FALSE; otherwise, verify the integrity of the requested data block based on the challenge information Q and the proof P; determine whether the equation is true: If the equation is true, it means the data is intact and has not been corrupted, and TRUE is returned; otherwise, the data is corrupted, and FALSE is returned.

4. The big data dynamic update method according to claim 2, characterized in that, Step 6 includes: 6-1) Data block modification operation: The user wants to modify the θ-th data block in the α-th internal multi-branch path tree to... Users generate new labels for new data blocks, namely: The user updates the information in step 4-3), and σ α,θ′ Add to the data block label set {σ ij } and recalculate the encrypted data block. File tag set {σ i The root hash value H(interMBT) of the internal multi-branch path tree. i ) and the corresponding signature sig(H(interMBT) i The root hash value H(R) and the corresponding signature sig(H(R)) of the external multi-branch path tree are uploaded to the cloud storage server for update operations. 6-2) Data block insertion operation: Suppose the user intends to insert a new data block before the (θ+1)th data block in the αth internal multi-branch path tree. The user generates new labels for the data blocks to be inserted, namely: The user updates the information in step 4-3), and adds the data block label set {σ}. ij } in σ α,θ Replace with σ α,β And recalculate the encrypted data blocks. File tag set {σ i The root hash value H(interMBT) of the internal multi-branch path tree. i ) and the corresponding signature sig(H(interMBT) i The root hash value H(R) and the corresponding signature sig(H(R)) of the external multi-branch path tree are uploaded to the cloud storage server for update operations. 6-3) Data block deletion operation: Suppose the user wants to delete the data block at position θ in the α-th internal multi-branch path tree; the user first labels the data block to be deleted with σ. α,θ Perform the deletion; then, set the hash value corresponding to the tag to null; The user updates the information in step 4-3), and adds the data block label set {σ}. ij } in σ α,θ Replace with null and recalculate the encrypted data block. File tag set {σ i The root hash value H(interMBT) of the internal multi-branch path tree. i ) and the corresponding signature sig(H(interMBT) i The root hash value H(R) and the corresponding signature sig(H(R)) of the external multi-branch path tree are uploaded to the cloud storage server for update operations. 6-4) File modification operations: Assume the α-th internal multi-branch path tree interMBT α Change to interMBT' α ; 6-5) File insertion operation: Insert one external multi-branch path tree before another, assuming the user is in the αth interMBT. α Insert new interMBT before β ; 6-6) File deletion operation: Deleting an external multi-branch path tree is similar to inserting it; it refers to deleting the external multi-branch path tree at a specific location. After receiving the update request message, the cloud storage server performs the deletion operation; first, it deletes the interMBT. α And H(interMBT) α The root node R' is set to null, and then a new root node R' is generated. Otherwise, the deletion process is similar to the modification and insertion operations.

5. The big data dynamic update method according to claim 4, characterized in that, Step 6-4) includes: (1) File chunking: The user first chunks the interMBT' α Divide the message into m blocks and re-encrypt the blocks to obtain the desired message. Regenerate interMBT α New tag: (2) User sends update request: The user sends an update request to the third-party auditing firm, which then sends the update request to the cloud storage server; Update request: Where M represents the modification operation, and α represents the modification of the αth internal multi-branch path tree; (3) The cloud storage server generates an update certificate; after receiving the update request, the cloud storage server replaces the data in the original α-th file. Replace with The file's tag σ α Replace with σ α‘ Recalculate the hash value of the α-th file, H(interMBT) α Replace ) with H(interMBT' α The cloud storage server returns an update proof V, and regenerates a new root R'. update =(sig(H(R)),R′,H(interMBT) α Send it to a third-party auditing firm; (4) A third-party auditing firm performs updated verification; the third-party auditing firm uses H(interMBT) α ) and ∩ α Generate the root node R of the original nested multi-branch path tree. Verify the integrity of the data before the cloud storage server performs the data update. If the verification fails, output FALSE; otherwise, the third-party auditing agency continues the verification. The third-party auditing agency uses H(interMBT') α ) and ∩ α Generate the root node R”' of the updated nested multi-branch path tree and compare it with R' to verify whether the cloud storage server has correctly executed the update operation. If they match, output TRUE; otherwise, output FALSE. If the final result is TRUE, the user signs the recalculated root node of the nested multi-branch path tree with sig(H(R′)) and sends it to the cloud storage server for update. After the update operation is executed, delete the user's local data.

6. The big data dynamic update method according to claim 4, characterized in that, Step 6-5) includes: (1) File chunking: The user first chunks the interMBT file. β Divide the message into m blocks and re-encrypt the blocks to obtain the desired message. Regenerate new file tags: (2) User sends update request: The user sends an update request to the third-party auditing firm, which then sends the update request to the cloud storage server; Update request: Where I represents the insertion operation, and α indicates that the insertion is performed after the αth internal multi-branch path tree; (3) The cloud storage server generates an update certificate; storage Add σ to the file's tag set. β Recalculate the hash value H(interMBT) of the new file. β And regenerate a new root R'; record the auxiliary path ∩ from the new file to the root node. β The cloud storage server returned an update certificate V. update =(sig(H(R)),R′,∩ β Send it to a third-party auditing firm; (4) A third-party auditing firm performs updated verification; the third-party auditing firm uses H(interMBT) α ) and ∩ α Generate the root node R of the original nested multi-branch path tree. Verify the integrity of the data before the cloud storage server performs the data update. If the verification fails, output FALSE; otherwise, the third-party auditing agency continues the verification. The third-party auditing agency uses H(interMBT') β ) and ∩ β Generate the root node R”' of the updated nested multi-branch path tree and compare it with R' to verify whether the cloud storage server has correctly executed the update operation. If they match, output TRUE; otherwise, output FALSE. If the final result is TRUE, the user signs the recalculated root node of the nested multi-branch path tree with sig(H(R′)) and sends it to the cloud storage server for update. After performing the insertion operation, delete the user's local data.

Images