File hash addressing method and system based on MerkleTree
By dynamically adjusting the block partitioning strategy and constructing a MerkleTree structure, the problem of low file verification efficiency in distributed storage systems is solved, achieving efficient and secure data integrity verification.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HANGZHOU GUANGYU CLOUD TECHNOLOGY CO LTD
- Filing Date
- 2026-04-15
- Publication Date
- 2026-05-12
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In existing distributed storage systems, the fixed block strategy leads to low file verification efficiency and insufficient security, especially in files of different formats or sizes, where the critical data boundaries are cut off, affecting the efficiency and security of encryption and verification.
By obtaining the file attribute information of the target file, the block segmentation strategy is dynamically adjusted, the file attribute representation value is quantified based on the file type and data size, the hash calculation of the encrypted data block is performed, a MerkleTree structure is constructed, an addressing identifier is generated, and the integrity of the data block is verified by comparing the root hash value with the storage root hash value.
It improves block adaptability and verification efficiency, ensures data security, prevents malicious attacks, enables fast and accurate integrity verification, and optimizes the storage and transmission process.
Smart Images
Figure CN122019475A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and in particular to a file hash addressing method and system based on MerkleTree. Background Technology
[0002] In today's digital information age, data has become a core asset for businesses and individuals. With the rapid development of cloud computing, big data, and distributed storage technologies, massive amounts of data are typically stored in shards or copies across multiple physical nodes to improve storage efficiency and data availability. However, while distributed storage environments offer convenience, they also introduce serious data security challenges. Hardware failures, software errors, human error, and malicious network attacks on storage nodes can all lead to the loss or corruption of stored data. Against this backdrop, how to efficiently and accurately verify the integrity of remotely stored data without frequently transferring complete data copies has become a research hotspot in the field of information security.
[0003] Traditional integrity verification methods, such as cyclic redundancy check (CRC) or one-way hash functions, typically require obtaining the entire file for calculation and comparison. However, in distributed systems, due to the large file size and dispersed storage locations, frequent reading of the entire file consumes significant network bandwidth and I / O resources, resulting in low verification efficiency and a lack of fault tolerance and location capabilities. To address these issues, the industry has introduced the Merkle Tree data structure. As a hash binary tree, the Merkle Tree can divide file data into blocks and calculate hash values layer by layer, ultimately generating a unique root hash. When verifying a specific data block, only the data block and the hash nodes on its adjacent paths need to be provided to quickly complete the verification without obtaining the entire file. However, some technical blind spots still exist in actual deployments. Existing solutions often use fixed block sizes when dividing data, lacking adaptability to the file's inherent attributes. For example, for files of different formats or sizes, a fixed block strategy may result in the severing of critical data boundaries, affecting the efficiency and security of subsequent encryption and verification. Summary of the Invention
[0004] To address this, the present invention provides a file hash addressing method and system based on MerkleTree, which overcomes the problems of low verification efficiency caused by reading the complete file in the prior art and the impact of file data block division on verification security.
[0005] To achieve the above objectives, in one aspect, the present invention provides a file hash addressing method based on MerkleTree, comprising: The file attribute information of the target file is obtained to determine the file attribute representation value, and the target file is divided into several key data blocks according to the file attribute representation value. The file attribute information includes file type and data size. Each key data block is encrypted, and the encryption representation value is determined based on the data size of each key data block after encryption, so as to determine whether the segmentation process meets the expected standard. If the expected criteria are met, hash calculations are performed on each of the key data blocks to determine the hash value corresponding to each key data block, so as to generate the leaf node hash set corresponding to the target file. The root hash value is generated based on the leaf node hash set to construct the target MerkleTree structure; Based on the target MerkleTree structure, the path information of each key data block is determined, and the path information of each key data block is encrypted to generate the addressing identifier corresponding to each key data block. The verification root hash value is determined based on the addressing identifier corresponding to the data block to be verified, and the verification root hash value is compared with the storage root hash value to determine whether the data block to be verified meets the data integrity standard; wherein, the data block to be verified is any critical data block.
[0006] Further, determining the file attribute representation value includes: The file type representation value is determined based on the file type of the target file and the preset type mapping table; The file data representation value is determined based on the comparison result between the data size of the target file and the preset data size; The file attribute representation value is determined based on the file type representation value and the file data representation value.
[0007] Furthermore, the target file is divided into several key data blocks based on file attribute representation values, including: The target number of segments is determined based on the file attribute representation values and the preset number of segments. The target file is segmented based on the target segmentation number to obtain several key data blocks.
[0008] Further, determining the encrypted representation value includes: The first comparison value is determined based on the comparison results of the data size before and after encryption of each key data block; The second comparison value is determined based on the comparison results of the encrypted data sizes of each of the key data blocks; The encrypted representation value is determined based on the first comparison value and the second comparison value.
[0009] Further, determine whether the segmentation process meets the expected criteria, including: The comparison between the encrypted representation value and the preset representation value determines whether the segmentation process meets the expected standard.
[0010] Furthermore, the target MerkleTree structure is constructed, including: The leaf node hashes in the leaf node hash set are grouped in pairs to obtain several leaf node hash groups; The hash values of each leaf node hash group are concatenated and then hashed to obtain several parent node hash values. The above steps are recursively executed to generate the root hash value for storage, in order to construct the target MerkleTree structure.
[0011] Further, generating the addressing identifier corresponding to any of the aforementioned key data blocks includes: The leaf nodes of the key data blocks are determined based on the metadata of the key data blocks, wherein the metadata includes the data block number of the key data blocks; Based on the target MerkleTree structure, determine the branch direction from the leaf node to the root node of the key data block to determine the path information of the key data block; The path information of the key data block is encrypted to generate the addressing identifier corresponding to the key data block.
[0012] Further, determining the verification root hash value includes: The hash value of the sibling node and the verification path of the data block to be verified are determined based on the addressing identifier corresponding to the data block to be verified. The root hash value is obtained by calculating the hash value layer by layer based on the hash value of the sibling node and the verification path.
[0013] Further, determine whether the target data block conforms to data integrity standards, including: If the verification root hash value is consistent with the storage root hash value, then the data block to be verified is determined to meet the data integrity standard.
[0014] On the other hand, the present invention also provides a file hash addressing system, comprising: The data acquisition module is used to acquire file attribute information of the target file, wherein the file attribute information includes file type and data size; The file segmentation module is used to determine the file attribute representation value based on the file attribute information of the target file, so as to segment the target file into several key data blocks; The segmentation verification module is used to encrypt each key data block and determine the encryption representation value based on the data size of each key data block after encryption, so as to determine whether the segmentation process meets the expected standard. The tree construction module is used to perform hash calculations on each of the key data blocks under the expected criteria to determine the hash value corresponding to each key data block, so as to generate the leaf node hash set corresponding to the target file, and generate the storage root hash value based on the leaf node hash set to construct the target MerkleTree structure. The addressing identifier generation module is used to determine the path information of each key data block based on the target MerkleTree structure, and to encrypt the path information of each key data block to generate the addressing identifier corresponding to each key data block. The integrity verification module is used to determine the verification root hash value based on the addressing identifier corresponding to the data block to be verified, and to determine whether the data block to be verified conforms to the data integrity standard based on the comparison result between the verification root hash value and the storage root hash value; wherein, the data block to be verified is any critical data block.
[0015] Compared with existing technologies, the advantages of this invention are as follows: By quantifying file attribute representation values based on the target file's attribute information and using this as the basis for block segmentation, the block segmentation strategy is dynamically adjusted, improving block adaptability and optimizing storage and transmission. Encrypting key data blocks after segmentation ensures data security. The encrypted representation value is obtained by quantifying the size of the encrypted data, which verifies whether the segmentation process meets the expected standards, enabling rapid and accurate determination of block compliance and ensuring the reliability of the file segmentation process. Hash calculation maps each key data block to a fixed-length digest, forming a unique fingerprint for the data block. This ensures the validity of the leaf node hash set, avoids polluting the MerkleTree structure with the hash values of abnormal data blocks, and guarantees the accuracy and security of subsequent addressing and verification. Hash calculation is performed on fixed-size data blocks, eliminating the need to process cross-block data, thus improving verification efficiency. By constructing the target MerkleTree structure, file integrity verification is transformed from full verification to hierarchical verification, providing structured support for subsequent path information extraction and addressing identifier generation. This ensures that the location of data blocks in the target MerkleTree structure can be quickly located during addressing, improving integrity verification efficiency. Addressing identifiers integrate MerkleTree path information and, combined with encryption mechanisms, ensure that the addressing identifier for each data block is globally unique. This prevents malicious attackers from reverse engineering the MerkleTree structure and tampering with data blocks using path information, thereby improving data security and addressing efficiency, and ensuring the security and reliability of the verification process. The verification root hash can be calculated solely from the addressing identifier of the data block to be verified. Comparing this hash with the storage root hash allows for rapid determination of data block integrity, improving the efficiency and security of integrity verification for any data block in a file.
[0016] Furthermore, this invention improves the efficiency of file segmentation by comparing the segmentation characteristics of various types of files with a preset type mapping table. By quantifying the data size of the target file, it achieves precise quantification of the data size dimension, enabling the segmentation strategy to allocate file size and file type. This comprehensive approach yields file attribute representation values, which more accurately reflect file characteristics and provide a scientific basis for subsequent dynamic segmentation, thereby improving the rationality and efficiency of segmentation.
[0017] Furthermore, the present invention dynamically adjusts the number of segments based on a preset number of segments to achieve adaptive segmentation. This ensures that the number of segments is within a controllable range and optimizes the granularity of segments according to file attributes, which facilitates subsequent encryption, indexing and verification, ensures clear boundaries of key data blocks, and reduces the management complexity caused by cross-block dependencies.
[0018] Furthermore, by comparing the data size of each key data block before and after encryption, this invention quantifies and analyzes the impact of encryption on the data block size, which can quickly identify abnormal encryption and prevent data block damage due to encryption defects. By comparing the data size of each key data block after encryption, the invention quantifies and analyzes the dispersion of each key data block, and determines the encryption characterization value by combining the first comparison value and the second comparison value, which can comprehensively evaluate the quality of the segmentation and encryption process.
[0019] Furthermore, this invention employs a binary tree structure, grouping leaf node hashes pairwise. This clear grouping facilitates recursive calculations and ensures a balanced tree height, reducing the verification path length. Hash operations bind two child nodes together, ensuring that any change to a child node is propagated to its parent node, forming a reliable chain of integrity. The root hash value is recursively obtained and stored as a global digest of the entire file, enabling verification of the entire file's integrity.
[0020] Furthermore, this invention utilizes metadata such as data block sequence numbers to quickly locate leaf nodes, ensuring that each data block has a unique corresponding position in the Merkle Tree, which facilitates subsequent path extraction. The path information records the sibling nodes required to move upward from the leaf node and their left and right directions, serving as the core credential for local verification. This ensures that verification can be completed with only a small number of nodes, and encryption prevents the path information from being tampered with or forged, thus guaranteeing the authenticity and integrity of the verification evidence.
[0021] Furthermore, this invention parses the necessary hash nodes and path directions from the addressing identifier, without accessing the complete Merkle Tree, thus achieving lightweight verification. It performs layer-by-layer hash calculation according to the path direction to finally obtain the verification root hash, which can be compared with the storage root hash to determine whether the data block is complete. This can improve the efficiency and security of integrity verification for any data block in a file. Attached Figure Description
[0022] Figure 1This is a flowchart illustrating the file hash addressing method based on MerkleTree according to an embodiment of the present invention. Figure 2 This is a schematic diagram illustrating the structure for determining file attribute representation values in an embodiment of the present invention. Figure 3 This is a logical judgment diagram for determining whether a target data block conforms to data integrity standards in an embodiment of the present invention; Figure 4 This is a structural block diagram of the file hash addressing system according to an embodiment of the present invention. Detailed Implementation
[0023] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.
[0024] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.
[0025] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.
[0026] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.
[0027] Please see Figures 1-3 As shown, Figure 1 This is a flowchart illustrating the file hash addressing method based on MerkleTree according to an embodiment of the present invention. Figure 2 This is a schematic diagram illustrating the structure for determining file attribute representation values in an embodiment of the present invention. Figure 3 This invention provides a logical judgment diagram for determining whether a target data block conforms to data integrity standards. The invention also provides a MerkleTree-based file hash addressing method, including: Step S1: Obtain the file attribute information of the target file to determine the file attribute representation value, and divide the target file into several key data blocks according to the file attribute representation value. The file attribute information includes file type and data size. Specifically, in step S1, determining the file attribute representation value includes: Step S11: Determine the file type representation value based on the file type of the target file and the preset type mapping table; Step S12: Determine the file data representation value based on the comparison result between the data size of the target file and the preset data size; Step S13: Determine the file attribute representation value based on the file type representation value and the file data representation value.
[0028] In implementation, file types can be obtained through file extensions, magic numbers, or MIME types. Implementers can pre-build a preset type mapping table, which maps common file types (such as text files, image files, audio files, video files, compressed files, etc.) to unique numerical codes (between 0 and 1). Higher values correspond to coarser block granularity (larger single data block size). For example, a text file's file type representation value is 0.1, an image file's is 0.3, an audio file's is 0.5, a video file's is 0.7, and a compressed file's is 0.9. The data size is the number of bytes in the file. The ratio of the target file's data size to the preset data size is determined as the file data representation value. Implementers can set the preset data size for this file type based on actual conditions or historical data showing the maximum data size of files of the same file type that passed the compliance test.
[0029] It is understandable that the file attribute representation value is determined by weighted fusion of the file type representation value and the file data representation value. For example, the file attribute representation value WS = a1 × W1 + a2 × W2, where W1 is the file type representation value, W2 is the file data representation value, a1 is the weight coefficient corresponding to the file type representation value, a2 is the weight coefficient corresponding to the file data representation value, and a1 + a2 = 1. Preferably, a1 = 0.6 and a2 = 0.4.
[0030] Specifically, this invention improves the efficiency of file segmentation by comparing the segmentation characteristics of various types of files with a preset type mapping table. By quantifying the data size of the target file, it achieves precise quantification of the data size dimension, enabling the segmentation strategy to allocate file size and file type. This comprehensive approach yields file attribute representation values, which more accurately reflect file characteristics and provide a scientific basis for subsequent dynamic segmentation, thereby improving the rationality and efficiency of segmentation.
[0031] Specifically, in step S1, the target file is divided into several key data blocks based on file attribute representation values, including: Step S14: Determine the target number of segments based on the file attribute representation value and the preset number of segments; Step S15: The target file is segmented based on the target segmentation quantity to obtain several key data blocks.
[0032] In implementation, the target number of segments is determined by rounding up the product of the file attribute representation value and the preset number of segments. Implementers can set the preset number of segments based on actual conditions or the average number of segments for the same file type in historical data. The target file is then uniformly divided into key data blocks of the target number of segments. The theoretical size of each key data block is then calculated. ,in, The default rounding function is used, where S is the size of the target file and N is the number of target segments. However, considering that the file size may not be divisible, the first... The size of each block is The size of the last block is During the splitting process, the file content is read sequentially according to the byte offset, generating N consecutive key data blocks.
[0033] Specifically, the present invention dynamically adjusts the number of segments based on a preset number of segments to achieve adaptive segmentation. This ensures that the number of segments is within a controllable range and optimizes the segmentation granularity according to file attributes, which facilitates subsequent encryption, indexing and verification, ensures clear boundaries of key data blocks, and reduces the management complexity caused by cross-block dependencies.
[0034] Step S2: Encrypt each key data block and determine the encryption representation value based on the data size of each key data block after encryption, so as to determine whether the segmentation process meets the expected standard. In implementation, no specific restrictions are placed on the encryption algorithm. For example, the AES-256-GCM symmetric encryption algorithm can be used to generate an encryption key based on the target file attribute representation value and a 16-byte random salt value (the salt value is stored in the file metadata). Each key data block independently generates a 12-byte random initial vector, and pads it with integer multiples of 0 to 16 bytes before encryption.
[0035] Specifically, in step S2, determining the encrypted representation value includes: Step S21: Determine the first comparison value based on the comparison results of the data size before and after encryption of each key data block; Step S22: Determine the second comparison value based on the comparison results of the encrypted data sizes of each key data block; Step S23: Determine the encrypted representation value based on the first comparison value and the second comparison value.
[0036] In implementation, the data sizes of each key data block before encryption are Y1, Y2, ..., Y... j , ..., Y m The encrypted data sizes are E1, E2, ..., E j , ..., E m Then the first comparison value D1 = (∑ m j=1 Y j ×E j ) / (sqrt(∑ m j=1 (Y j ) 2 )×sqrt(∑ m j=1 (E j ) 2 The second comparison value D2 = (∑ m j=1 (E j -(∑ m j=1 E j / m)) 2 ) / m, where j=1,2,…,m; m is the number of critical data blocks, and sqrt() is the preset square root determination function.
[0037] It is understandable that the encrypted representation value JM = D1 × D2.
[0038] Specifically, this invention quantifies the impact of encryption on data block size by comparing the data size of each key data block before and after encryption, which can quickly identify abnormal encryption and prevent data block damage due to encryption defects. By comparing the data size of each key data block after encryption, the invention quantifies the dispersion of each key data block, and determines the encryption characterization value by combining the first comparison value and the second comparison value, which can comprehensively evaluate the quality of the segmentation and encryption process.
[0039] Specifically, in step S2, determining whether the segmentation process meets the expected criteria includes: The comparison between the encrypted representation value and the preset representation value determines whether the segmentation process meets the expected standard.
[0040] During implementation, if the encrypted representation value is less than the preset representation value, the segmentation process is determined to meet the expected standard; if the encrypted representation value is greater than or equal to the preset representation value, the segmentation process is determined to fail to meet the expected standard. Implementers can set the preset representation value based on actual conditions or the average of encrypted representation values that passed the compliance test in historical data.
[0041] Understandably, if the segmentation process is determined not to meet the expected criteria, the segmentation will be repeated.
[0042] Step S3: If the expected criteria are met, perform hash calculations on each of the key data blocks to determine the hash value corresponding to each key data block, so as to generate the leaf node hash set corresponding to the target file. In implementation, there are no restrictions on the method of hash calculation. For example, the SHA-256 hash algorithm can be used to perform hash calculation on each key data block to determine the hash value corresponding to each key data block. The hash values are then arranged into an ordered set according to the data block number of the key data block, which is the leaf node hash set. This set is the basis for constructing the Merkle Tree.
[0043] Step S4: Generate a storage root hash value based on the leaf node hash set to construct the target MerkleTree structure; Specifically, in step S4, constructing the target MerkleTree structure includes: Step S41: Group the leaf node hashes in the leaf node hash set into pairs to obtain several leaf node hash groups; Step S42: After concatenating the hash values of each leaf node hash group, perform hash calculation to obtain several parent node hash values; Step S43: Recursively execute the above steps to generate the storage root hash value in order to construct the target MerkleTree structure.
[0044] In implementation, for the leaf node hash set, adjacent leaf node hashes are grouped in pairs sequentially. If the number of nodes is odd, the last leaf node hash is copied and paired with itself to form a group. For any leaf node hash group, the two hash values are concatenated into a byte string in order, following the principle of left node first, right node last, to ensure the determinism of the tree. The concatenated byte string is then hashed to obtain the parent node hash value, which is the parent node corresponding to the group. The above operation is performed on all groups to obtain the hash values of all parent nodes. For example, for the leaf node hash group (ha, hb), the concatenated data is concat = haⅡhb. The hash is then calculated again to obtain the corresponding parent node hash value, h. parent =Hash(concat).
[0045] Understandably, steps S41 and S42 are executed recursively. The recursion condition is: the obtained parent node hash value is used as the new input, and sub-steps S41 and S42 are executed repeatedly. That is, the parent node hash values are concatenated into a new byte string in order, and the hash calculation is performed on the concatenated byte string to form a new parent node hash value. This process continues until a unique node is generated, at which point the recursion ends. This unique node is the root node of the MerkleTree, and its hash value is the storage root hash value. During the recursion process, the nodes generated at each level can be recorded to form a complete binary tree, which is the target MerkleTree structure. In practical applications, only the storage root hash value needs to be stored, while the path information can be dynamically calculated during verification (if the complete tree is known) or obtained from the addressing identifier.
[0046] Specifically, this invention employs a binary tree structure, grouping leaf node hashes pairwise. This clear grouping facilitates recursive calculations and ensures a balanced tree height, reducing the verification path length. Hash operations bind two child nodes together, ensuring that any change to a child node is propagated to its parent node, forming a reliable chain of integrity. The root hash value is recursively obtained and stored as a global digest of the entire file, enabling verification of the entire file's integrity.
[0047] Step S5: Determine the path information of each key data block based on the target MerkleTree structure, and encrypt the path information of each key data block to generate the addressing identifier corresponding to each key data block. Specifically, in step S5, generating the addressing identifier corresponding to any of the key data blocks includes: Step S51: Determine the leaf node of the key data block based on the metadata of the key data block, wherein the metadata includes the data block number of the key data block; Step S52: Determine the branch direction from the leaf node to the root node of the key data block based on the target MerkleTree structure, so as to determine the path information of the key data block; Step S53: Encrypt the path information of the key data block to generate the addressing identifier corresponding to the key data block.
[0048] In implementation, each key data block is accompanied by metadata during partitioning, including at least the data block sequence number. Additionally, metadata may include file identifiers, block sizes, etc., but the core is the block sequence number, used to locate the corresponding leaf node in the target MerkleTree structure. Path information includes the hashes of all sibling nodes required to reach the root node from the leaf node, along with their left and right directions. This allows for layer-by-layer hash calculation starting from the leaf node to obtain the root hash. The path direction can be represented using binary encoding, where "0" represents the left branch and "1" represents the right branch.
[0049] It is understandable that the path information of critical data blocks can be encrypted using the AES-256-GCM symmetric encryption algorithm or SM2 asymmetric encryption, which will not be elaborated here.
[0050] Specifically, this invention utilizes metadata such as data block sequence numbers to quickly locate leaf nodes, ensuring that each data block has a unique corresponding position in the Merkle Tree, which facilitates subsequent path extraction. The path information records the sibling nodes required to move upward from the leaf node and their left and right directions, which are the core credentials for local verification. This ensures that only a small number of nodes are needed to complete the verification. Encryption prevents the path information from being tampered with or forged, thus guaranteeing the authenticity and integrity of the verification evidence.
[0051] Step S6: Determine the verification root hash value based on the addressing identifier corresponding to the data block to be verified, and determine whether the data block to be verified meets the data integrity standard based on the comparison result between the verification root hash value and the storage root hash value; wherein, the data block to be verified is any key data block.
[0052] Specifically, in step S6, determining the verification root hash value includes: Step S61: Determine the hash value of the sibling node and the verification path of the data block to be verified based on the addressing identifier corresponding to the data block to be verified. Step S62: Calculate the hash value layer by layer based on the hash value of the sibling node and the verification path to obtain the verification root hash value.
[0053] In implementation, any key data block can be verified, and the key data block to be verified is taken as the data block to be verified. The addressing identifier of the data block to be verified is decrypted to obtain the corresponding path information, including the hash value of the sibling node of the data block to be verified and the verification path. The hash value is calculated layer by layer along the verification path until the root hash value of the verification is generated.
[0054] Specifically, this invention parses the necessary hash nodes and path directions from the addressing identifier, without accessing the complete Merkle Tree, thus achieving lightweight verification. It performs layer-by-layer hash calculation according to the path direction to finally obtain the verification root hash, which is compared with the storage root hash to determine whether the data block is complete. This can improve the efficiency and security of integrity verification for any data block in a file.
[0055] Specifically, in step S6, determining whether the target data block meets the data integrity standard includes: If the verification root hash value is consistent with the storage root hash value, the data block to be verified is determined to meet the data integrity standard; if the verification root hash value is inconsistent with the storage root hash value, the data block to be verified is determined to not meet the data integrity standard.
[0056] This invention uses file attribute representation values obtained by quantifying the file attribute information of the target file as the basis for block segmentation, and dynamically adjusts the segmentation strategy to improve block adaptability and optimize storage and transmission. By encrypting key data blocks after segmentation, data security is ensured. The size of the encrypted data is quantified to obtain an encrypted representation value, which is used to verify whether the segmentation process meets the expected standards. This enables rapid and accurate determination of block compliance, ensuring the reliability of the file segmentation process. Hash calculation maps each key data block to a fixed-length digest, forming a unique fingerprint for the data block. This ensures the validity of the leaf node hash set and avoids the hash values of abnormal data blocks polluting the MerkleTree structure, guaranteeing the accuracy and security of subsequent addressing and verification. Hash calculation is performed on fixed-size data blocks, eliminating the need to handle cross-block data and improving verification efficiency. By constructing a target MerkleTree structure, file integrity verification is transformed from full verification to hierarchical verification, providing structured support for subsequent path information extraction and addressing identifier generation. This ensures that the position of data blocks in the target MerkleTree structure can be quickly located during addressing, improving integrity verification efficiency. Addressing identifiers integrate MerkleTree path information and, combined with encryption mechanisms, ensure that the addressing identifier for each data block is globally unique. This prevents malicious attackers from reverse engineering the MerkleTree structure and tampering with data blocks using path information, thereby improving data security and addressing efficiency, and ensuring the security and reliability of the verification process. The verification root hash can be calculated solely from the addressing identifier of the data block to be verified. Comparing this hash with the storage root hash allows for rapid determination of data block integrity, improving the efficiency and security of integrity verification for any data block in a file. Example
[0057] File segmentation and tree construction: Assuming the target file F is segmented into 4 key data blocks: B1, B2, B3, B4, then calculate the leaf node hashes: H1=Hash(B1), H2=Hash(B2), H3=Hash(B3), H4=Hash(B4); construct the parent node hashes: H12=Hash(H1||H2), H34=Hash(H3||H4); generate the root hash value: H_root=Hash(H12||H34); Addressing identifier generation: Taking data block B3 as an example, the path is: B3→H34→H_root; Path encoding: From the leaf node to the root node, the left branch is recorded as "0" and the right branch is recorded as "1". The path of B3 is "right→left", and the encoding is "10"; then the addressing identifier is: ID_3="Block3_10"; Verify the integrity of B3 data: Obtain the verification path based on ID_3: [H4, H12] (sibling node hash); Calculation process: H34_cal = Hash(H3||H4) H_root_cal=Hash(H12||H34_cal) Compare and verify the root hash value H_root_cal with the stored root hash value H_root. If they match, then B3 meets the data integrity standard.
[0058] Please see Figure 4 The diagram shown is a structural block diagram of a file hash addressing system according to an embodiment of the present invention; the present invention also provides a file hash addressing system, including: The data acquisition module is used to acquire file attribute information of the target file, wherein the file attribute information includes file type and data size; The file segmentation module, which is connected to the data acquisition module, is used to determine the file attribute representation value based on the file attribute information of the target file, so as to segment the target file into several key data blocks. The segmentation verification module, which is connected to the file segmentation module, is used to encrypt each key data block and determine the encryption representation value based on the data size of each key data block after encryption, so as to determine whether the segmentation process meets the expected standard. The tree construction module is connected to the segmentation verification module and the file segmentation module respectively. It is used to perform hash calculation on each key data block to determine the hash value corresponding to each key data block under the condition of meeting the expected standard, so as to generate the leaf node hash set corresponding to the target file, and generate the storage root hash value based on the leaf node hash set to construct the target MerkleTree structure. An addressing identifier generation module, which is connected to the tree construction module, is used to determine the path information of each key data block based on the target MerkleTree structure, and to encrypt the path information of each key data block to generate an addressing identifier corresponding to each key data block. The integrity verification module is connected to the addressing identifier generation module and the tree construction module, respectively. It is used to determine the verification root hash value based on the addressing identifier corresponding to the data block to be verified, and to determine whether the data block to be verified conforms to the data integrity standard based on the comparison result between the verification root hash value and the storage root hash value; wherein, the data block to be verified is any critical data block.
[0059] Specifically, the file hash addressing system provided in this embodiment of the invention can achieve the same technical effect by using the above-described file hash addressing method based on MerkleTree, which will not be elaborated further here.
[0060] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.
Claims
1. A file hash addressing method based on MerkleTree, characterized in that, include: The file attribute information of the target file is obtained to determine the file attribute representation value, and the target file is divided into several key data blocks according to the file attribute representation value. The file attribute information includes file type and data size. Each key data block is encrypted, and the encryption representation value is determined based on the data size of each key data block after encryption, so as to determine whether the segmentation process meets the expected standard. If the expected criteria are met, hash calculations are performed on each of the key data blocks to determine the hash value corresponding to each key data block, so as to generate the leaf node hash set corresponding to the target file. The root hash value is generated based on the leaf node hash set to construct the target MerkleTree structure; Based on the target MerkleTree structure, the path information of each key data block is determined, and the path information of each key data block is encrypted to generate the addressing identifier corresponding to each key data block. The verification root hash value is determined based on the addressing identifier corresponding to the data block to be verified, and the verification root hash value is compared with the storage root hash value to determine whether the data block to be verified meets the data integrity standard; wherein, the data block to be verified is any critical data block.
2. The file hash addressing method based on MerkleTree according to claim 1, characterized in that, Determining the file attribute representation value includes: The file type representation value is determined based on the file type of the target file and the preset type mapping table; The file data representation value is determined based on the comparison result between the data size of the target file and the preset data size; The file attribute representation value is determined based on the file type representation value and the file data representation value.
3. The file hash addressing method based on MerkleTree according to claim 2, characterized in that, The target file is divided into several key data blocks based on file attribute representation values, including: The target number of segments is determined based on the file attribute representation values and the preset number of segments. The target file is segmented based on the target segmentation number to obtain several key data blocks.
4. The file hash addressing method based on MerkleTree according to claim 3, characterized in that, Determining the encrypted representation value includes: The first comparison value is determined based on the comparison results of the data size before and after encryption of each key data block; The second comparison value is determined based on the comparison results of the encrypted data sizes of each of the key data blocks; The encrypted representation value is determined based on the first comparison value and the second comparison value.
5. The file hash addressing method based on MerkleTree according to claim 4, characterized in that, Determining whether the segmentation process meets the expected criteria includes: The comparison between the encrypted representation value and the preset representation value determines whether the segmentation process meets the expected standard.
6. The file hash addressing method based on MerkleTree according to claim 5, characterized in that, Construct the target MerkleTree structure, including: The leaf node hashes in the leaf node hash set are grouped in pairs to obtain several leaf node hash groups; The hash values of each leaf node hash group are concatenated and then hashed to obtain several parent node hash values. The above steps are recursively executed to generate the root hash value for storage, in order to construct the target MerkleTree structure.
7. The file hash addressing method based on MerkleTree according to claim 6, characterized in that, Generating the addressing identifier corresponding to any of the aforementioned key data blocks includes: The leaf nodes of the key data blocks are determined based on the metadata of the key data blocks, wherein the metadata includes the data block number of the key data blocks; Based on the target MerkleTree structure, determine the branch direction from the leaf node to the root node of the key data block to determine the path information of the key data block; The path information of the key data block is encrypted to generate the addressing identifier corresponding to the key data block.
8. The file hash addressing method based on MerkleTree according to claim 7, characterized in that, Determining the verification root hash value includes: The hash value of the sibling node and the verification path of the data block to be verified are determined based on the addressing identifier corresponding to the data block to be verified. The root hash value is obtained by calculating the hash value layer by layer based on the hash value of the sibling node and the verification path.
9. The file hash addressing method based on MerkleTree according to claim 8, characterized in that, Determine whether the target data block meets data integrity standards, including: If the verification root hash value is consistent with the storage root hash value, then the data block to be verified is determined to meet the data integrity standard.
10. A file hash addressing system, which employs the MerkleTree-based file hash addressing method according to any one of claims 1-9, characterized in that, include: The data acquisition module is used to acquire file attribute information of the target file, wherein the file attribute information includes file type and data size; The file segmentation module is used to determine the file attribute representation value based on the file attribute information of the target file, so as to segment the target file into several key data blocks; The segmentation verification module is used to encrypt each key data block and determine the encryption representation value based on the data size of each key data block after encryption, so as to determine whether the segmentation process meets the expected standard. The tree construction module is used to perform hash calculations on each of the key data blocks under the expected criteria to determine the hash value corresponding to each key data block, so as to generate the leaf node hash set corresponding to the target file, and generate the storage root hash value based on the leaf node hash set to construct the target MerkleTree structure. The addressing identifier generation module is used to determine the path information of each key data block based on the target MerkleTree structure, and to encrypt the path information of each key data block to generate the addressing identifier corresponding to each key data block. The integrity verification module is used to determine the verification root hash value based on the addressing identifier corresponding to the data block to be verified, and to determine whether the data block to be verified conforms to the data integrity standard based on the comparison result between the verification root hash value and the storage root hash value; wherein, the data block to be verified is any critical data block.