MERKLE tree authentication method and device based on frequency and time dual feature perception

Abstract

The invention provides an MERKLE tree authentication method and device based on frequency and time dual feature perception, and relates to the technical field of network security. The method comprises the following steps of: upwards generating a hash node for each data node through an SM3 algorithm; arranging all the Hash nodes according to an increasing sequence of the failure timestamps; traversing the ordered queue from left to right, finding a plurality of adjacent nodes, combining to generate a father node, replacing the adjacent nodes, and constructing a Merkle tree structure; and performing recursive binary search from the root node from top to bottom according to the timestamp to position the target node and record the Merkle path, performing reverse verification on the hash value of the Merkle path from bottom to top, performing hash aggregation on brother nodes of the Merkle path layer by layer, and verifying whether the hash values of the root node are consistent or not. According to the method, the data verification efficiency of the Merkle tree is optimized by combining dual information of the timestamp and the frequency value, and the method is suitable for scenes with timeliness and access frequency characteristics.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to a frequency-time dual feature perception MERKLE tree authentication method and device. Background Art

[0002] With the rapid development of information technology, the scale and complexity of data in various fields have exploded. From financial transaction records and healthcare data to the massive user information on internet platforms, the integrity and consistency of this data are directly related to the stable operation of the business, the accuracy of decision-making, and the protection of user rights. Ensuring that data is not tampered with, lost, or mishandled during data storage and transmission has become a core challenge in the field of information security.

[0003] As an efficient data structure, Merkle trees are widely used in scenarios such as blockchains, distributed storage systems, and data backup to verify data integrity. In blockchains, Merkle trees compress large amounts of transaction data into a single, easily verifiable hash value, enabling nodes to quickly verify the authenticity and integrity of transactions without traversing the entire transaction ledger. In distributed storage systems, Merkle trees can efficiently detect whether data blocks have been altered during transmission or storage, ensuring data reliability. However, traditional Merkle trees exhibit significant shortcomings when dealing with the complex and volatile data characteristics of the real world.

[0004] On the one hand, much data is time-sensitive, such as real-time market data in financial markets and cargo location information in logistics. Traditional Merkle trees fail to account for the temporal nature of data, making it impossible to quickly distinguish invalid data when processing this time-sensitive data, resulting in inefficient verification. On the other hand, the frequency of data access varies significantly across different scenarios. For example, popular news articles receive far more visits than general information, and frequently accessed data requires more efficient verification mechanisms to reduce response time. Traditional Merkle trees ignore data access frequency, resulting in a large number of unnecessary hash calculations in high-frequency data access scenarios, which reduces overall system performance.

[0005] In the field of cybersecurity, the large-scale deployment of IoT devices has led to increasingly severe data security challenges. Billions of IoT devices, from smart home appliances to industrial sensors, are interconnected and generate massive amounts of data. The data collected by these devices is the cornerstone for building intelligent applications and decision-making. If a device node is maliciously attacked, or data is tampered with or forged, it poses a serious threat to the reliability and security of the entire IoT system. Ensuring mutual trust between devices and rapid data verification is crucial in the interconnected process. However, the traditional Merkle algorithm has room for improvement in its adaptability to these complex data characteristics. Summary of the Invention

[0006] To address the existing technical issues that the traditional Merkle tree does not take the time attribute of data into consideration, making it impossible to quickly distinguish invalid data when processing such time-sensitive data, resulting in low verification efficiency, and that the traditional Merkle tree ignores the frequency of data access, resulting in a large number of unnecessary hash calculations that reduce the overall performance of the system in high-frequency data access scenarios, the embodiments of the present invention provide a MERKLE tree authentication method and device that perceives the dual characteristics of frequency and time. The technical solution is as follows:

[0007] In one aspect, a frequency-time dual feature perception MERKLE tree authentication method is provided. The method is implemented by a MERKLE tree authentication device and includes:

[0008] S1. Obtain the data node set to be used for Merkle tree construction, and generate a corresponding hash node for each data node in the data node set using the SM3 algorithm. The pointer of the generated hash node points to the corresponding data node, and the generated hash node inherits the timestamp and frequency value of the corresponding data node.

[0009] S2. Arrange all hash nodes in ascending order of their expiration timestamps to obtain an ordered queue.

[0010] S3. Traverse the ordered queue from left to right, find multiple adjacent nodes, merge them, and generate parent nodes of multiple adjacent nodes.

[0011] S4. Use the generated parent node to replace the corresponding multiple adjacent nodes in the ordered queue, and recursively repeat the merging and replacing process until only one hash node remains in the ordered queue as the root node of the Merkle tree, thereby obtaining the constructed Merkle tree structure.

[0012] S5. Starting from the root node of the Merkle tree, perform a recursive binary search from top to bottom based on the timestamps of the hash nodes to locate the target node, and record the path from the target node to the root node as the Merkle path. Verify the hash values ​​of the hash nodes of the Merkle path in reverse order from bottom to top, hash the sibling nodes of the Merkle path layer by layer, verify whether the hash value of the root node is consistent, and then verify the integrity and correctness of the target node.

[0013] Optionally, the method for adding a new data node to the Merkle tree includes:

[0014] The Merkle tree is used to locate and delete invalid nodes through binary search to obtain a Merkle tree that retains valid data.

[0015] According to the increasing order of the expiration timestamps, the insertion position of the new data node is determined by binary search, and the Merkle tree is rebuilt according to the frequency value.

[0016] Optionally, performing a recursive binary search to locate the target node in S5 includes:

[0017] In the process of performing recursive binary search to locate the target node, the nodes with invalid hash values ​​and the subtrees corresponding to the nodes with invalid hash values ​​are pruned to avoid repeated verification of invalid nodes in subsequent queries.

[0018] Optionally, multiple adjacent nodes are found in S3 and merged to generate a parent node of the multiple adjacent nodes, including:

[0019] Find the two adjacent nodes with the smallest sum of frequency values ​​and merge them to generate the parent node of multiple adjacent nodes.

[0020] Alternatively, the weight of each hash node is calculated based on the dynamic weight calculation formula, and the two adjacent nodes with the smallest sum of weights are found to merge. If there are multiple adjacent node pairs whose sum of weights is less than the preset similarity threshold, the adjacent node pair with the smallest sum of timestamps is selected to merge and generate the parent node of multiple adjacent nodes.

[0021] Among them, the timestamp of the parent node is the average of the timestamps of the corresponding two adjacent nodes; the frequency value of the parent node is the sum of the frequency values ​​of the corresponding two adjacent nodes; the hash value of the parent node is the sum of the hash values ​​of the corresponding two adjacent nodes calculated by the SM3 algorithm.

[0022] The dynamic weight calculation formula is as follows (1):

[0023] (1).

[0024] Optionally, the step of finding multiple adjacent nodes in S3 and merging them to generate a parent node of the multiple adjacent nodes further includes:

[0025] A frequency threshold is set according to the size of the data node set, and multiple consecutive adjacent nodes whose sum of frequency values ​​is not greater than the frequency threshold are obtained and merged to generate a parent node of the multiple adjacent nodes.

[0026] Among them, the timestamp of the parent node is the median of the timestamps of the corresponding multiple adjacent nodes; the frequency value of the parent node is the sum of the frequency values ​​of the corresponding multiple adjacent nodes; the hash value of the parent node is the SM3 hash result of the hash values ​​of multiple adjacent nodes.

[0027] Optionally, the method further includes: automatic cleaning and reconstruction of failed nodes.

[0028] Automatic cleanup and reconstruction of failed nodes, including:

[0029] Periodically scan the ordered queue, use the skip list structure to find the hash nodes whose invalid timestamps are earlier than the current time, and mark them as invalid nodes.

[0030] After deleting the invalid node, if the parent node of the invalid node has only one valid child node left, recursively search for the sibling node and perform a merge.

[0031] The hash nodes except the invalid nodes are rearranged in the ascending order of the invalid timestamps, and the merge operation is re-executed according to the frequency value, and the Merkle tree structure is updated to maintain the validity of the authentication.

[0032] Optionally, the method further comprises:

[0033] The weight of each Merkle path is calculated based on the cache priority weight calculation formula. The LRU-K algorithm is used to record the timestamps of the two most recent accesses to each Merkle path, and the access interval is calculated based on the timestamps of the two most recent accesses. When the cache space reaches the upper limit, Merkle paths with low weight and long access intervals are eliminated first.

[0034] The cache priority weight calculation formula is as follows (2):

[0035] (2)

[0036] Where, and It is an adjustable parameter.

[0037] On the other hand, a MERKLE tree authentication device based on frequency and time dual feature perception is provided. The device is applied to a MERKLE tree authentication method based on frequency and time dual feature perception. The device includes:

[0038] The acquisition module is used to obtain the data node set to be used for Merkle tree construction, and generate a corresponding hash node for each data node in the data node set through the SM3 algorithm. The pointer of the generated hash node points to the corresponding data node, and the generated hash node inherits the timestamp and frequency value of the corresponding data node.

[0039] The arrangement module is used to arrange all hash nodes in the increasing order of the expiration timestamps of the hash nodes to obtain an ordered queue.

[0040] The merging module is used to traverse the ordered queue from left to right, find multiple adjacent nodes to merge, and generate parent nodes of multiple adjacent nodes.

[0041] The construction module is used to replace the corresponding multiple adjacent nodes in the ordered queue with the generated parent node, and recursively repeat the merging and replacing process until only one hash node remains in the ordered queue as the root node of the Merkle tree, thereby obtaining the constructed Merkle tree structure.

[0042] The authentication module is used to start from the root node of the Merkle tree, perform recursive binary search from top to bottom based on the timestamps of the hash nodes to locate the target node, and record the path from the target node to the root node as the Merkle path. The hash values ​​of the hash nodes of the Merkle path are verified in reverse order from bottom to top, and the sibling nodes of the Merkle path are hashed layer by layer to verify whether the hash value of the root node is consistent, thereby verifying the integrity and correctness of the target node.

[0043] Optionally, the method for adding a new data node to the Merkle tree includes:

[0044] The Merkle tree is used to locate and delete invalid nodes through binary search to obtain a Merkle tree that retains valid data.

[0045] According to the increasing order of the expiration timestamps, the insertion position of the new data node is determined by binary search, and the Merkle tree is rebuilt according to the frequency value.

[0046] Optionally, the authentication module is further configured to:

[0047] In the process of performing recursive binary search to locate the target node, the nodes with invalid hash values ​​and the subtrees corresponding to the nodes with invalid hash values ​​are pruned to avoid repeated verification of invalid nodes in subsequent queries.

[0048] Optionally, the merge module is further configured to:

[0049] Find the two adjacent nodes with the smallest sum of frequency values ​​and merge them to generate the parent node of multiple adjacent nodes.

[0050] Alternatively, the weight of each hash node is calculated based on the dynamic weight calculation formula, and the two adjacent nodes with the smallest sum of weights are found to merge. If there are multiple adjacent node pairs whose sum of weights is less than the preset similarity threshold, the adjacent node pair with the smallest sum of timestamps is selected to merge and generate the parent node of multiple adjacent nodes.

[0051] Among them, the timestamp of the parent node is the average of the timestamps of the corresponding two adjacent nodes; the frequency value of the parent node is the sum of the frequency values ​​of the corresponding two adjacent nodes; the hash value of the parent node is the sum of the hash values ​​of the corresponding two adjacent nodes calculated by the SM3 algorithm.

[0052] The dynamic weight calculation formula is as follows (1):

[0053] (1).

[0054] Optionally, the merge module is further configured to:

[0055] A frequency threshold is set according to the size of the data node set, and multiple consecutive adjacent nodes whose sum of frequency values ​​is not greater than the frequency threshold are obtained and merged to generate a parent node of the multiple adjacent nodes.

[0056] Among them, the timestamp of the parent node is the median of the timestamps of the corresponding multiple adjacent nodes; the frequency value of the parent node is the sum of the frequency values ​​of the corresponding multiple adjacent nodes; the hash value of the parent node is the SM3 hash result of the hash values ​​of multiple adjacent nodes.

[0057] Optionally, the method further includes: automatic cleaning and reconstruction of failed nodes.

[0058] Automatic cleanup and reconstruction of failed nodes, including:

[0059] Periodically scan the ordered queue, use the skip list structure to find the hash nodes whose invalid timestamps are earlier than the current time, and mark them as invalid nodes.

[0060] After deleting the invalid node, if the parent node of the invalid node has only one valid child node left, recursively search for the sibling node and perform a merge.

[0061] The hash nodes except the invalid nodes are rearranged in the ascending order of the invalid timestamps, and the merge operation is re-executed according to the frequency value, and the Merkle tree structure is updated to maintain the validity of the authentication.

[0062] Optionally, the method further comprises:

[0063] The weight of each Merkle path is calculated based on the cache priority weight calculation formula. The LRU-K algorithm is used to record the timestamps of the two most recent accesses to each Merkle path, and the access interval is calculated based on the timestamps of the two most recent accesses. When the cache space reaches the upper limit, Merkle paths with low weight and long access intervals are eliminated first.

[0064] The cache priority weight calculation formula is as follows (2):

[0065] (2)

[0066] Where, and It is an adjustable parameter.

[0067] On the other hand, a MERKLE tree authentication device is provided, comprising: a processor; a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the above-mentioned frequency-time dual feature perception MERKLE tree authentication methods is implemented.

[0068] On the other hand, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned frequency-time dual feature perception MERKLE tree authentication methods.

[0069] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0070] This paper addresses the challenges faced in data integrity verification and network security by proposing a Merkle tree authentication method based on dual frequency and time signatures. By innovatively integrating timestamp and frequency signatures into the Merkle tree construction and authentication process, it aims to improve data verification efficiency, reduce storage resource consumption, and enhance data security, meeting the stringent data processing and security requirements of the digital age. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0072] Figure 1 This is a flow chart of a frequency-time dual feature perception MERKLE tree authentication method provided by an embodiment of the present invention;

[0073] Figure 2 This is a flow chart of a frequency-time dual feature perception MERKLE tree authentication method provided by an embodiment of the present invention;

[0074] Figure 3 This is a 4-node Merkle tree structure diagram provided by an embodiment of the present invention;

[0075] Figure 4 This is a block diagram of a frequency-time dual feature perception MERKLE tree authentication device provided by an embodiment of the present invention;

[0076] Figure 5 It is a structural diagram of a MERKLE tree authentication device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0077] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0078] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "exemplary" in the present invention should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. Furthermore, in the embodiments of the present invention, "and / or" can mean both or either of the two.

[0079] In the embodiments of the present invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same. The terms "of," "corresponding," and "corresponding" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same.

[0080] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0081] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0082] The embodiment of the present invention provides a MERKLE tree authentication method based on frequency and time dual feature perception. The method can be implemented by a MERKLE tree authentication device, which can be a terminal or a server. Figure 1 、 Figure 2 The flowchart of the frequency-time dual feature perception MERKLE tree authentication method shown in FIG. 1 may include the following steps:

[0083] S1. Obtain the data node set to be used for Merkle tree construction, and generate a corresponding hash node for each data node in the data node set using the SM3 algorithm. The pointer of the generated hash node points to the corresponding data node, and the generated hash node inherits the timestamp and frequency value of the corresponding data node.

[0084] In one feasible implementation, for all data nodes in the Merkle tree, the data node set is , Indicates the number of data nodes, for each data node Generate hash nodes upward using the SM3 algorithm , The pointer of each hash node accurately points to its corresponding data node, and all hash nodes inherit the timestamp and frequency value of the data node. The timestamp is used to identify the timeliness of the data, and the frequency value is used to record the access frequency of the data node.

[0085] S2. Arrange all hash nodes in ascending order of their expiration timestamps to obtain an ordered queue.

[0086] In one feasible implementation, all hash nodes are arranged in ascending order of their expiration timestamps, forming an ordered queue. Expiration timestamps reflect the timeliness of data nodes, with nodes that expired earlier placed at the front of the queue and nodes that expired later placed at the back. This step ensures that nodes with less timeliness are prioritized during subsequent merges.

[0087] S3. Traverse the ordered queue from left to right, find multiple adjacent nodes, merge them, and generate parent nodes of multiple adjacent nodes.

[0088] Optionally, multiple adjacent nodes are found in S3 and merged to generate a parent node of the multiple adjacent nodes, including:

[0089] The hash node frequency set is , find the two adjacent nodes with the smallest sum of frequency values Merge and find these two nodes Then, generate their parent nodes. ,in, Represents a frequency node.

[0090] In one feasible implementation, the ordered queue is traversed from left to right to find the two adjacent nodes with the smallest frequency sum and merge them. After finding these two nodes, their parent nodes are generated. The generation rule of the parent node is as follows: The timestamp of the parent node is the average of the timestamps of the two child nodes, the frequency value of the parent node is the sum of the frequency values ​​of the two child nodes, and the hash value of the parent node is the sum of the hash values ​​of the two child nodes (calculated by the SM3 algorithm). ,in, Represents the hash value of the parent node, Represents the hash value of the left child node, Represents the hash value of the right child node.

[0091] Alternatively, the weight of each hash node is calculated based on the dynamic weight calculation formula, and the two adjacent nodes with the smallest sum of weights are found to merge. If there are multiple adjacent node pairs whose sum of weights is less than the preset similarity threshold, the adjacent node pair with the smallest sum of timestamps is selected for merging to generate the parent node of multiple adjacent nodes. The weight reflects the importance of the node and the remaining valid time.

[0092] In one possible implementation, the dynamic weight of each hash node is calculated by formula (1), which ensures that the weight of nodes approaching failure is reduced to give priority to processing: (1).

[0093] Furthermore, all hash nodes are arranged in ascending order of expiration timestamps to form an ordered queue. Traversing the queue from left to right, the pair of adjacent nodes with the smallest sum of weights is selected for merging. If multiple pairs of nodes have similar sums of weights, the nodes with the smallest timestamps are preferentially merged.

[0094] Furthermore, the timestamp of the parent node is the weighted average of the timestamps of the two child nodes, and the frequency value is the sum of the child node frequencies. The hash value of the parent node is the concatenation of the hash values ​​of the two child nodes, calculated using the SM3 algorithm.

[0095] Furthermore, the parent node generated by the merger replaces the atomic node, and the above process is repeated until all nodes are merged into a root node.

[0096] Optionally, the step of finding multiple adjacent nodes in S3 and merging them to generate a parent node of the multiple adjacent nodes further includes:

[0097] A frequency threshold is set according to the size of the data node set, and multiple consecutive adjacent nodes whose sum of frequency values ​​is not greater than the frequency threshold are obtained and merged to generate a parent node of the multiple adjacent nodes.

[0098] Among them, the timestamp of the parent node is the median of the timestamps of the corresponding multiple adjacent nodes; the frequency value of the parent node is the sum of the frequency values ​​of the corresponding multiple adjacent nodes; the hash value of the parent node is the SM3 hash result of the hash values ​​of multiple adjacent nodes.

[0099] In a feasible implementation, the present invention proposes a multi-fork Merkle tree support mechanism, specifically:

[0100] Set a frequency threshold based on the data size (such as setting a merge frequency and a rule of less than or equal to 100) for multi-node merge determination.

[0101] Further, in the ordered queue of hash nodes, find the consecutive The sum of the frequency values ​​of the nodes is less than or equal to the set threshold. If the condition is met, the node Nodes are merged to form a parent node.

[0102] Furthermore, the hash value of the parent node is The SM3 hash result after concatenating the hash values ​​of the child nodes is The timestamp is the median of the child node timestamps, and the frequency value is The sum of the frequencies of the child nodes.

[0103] Furthermore, the parent node is inserted into the original queue to replace the original child nodes, and repeat the process until all nodes are merged and the Merkle tree root node is generated, thus forming a multi-branch Merkle tree structure.

[0104] S4. Use the generated parent node to replace the corresponding multiple adjacent nodes in the ordered queue, and recursively repeat the merging and replacing process until only one hash node remains in the ordered queue as the root node of the Merkle tree, thereby obtaining the constructed Merkle tree structure.

[0105] In a feasible implementation, the above steps S1-S4 are the initialization of the Merkle tree: the data is organized into a Merkle tree structure according to time and frequency. The 4-node Merkle tree structure is shown in the figure below: Figure 3 shown.

[0106] Specifically, the generated parent node replaces the two child nodes in the original queue. The parent node's timestamp, frequency value, and hash value are updated according to the rules in step S3. This merging process is repeated recursively until only one node remains in the queue. This node becomes the root node of the Merkle tree.

[0107] S5. Starting from the root node of the Merkle tree, perform a recursive binary search from top to bottom based on the timestamps of the hash nodes to locate the target node, and record the path from the target node to the root node as the Merkle path. Verify the hash values ​​of the hash nodes of the Merkle path in reverse order from bottom to top, hash the sibling nodes of the Merkle path layer by layer, verify whether the hash value of the root node is consistent, and then verify the integrity and correctness of the target node.

[0108] In a feasible implementation, step S5 is data verification: starting from the Merkle root from top to bottom, a recursive algorithm is used to perform a binary search, and the query frequency of the node is increased, the Merkle path is recorded, and integrity verification is performed.

[0109] Specifically, starting from the root node of the Merkle tree, a recursive binary search is performed from top to bottom based on the timestamp feature value. During the search process, if a node with an invalid hash value is encountered (that is, the node's expiration timestamp is earlier than the current time), the node and its subtree are pruned to avoid repeated verification of invalid nodes in subsequent queries.

[0110] Furthermore, the search process records the query path, finds the specified data node, and records the path as the Merkle path, and increases the frequency of the data node and the parent node: (2)

[0111] Where, represents the Merkel path, Indicates the first The timestamp of each node, Indicates the The hash value of each node, Indicates the number of nodes on the path.

[0112] Furthermore, according to the Merkle path, the node hash value of the path is verified in reverse order from bottom to top, and the brother nodes of the path are Hash aggregation layer by layer:

[0113] (3)

[0114] Where, Represents the Merkle root node, Represents the root node hash value, Represents the node hash value on the Merkle path, Indicates the hash value of the corresponding node's sibling node. The two are hashed and aggregated, step by step upward until the root node.

[0115] Verify the consistency of the Merkle root hash value. This bottom-up hash verification method can accurately verify the integrity and correctness of the target node, ensuring that the data has not been tampered with during storage and transmission.

[0116] Optionally, the method for adding a new data node to the Merkle tree includes:

[0117] The Merkle tree is used to locate and delete invalid nodes through binary search to obtain a Merkle tree that retains valid data.

[0118] According to the increasing order of the expiration timestamps, the insertion position of the new data node is determined by binary search, and the Merkle tree is rebuilt according to the frequency value.

[0119] In a feasible implementation, the present invention also includes Merkle reconstruction: reconstructing the Merkle tree when adding a new node, deleting expired nodes, inserting nodes into the linked list in chronological order, and reconstructing the Merkle tree from bottom to top according to frequency.

[0120] Specifically, when a new node is needed, invalid nodes are first quickly removed through a binary search, ensuring that the queue contains only valid nodes, improving data validity. The new node is then inserted into the ordered queue in timestamp order. Finally, the Merkle tree structure is reconstructed based on frequency characteristics, forming a tilted Merkle sorted tree structure. This structure better adapts to the timeliness and access frequency characteristics of data nodes.

[0121] For the authentication process of big data integrity and consistency using Merkle trees, the present invention proposes a Merkle tree construction and verification method based on the dual feature perception of time and frequency, which aims to optimize the data verification efficiency of the Merkle tree by combining the dual information of timestamp and frequency value, and is particularly suitable for scenarios with timeliness and access frequency characteristics. When constructing a Merkle tree, all data nodes are first generated upwards according to the SM3 algorithm to generate their own hash nodes. The hash nodes inherit the timestamp and frequency value of the data nodes. All hash nodes are arranged in ascending order of the expiration timestamp to form an ordered queue. The expiration timestamp reflects the timeliness of the data nodes. The nodes that expire earlier are arranged in front, and the nodes that expire later are arranged in the back. Then, the queue is traversed from left to right to find the two adjacent nodes with the smallest frequency sum. The frequency value reflects the access frequency or importance of the data node. The smaller the frequency sum, the lower the access frequency of the two nodes, which is suitable for priority merging. After finding these two nodes, their parent node is generated and replaced with the two child nodes in the original queue. The parent node's timestamp is the average of the child node timestamps, its frequency is the sum of the child node frequencies, and its hash value is the sum of the child node hash values. This process ensures that the parent node's timestamp and frequency values ​​comprehensively reflect the timeliness and access frequency of the child nodes. This merging process is recursively repeated until only the last two nodes remain in the queue. These two nodes are then merged to form the Merkle root node, ultimately forming a tilted Merkle tree structure that better adapts to the timeliness and frequency characteristics of data nodes. When verifying data nodes, a binary search algorithm is used to recursively locate the target node and record the path from the target node to the Merkle root, known as the Merkle path. The integrity and correctness of the target node are verified by hashing the nodes in the path. When adding a new data node, we first use binary search to quickly locate and delete the currently invalid nodes to ensure that only valid data is retained in the tree. Then, we use binary search to determine the insertion position of the new node so that it is arranged in ascending order of the invalidation timestamp. Finally, we rebuild the Merkle tree based on the latest frequency value to ensure dynamic update and optimization of the tree structure.

[0122] The construction process of the present invention is simple. Compared with the current Merkle tree structure, it introduces dual feature perception of timestamp and frequency, which greatly improves the speed of verifying data nodes. The tilted Merkle tree structure uses fewer hash nodes, reducing the cost of space storage. It uses the national secret SM3 algorithm, which increases the security of data nodes compared to the SHA hash algorithm. It can be effectively applied to large-scale data verification scenarios in blockchain.

[0123] Optionally, to further improve the performance and timeliness of the Merkle tree structure in processing dynamic data, the method also includes: automatic cleaning and reconstruction of failed nodes.

[0124] Automatic cleanup and reconstruction of failed nodes, including:

[0125] Scan the hash node queue periodically (e.g., every minute), use a skip list structure to find nodes whose expiration timestamps are earlier than the current time, and mark them as invalid.

[0126] Furthermore, after deleting an invalid node, if its parent node has only one valid child node left, recursively search for sibling nodes upwards and perform merge or prune operations to avoid redundant structures remaining.

[0127] Furthermore, the remaining valid nodes are re-sorted by timestamp, and the merge operation is re-executed according to the frequency value to update the Merkle tree structure and maintain the integrity and timeliness of the root hash value authentication path.

[0128] Optionally, the method also includes cache optimization of time frequency:

[0129] Define a path weight calculation formula to calculate the cache priority weight for each Merkle path, which is used to prioritize recently frequently accessed paths in the cache: (4)

[0130] Where, and It is an adjustable parameter, the default , .

[0131] Furthermore, the LRU-K algorithm (K=2) is used to record the timestamps of the two most recent accesses of each path. When the cache space reaches its upper limit, paths with low weight and long time intervals between the two accesses are eliminated first.

[0132] Furthermore, if the path in the cache contains an invalid node, the path is immediately removed from the cache, and the invalid node cleanup mechanism of the Merkle tree main structure is triggered to ensure that the cached data is consistent with the main structure and improve verification performance.

[0133] In this embodiment, a Merkle tree authentication method based on dual frequency and time features is proposed to address the challenges faced in data integrity verification and network security. By innovatively integrating timestamp and frequency features into the Merkle tree construction and authentication process, this method aims to improve data verification efficiency, reduce storage resource consumption, and enhance data security, adapting to the stringent data processing and security requirements of the digital age.

[0134] Figure 4 This is a block diagram of a MERKLE tree authentication device based on frequency and time dual feature perception according to an exemplary embodiment. The device is used in a MERKLE tree authentication method based on frequency and time dual feature perception. Figure 4 The device includes an acquisition module 310, an arrangement module 320, a merging module 330, a construction module 340, and an authentication module 350.

[0135] The acquisition module 310 is used to obtain a set of data nodes to be used for Merkle tree construction, and to generate a corresponding hash node for each data node in the data node set through the SM3 algorithm. The pointer of the generated hash node points to the corresponding data node, and the generated hash node inherits the timestamp and frequency value of the corresponding data node.

[0136] The arrangement module 320 is used to arrange all hash nodes in the increasing order of the expiration timestamps of the hash nodes to obtain an ordered queue.

[0137] The merging module 330 is used to traverse the ordered queue from left to right, find multiple adjacent nodes to merge, and generate parent nodes of the multiple adjacent nodes.

[0138] The construction module 340 is used to replace the corresponding multiple adjacent nodes in the ordered queue with the generated parent node, and recursively repeat the merging and replacing process until only one hash node remains in the ordered queue as the root node of the Merkle tree, thereby obtaining a constructed Merkle tree structure.

[0139] The authentication module 350 is used to start from the root node of the Merkle tree, perform a recursive binary search from top to bottom based on the timestamps of the hash nodes to locate the target node, and record the path from the target node to the root node as the Merkle path. The hash values ​​of the hash nodes of the Merkle path are verified in reverse order from bottom to top, and the sibling nodes of the Merkle path are hashed layer by layer to verify whether the hash value of the root node is consistent, thereby verifying the integrity and correctness of the target node.

[0140] In this embodiment, a Merkle tree authentication method based on dual frequency and time features is proposed to address the challenges faced in data integrity verification and network security. By innovatively integrating timestamp and frequency features into the Merkle tree construction and authentication process, this method aims to improve data verification efficiency, reduce storage resource consumption, and enhance data security, adapting to the stringent data processing and security requirements of the digital age.

[0141] Figure 5 : is a structural diagram of a MERKLE tree authentication device provided by an embodiment of the present invention, such as Figure 5 As shown, the MERKLE tree authentication device may include the above Figure 4 Optionally, the MERKLE tree authentication device 410 may include a first processor 2001 .

[0142] Optionally, the MERKLE tree authentication device 410 may further include a memory 2002 and a transceiver 2003 .

[0143] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.

[0144] The following combination Figure 5 The following describes the components of the MERKLE tree authentication device 410:

[0145] The first processor 2001 is the control center of the MERKLE tree authentication device 410 and can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs) or one or more field programmable gate arrays (FPGAs).

[0146] Optionally, the first processor 2001 may execute various functions of the MERKLE tree authentication device 410 by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002 .

[0147] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 5 CPU0 and CPU1 are shown in FIG.

[0148] In a specific implementation, as an embodiment, the MERKLE tree authentication device 410 may also include multiple processors, such as Figure 5 1 and 2. The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). A processor herein can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0149] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled by the first processor 2001 for execution. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0150] Alternatively, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, a random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, an optical disc storage (including a compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or exist independently and be authenticated by the interface circuit ( Figure 5 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0151] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0152] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 5 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0153] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently and authenticate the device 410 through the interface circuit ( Figure 5 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0154] It should be noted that Figure 5 The structure of the MERKLE tree authentication device 410 shown in the figure does not constitute a limitation on the router. The actual knowledge structure identification device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0155] In addition, the technical effects of the MERKLE tree authentication device 410 can refer to the technical effects of the frequency-time dual feature perception MERKLE tree authentication method described in the above method embodiment, and will not be repeated here.

[0156] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc.

[0157] It should also be understood that the memory in the embodiments of the present invention may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0158] The above embodiments can be implemented in whole or in part via software, hardware (e.g., circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product comprises one or more computer instructions or computer programs. When loaded or executed on a computer, the processes or functions described in accordance with the embodiments of the present invention are fully or partially performed. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired means (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server or data center that contains a collection of one or more available media. The available medium can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0159] It should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. A and B can be singular or plural. Furthermore, the character " / " as used herein generally indicates an "or" relationship between the associated objects, but it may also indicate an "and / or" relationship. For specific understanding, please refer to the context.

[0160] In this disclosure, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, "at least one of a, b, or c" can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0161] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0162] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0163] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0164] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of the device or unit, which can be electrical, mechanical or other forms.

[0165] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0166] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0167] If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical disks.

[0168] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A frequency-time dual feature perception MERKLE tree authentication method, characterized in that: The method comprises: S1. Obtain a set of data nodes to be used for Merkle tree construction. For each data node in the set of data nodes, generate a corresponding hash node using the SM3 algorithm. The pointer of the generated hash node points to the corresponding data node, and the generated hash node inherits the timestamp and frequency value of the corresponding data node. S2. Arrange all hash nodes in ascending order of their expiration timestamps to obtain an ordered queue. S3. Traverse the ordered queue from left to right, find multiple adjacent nodes, merge them, and generate parent nodes of the multiple adjacent nodes; S4. Use the generated parent node to replace the corresponding multiple adjacent nodes in the ordered queue, and recursively repeat the merging and replacing process until only one hash node remains in the ordered queue as the root node of the Merkle tree, thereby obtaining the constructed Merkle tree structure. S5. Starting from the root node of the Merkle tree, perform a recursive binary search from top to bottom based on the timestamps of the hash nodes to locate the target node, and record the path from the target node to the root node as the Merkle path. Verify the hash values ​​of the hash nodes of the Merkle path in reverse order from bottom to top, hash the sibling nodes of the Merkle path layer by layer, verify whether the hash value of the root node is consistent, and then verify the integrity and correctness of the target node.

2. The frequency-time dual feature-aware MERKLE tree authentication method according to claim 1, characterized in that: The method for adding a new data node to the Merkle tree includes: Use binary search to locate and delete invalid nodes in the Merkle tree to obtain a Merkle tree that retains valid data; According to the increasing order of the expiration timestamps, the insertion position of the new data node is determined by binary search, and the Merkle tree is rebuilt according to the frequency value.

3. The frequency-time dual feature-aware MERKLE tree authentication method according to claim 1, characterized in that: The recursive binary search in S5 to locate the target node includes: In the process of performing recursive binary search to locate the target node, the nodes with invalid hash values ​​and the subtrees corresponding to the nodes with invalid hash values ​​are pruned to avoid repeated verification of invalid nodes in subsequent queries.

4. The frequency-time dual feature-aware MERKLE tree authentication method according to claim 1, characterized in that: The step of finding multiple adjacent nodes and merging them to generate parent nodes of the multiple adjacent nodes in S3 includes: Find the two adjacent nodes with the smallest sum of frequency values ​​and merge them to generate the parent node of multiple adjacent nodes; Alternatively, the weight of each hash node is calculated based on a dynamic weight calculation formula, and the two adjacent nodes with the smallest sum of weights are found and merged. If there are multiple adjacent node pairs whose sum of weights is less than a preset similarity threshold, the adjacent node pair with the smallest sum of timestamps is selected for merging to generate the parent node of the multiple adjacent nodes. The timestamp of the parent node is the average of the timestamps of the two corresponding adjacent nodes; the frequency value of the parent node is the sum of the frequency values ​​of the two corresponding adjacent nodes; the hash value of the parent node is the sum of the hash values ​​of the two corresponding adjacent nodes calculated by the SM3 algorithm; The dynamic weight calculation formula is as follows (1): （1）。 5. The frequency-time dual feature-aware MERKLE tree authentication method according to claim 1, characterized in that: The step of finding multiple adjacent nodes and merging them to generate a parent node of the multiple adjacent nodes in S3 further includes: A frequency threshold is set according to the size of the data node set, and multiple consecutive adjacent nodes whose sum of frequency values ​​is not greater than the frequency threshold are obtained and merged to generate a parent node of the multiple adjacent nodes; Among them, the timestamp of the parent node is the median of the timestamps of the corresponding multiple adjacent nodes; the frequency value of the parent node is the sum of the frequency values ​​of the corresponding multiple adjacent nodes; the hash value of the parent node is the SM3 hash result of the hash values ​​of multiple adjacent nodes.

6. The frequency-time dual feature-aware MERKLE tree authentication method according to claim 1, characterized in that: The method further includes: automatic cleaning and reconstruction of failed nodes; The automatic cleanup and reconstruction of failed nodes includes: Periodically scan the ordered queue, use the skip table structure to find the hash nodes whose invalid timestamps are earlier than the current time, and mark them as invalid nodes; After deleting the invalid node, if the parent node of the invalid node has only one valid child node left, recursively search for the sibling nodes and perform a merge; The hash nodes except the invalid nodes are rearranged in the ascending order of the invalid timestamps, and the merge operation is re-executed according to the frequency value, and the Merkle tree structure is updated to maintain the validity of the authentication.

7. The frequency-time dual feature-aware MERKLE tree authentication method according to claim 1, characterized in that: The method further comprises: The weight of each Merkle path is calculated based on the cache priority weight calculation formula. The timestamps of the two most recent accesses to each Merkle path are recorded using the LRU-K algorithm, and the access interval is calculated based on the timestamps of the two most recent accesses. When the cache space reaches the upper limit, Merkle paths with low weights and long access intervals are eliminated first. The cache priority weight calculation formula is as follows (2): （2） Where, and It is an adjustable parameter.

8. A MERKLE tree authentication device based on dual frequency and time features, wherein the MERKLE tree authentication device based on dual frequency and time features is used to implement the MERKLE tree authentication method based on dual frequency and time features as claimed in any one of claims 1 to 7, characterized in that: The device comprises: An acquisition module is used to obtain a set of data nodes to be used for Merkle tree construction, and to generate a corresponding hash node for each data node in the set of data nodes using the SM3 algorithm. The pointer of the generated hash node points to the corresponding data node, and the generated hash node inherits the timestamp and frequency value of the corresponding data node; The arrangement module is used to arrange all hash nodes in the increasing order of their expiration timestamps to obtain an ordered queue; A merging module is used to traverse the ordered queue from left to right, find multiple adjacent nodes to merge, and generate parent nodes of the multiple adjacent nodes; The construction module is used to replace the corresponding multiple adjacent nodes in the ordered queue with the generated parent node, and recursively repeat the merging and replacing process until only one hash node remains in the ordered queue as the root node of the Merkle tree, thereby obtaining the constructed Merkle tree structure; The authentication module is used to start from the root node of the Merkle tree, perform recursive binary search from top to bottom based on the timestamps of the hash nodes to locate the target node, and record the path from the target node to the root node as the Merkle path. The hash values ​​of the hash nodes of the Merkle path are verified in reverse order from bottom to top, and the sibling nodes of the Merkle path are hashed layer by layer to verify whether the hash value of the root node is consistent, thereby verifying the integrity and correctness of the target node.

9. A MERKLE tree authentication device, characterized in that: The MERKLE tree authentication device includes: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, which can be called by a processor to execute the method according to any one of claims 1 to 7.

Images