Merkle tree authentication method and device with frequency-time dual feature perception

Abstract

The application provides a frequency-time double feature sensing MERKLE tree authentication method and device, and relates to the technical field of network security. The method comprises the following steps: generating a hash node upward for each data node through an SM3 algorithm; arranging all the hash nodes in an increasing order of invalidation time stamps; finding a parent node by merging a plurality of adjacent nodes from left to right and replacing the adjacent nodes to construct a Merkle tree structure; starting from a root node, locating a target node and recording a Merkle path by recursive binary search according to the time stamps from top to bottom; verifying the hash value of the Merkle path from bottom to top in a reverse order; and verifying whether the hash value of the root node is consistent by hashing and aggregating the sibling nodes of the Merkle path layer by layer. The application optimizes the data verification efficiency of the Merkle tree by combining the time stamp and frequency value double information, and is suitable for scenarios with time effectiveness and access frequency characteristics.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of network security, in particular to a frequency-time dual feature-aware MERKLE tree authentication method and device. BACKGROUND

[0002] With the rapid development of information technology, the scale and complexity of data in various fields are growing explosively. From financial transaction records, medical health data, to massive user information on Internet platforms, the integrity and consistency of these data are directly related to the stable operation of business, the accuracy of decision-making, and the protection of user rights. During data storage and transmission, ensuring that data is not tampered with, lost, or incorrectly processed is one of the core challenges in the field of information security.

[0003] Merkle tree, as an efficient data structure, is widely used in blockchain, distributed storage systems, and data backup scenarios to verify data integrity. In blockchain, Merkle tree compresses a large number of transaction data into a hash value that is easy to verify, allowing nodes to quickly verify the authenticity and integrity of transactions without traversing the entire transaction ledger. In distributed storage systems, Merkle tree can efficiently detect changes to data blocks during transmission or storage, ensuring data reliability. However, traditional Merkle trees have significant shortcomings when faced with the complex and changing data characteristics of the real world.

[0004] On the one hand, many data have obvious time effectiveness, such as real-time market data in financial markets, and cargo location information in logistics transportation. Traditional Merkle trees do not take into account the time attribute of data, making it difficult to quickly distinguish between invalid data when processing these time-sensitive data, resulting in low verification efficiency. On the other hand, data access frequency varies greatly in different scenarios. For example, the access volume of popular news articles is much higher than that of ordinary information, and frequently accessed data requires more efficient verification mechanisms to reduce response time. The traditional Merkle tree ignores the data access frequency, making it difficult to reduce unnecessary hash calculations in high-frequency access data scenarios, thereby reducing the overall system performance.

[0005] In the field of network security, the large-scale deployment of Internet of Things devices has made data security problems more serious. Billions of Internet of Things devices, from smart home devices to industrial sensors, are connected to each other and generate massive amounts of data. The data collected by these devices is the cornerstone of building intelligent applications and decision-making, and once a device node is maliciously attacked, tampered with or falsified, it will pose a serious threat to the reliability and security of the entire Internet of Things system. During device interconnection, ensuring mutual trust between devices and fast data verification is crucial. However, the traditional Merkle algorithm has room for improvement in terms of adaptability to these complex data characteristics. SUMMARY

[0006] In order to solve the technical problems in the prior art that the traditional Merkle tree does not consider the time attribute of the data, so that when processing these time-sensitive data, it is difficult to quickly distinguish invalid data, resulting in low verification efficiency, and the traditional Merkle tree ignores the data access frequency, so that in the high-frequency access data scene, a large number of unnecessary hash calculations reduce the overall performance of the system, the embodiments of the present application provide a frequency-time dual feature aware MERKLE tree authentication method and device. The technical solution is as follows:

[0007] On the one hand, a frequency-time dual feature aware MERKLE tree authentication method is provided, which is realized by a MERKLE tree authentication device, and the method comprises:

[0008] S1, obtaining a data node set to be constructed by a Merkle tree, generating a corresponding hash node for each data node in the data node set by an SM3 algorithm, the pointer of the generated hash node points to the corresponding data node, and the generated hash node inherits the timestamp and frequency value of the corresponding data node.

[0009] S2, arranging all hash nodes in ascending order of invalid timestamp of the hash node to obtain an ordered queue.

[0010] S3, traversing the ordered queue from left to right, finding a plurality of adjacent nodes for merging to generate parent nodes of the plurality of adjacent nodes.

[0011] S4, replacing the plurality of adjacent nodes in the ordered queue with the generated parent nodes, recursively repeating the merging and replacing process until only one hash node is left in the ordered queue as the root node of the Merkle tree, and obtaining a constructed Merkle tree structure.

[0012] S5, starting from the root node of the Merkle tree, recursively performing binary search according to the timestamp of the hash node from top to bottom to locate a target node, and recording a path from the target node to the root node as a Merkle path, performing bottom-up reverse order verification on the hash value of the hash node of the Merkle path, and performing hash aggregation on the sibling nodes of the Merkle path layer by layer, verifying whether the hash value of the root node is consistent, and further verifying the integrity and correctness of the target node.

[0013] Optionally, the method for adding a new data node to the Merkle tree comprises:

[0014] locating and deleting the invalid node in the Merkle tree by binary search to obtain a Merkle tree retaining valid data.

[0015] According to the increasing order of the invalid time stamp, the insertion position of the new data node is determined by binary search, and the Merkle tree is reconstructed according to the frequency value.

[0016] Optionally, the recursive binary search in S5 to locate the target node comprises:

[0017] In the process of locating the target node by recursive binary search, the hash value invalid node and the sub-tree corresponding to the hash value invalid node are pruned to avoid repeated verification of the invalid node in the subsequent query process.

[0018] Optionally, the finding of the plurality of adjacent nodes for merging to generate the parent node of the plurality of adjacent nodes in S3 comprises:

[0019] The two adjacent nodes with the smallest sum of frequency values are found for merging to generate the parent node of the plurality of adjacent nodes.

[0020] Or, the weight of each hash node is calculated based on a dynamic weight calculation formula, the two adjacent nodes with the smallest sum of weights are found for merging, and if there are multiple adjacent node pairs with the sum of weights less than a preset similarity threshold, the adjacent node pair with the smallest sum of time stamps is selected for merging to generate the parent node of the plurality of adjacent nodes.

[0021] The time stamp of the parent node is the average of the time stamps of the corresponding two adjacent nodes; the frequency value of the parent node is the sum of the frequency values of the corresponding two adjacent nodes; and the hash value of the parent node is the sum of the hash values of the corresponding two adjacent nodes calculated by the SM3 algorithm.

[0022] The dynamic weight calculation formula is as follows:

[0023] (1).

[0024] Optionally, the finding of the plurality of adjacent nodes for merging to generate the parent node of the plurality of adjacent nodes in S3 further comprises:

[0025] The frequency threshold is set according to the size of the data node set, the plurality of adjacent nodes with the sum of frequency values not greater than the frequency threshold are obtained for merging to generate the parent node of the plurality of adjacent nodes.

[0026] The time stamp of the parent node is the median of the time stamps of the corresponding plurality of adjacent nodes; the frequency value of the parent node is the sum of the frequency values of the corresponding plurality of adjacent nodes; and the hash value of the parent node is the SM3 hash result after splicing the hash values of the plurality of adjacent nodes.

[0027] Optionally, the method further comprises: automatic cleaning and reconstruction of the invalid node.

[0028] The automatic cleaning and reconstruction of the invalid node comprises:

[0029] Periodically scan the ordered queue, find the hash node with invalid timestamp earlier than the current time using skip list structure, and mark it as invalid node.

[0030] After deleting the invalid node, if the parent node of the invalid node has only one valid child node, recursively find the sibling node upwards and perform merging.

[0031] Arrange the hash nodes other than the invalid node in ascending order of invalid timestamp, and re-execute the merging operation according to the frequency value, and update the Merkle tree structure to maintain the authentication validity.

[0032] Optionally, the method further comprises:

[0033] Calculate the weight of each Merkle path based on the cache priority weight calculation formula; record the time stamp of the last two accesses of each Merkle path using LRU-K algorithm, and calculate the access interval according to the time stamp of the last two accesses; when the cache space reaches the upper limit, preferentially eliminate the Merkle path with low weight and long access interval.

[0034] wherein the cache priority weight calculation formula is as follows formula (2):

[0035] (2)

[0036] In the formula, and are adjustable parameters.

[0037] On the other hand, a frequency-time dual feature aware MERKLE tree authentication device is provided, which is applied to the frequency-time dual feature aware MERKLE tree authentication method, and the device comprises:

[0038] The acquisition module is configured to acquire a data node set to be constructed into a Merkle tree, generate a corresponding hash node for each data node in the data node set upwards through an SM3 algorithm, a pointer of the generated hash node points to the corresponding data node, and the generated hash node inherits the timestamp and frequency value of the corresponding data node.

[0039] The arrangement module is configured to arrange all hash nodes in ascending order of invalid timestamp of the hash nodes to obtain an ordered queue.

[0040] The merging module is configured to traverse the ordered queue from left to right, find a plurality of adjacent nodes for merging, and generate parent nodes of the plurality of adjacent nodes.

[0041] The constructing module is configured to replace the corresponding multiple adjacent nodes in the ordered queue with the generated parent node, and recursively repeat the merging and replacing process until only one hash node is left in the ordered queue as a root node of the Merkle tree, so as to obtain the constructed Merkle tree structure.

[0042] The authentication module is configured to locate the target node from top to bottom according to the time stamp of the hash node by recursive binary search from the root node of the Merkle tree, record a path from the target node to the root node as a Merkle path, verify the hash value of the hash node of the Merkle path from bottom to top in reverse order, and verify whether the hash value of the root node is consistent by hashing and aggregating the sibling nodes of the Merkle path layer by layer, so as to verify the integrity and correctness of the target node.

[0043] Optionally, the method for adding a new data node to the Merkle tree comprises:

[0044] The Merkle tree is located and the invalid node is deleted by binary search, so as to obtain a Merkle tree retaining valid data.

[0045] According to the increasing order of the invalid time stamp, the insertion position of the new data node is determined by binary search, and the Merkle tree is reconstructed according to the frequency value.

[0046] Optionally, the authentication module is further configured to:

[0047] In the process of locating the target node by recursive binary search, the node with a hash value and the subtree corresponding to the node with a hash value are pruned, so as to avoid repeated verification of the invalid node in the subsequent query process.

[0048] Optionally, the merging module is further configured to:

[0049] The two adjacent nodes with the smallest sum of frequency values are found to be merged, and the parent node of the multiple adjacent nodes is generated.

[0050] Or, the weight of each hash node is calculated based on a dynamic weight calculation formula, the two adjacent nodes with the smallest sum of weights are found to be merged, and if there are multiple adjacent node pairs with a sum of weights less than a preset similarity threshold, the adjacent node pair with the smallest sum of time stamps is selected to be merged, and the parent node of the multiple adjacent nodes is generated.

[0051] The time stamp of the parent node is the average of the time stamps of the corresponding two adjacent nodes; the frequency value of the parent node is the sum of the frequency values of the corresponding two adjacent nodes; and the hash value of the parent node is the sum of the hash values of the corresponding two adjacent nodes calculated by the SM3 algorithm.

[0052] The dynamic weight calculation formula is as follows:

[0053] (1).

[0054] Optionally, the merging module is further configured to:

[0055] The frequency threshold is set according to the size of the data node set, the continuous multiple adjacent nodes with a sum of frequency values not greater than the frequency threshold are merged, and the parent node of the multiple adjacent nodes is generated.

[0056] The timestamp of the parent node is the median of the timestamps of the corresponding multiple adjacent nodes; the frequency value of the parent node is the sum of the frequency values of the corresponding multiple adjacent nodes; and the hash value of the parent node is an SM3 hash result after splicing the hash values of the multiple adjacent nodes.

[0057] Optionally, the method further comprises automatically cleaning and reconstructing the invalid nodes.

[0058] The automatic cleaning and reconstruction of the invalid nodes comprises:

[0059] Periodically scanning the ordered queue, searching for hash nodes with invalid timestamps earlier than the current time by using a skip list structure, and marking the hash nodes as invalid nodes.

[0060] After deleting the invalid nodes, if the parent node of the invalid nodes has only one valid child node, the sibling nodes are recursively searched upwards and merged.

[0061] The hash nodes except the invalid nodes are rearranged in an increasing order of invalid timestamps, and the merging operation is re-performed according to the frequency values, and the Merkle tree structure is updated to maintain the authentication validity.

[0062] Optionally, the method further comprises:

[0063] The weight of each Merkle path is calculated based on a cache priority weight calculation formula; the timestamps of the last two accesses of each Merkle path are recorded using an LRU-K algorithm, and the access interval is calculated according to the timestamps of the last two accesses; when the cache space reaches the upper limit, the Merkle paths with low weights and long access intervals are preferentially eliminated.

[0064] The cache priority weight calculation formula is as follows:

[0065] (2)

[0066] In the formula, and are adjustable parameters.

[0067] In another aspect, a MERKLE tree authentication device is provided, comprising: a processor; a memory having computer readable instructions stored thereon, which, when executed by the processor, implement any one of the above frequency-time dual feature perception based MERKLE tree authentication methods.

[0068] In another aspect, a computer readable storage medium is provided, having at least one instruction stored therein, which is loaded and executed by a processor to implement any one of the above frequency-time dual feature perception based MERKLE tree authentication methods.

[0069] The technical solutions provided by the embodiments of the present application have at least the following beneficial effects:

[0070] In the present application, in view of the difficulties faced in the field of data integrity verification and network security, a frequency-time dual feature perception based Merkle tree authentication method is proposed. By innovatively integrating time stamp and frequency feature into the construction and authentication process of Merkle tree, the purpose is to improve the data verification efficiency, reduce the storage resource consumption, and strengthen the security of data, so as to adapt to the strict requirements of digital era on data processing and security protection. BRIEF DESCRIPTION OF DRAWINGS

[0071] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0072] Figure 1 is a frequency-time dual feature perception based Merkle tree authentication method flowchart provided by the embodiments of the present application;

[0073] Figure 2 is a frequency-time dual feature perception based Merkle tree authentication method flowchart provided by the embodiments of the present application;

[0074] Figure 3 is a 4-node Merkle tree structure diagram provided by the embodiments of the present application;

[0075] Figure 4 is a frequency-time dual feature perception based Merkle tree authentication device block diagram provided by the embodiments of the present application;

[0076] Figure 5 is a structure diagram of a Merkle tree authentication device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0077] The technical solutions in the present application will be described below with reference to the drawings.

[0078] In the embodiments of the present application, the words such as "example", "for example" are used to represent an example, illustration or description. Any embodiment or design scheme described as "example" in the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. Rather, the word "example" is intended to present the concept in a specific manner. In addition, in the embodiments of the present application, the meaning expressed by "and / or" can be both, or can be one of the two.

[0079] In the embodiments of the present application, "image" and "picture" can be used interchangeably at times, and it should be pointed out that the meanings expressed are consistent when the distinction is not emphasized. "Of", "corresponding" and "corresponding" can be used interchangeably at times, and it should be pointed out that the meanings expressed are consistent when the distinction is not emphasized.

[0080] In the embodiments of the present application, sometimes the subscript such as W1 can be written in the form of non-subscript such as W1, and the meanings expressed are consistent when the distinction is not emphasized.

[0081] In order to make the technical problems, technical solutions and advantages of the present application more clear, the following will be described in detail with reference to the drawings and specific embodiments.

[0082] The embodiments of the present application provide a frequency-time dual feature perception MERKLE tree authentication method, which can be implemented by a MERKLE tree authentication device, which can be a terminal or a server. As shown in the flow chart of the frequency-time dual feature perception MERKLE tree authentication method, the processing flow of the method can include the following steps: Figure 1 、 Figure 2 As shown in the flow chart of the frequency-time dual feature perception MERKLE tree authentication method, the processing flow of the method can include the following steps:

[0083] S1, obtaining a data node set to be constructed by a Merkle tree, generating a corresponding hash node for each data node in the data node set by an SM3 algorithm, the pointer of the generated hash node points to the corresponding data node, and the generated hash node inherits the timestamp and frequency value of the corresponding data node.

[0084] In a feasible implementation, for all data nodes of the Merkle tree, the data node set is , , represents the number of data nodes, and for each data node uses the SM3 algorithm to generate a hash node upwards , The pointer of each hash node points to the corresponding data node accurately, and all hash nodes inherit the timestamp and frequency value of the data node. The timestamp is used to identify the timeliness of the data, and the frequency value is used to record the access frequency of the data node.

[0085] S2, arrange all hash nodes in ascending order of the invalidation timestamp of the hash node to obtain an ordered queue.

[0086] In a feasible implementation, all hash nodes are arranged in ascending order of the invalidation timestamp to form an ordered queue. The invalidation timestamp reflects the timeliness of the data node, and the node invalidating earlier is arranged in front of the queue, and the node invalidating later is arranged in the back of the queue. This step ensures that the node with poor timeliness can be processed preferentially in the subsequent merging process.

[0087] S3, traverse the ordered queue from left to right to find multiple adjacent nodes to be merged to generate parent nodes of the multiple adjacent nodes.

[0088] Optionally, the finding of multiple adjacent nodes to be merged to generate parent nodes of the multiple adjacent nodes in S3 comprises:

[0089] The hash node frequency set is The two adjacent nodes with the smallest sum of frequency values are found to be merged, and the parent nodes of the two nodes are generated. , wherein represents the frequency node.

[0090] In a feasible implementation, the ordered queue is traversed from left to right to find the two adjacent nodes with the smallest sum of frequencies to be merged. After the two nodes are found, the parent nodes of the two nodes are generated. The generation rule of the parent node is as follows: the timestamp of the parent node takes the average of the timestamps of the two child nodes, the frequency value of the parent node takes the sum of the frequency values of the two child nodes, and the hash value of the parent node takes the sum of the hash values of the two child nodes (calculated through the SM3 algorithm), , wherein represents the hash value of the parent node, represents the left child node hash value, represents the right child node hash value.

[0091] Alternatively, the weight of each hash node is calculated based on a dynamic weight calculation formula, the two adjacent nodes with the smallest sum of weights are found to be merged, if the sum of weights of multiple adjacent node pairs is smaller than a preset similarity threshold, the adjacent node pair with the smallest sum of timestamps is selected to be merged to generate parent nodes of the multiple adjacent nodes. The weight reflects the importance and remaining effective time of the node.​​

[0092] In a feasible implementation, the dynamic weight of each hash node is calculated by formula (1), which ensures that the weight of the node close to failure is reduced to prioritize processing:

[0093] (1).

[0094] Further, all hash nodes are arranged in an ordered queue in ascending order of failure time stamps. From left to right, the queue is traversed, and the pair of nodes with the smallest weight sum in adjacent nodes is selected for merging. If the weight sums of multiple node pairs are similar, the node with the smaller time stamp is preferentially selected for merging.

[0095] Further, the time stamp of the parent node is the weighted average of the time stamps of the two child nodes, and the frequency value is the sum of the frequencies of the child nodes. The hash value of the parent node is the concatenation result of the hash values of the two child nodes, which is calculated using the SM3 algorithm.

[0096] Further, the parent node generated by merging is replaced with the original atomic node, and the above process is repeated until all nodes are merged into a root node.

[0097] Optionally, in S3, finding multiple adjacent nodes for merging to generate a parent node of the multiple adjacent nodes further includes:

[0098] According to the size of the data node set, a frequency threshold is set, and multiple adjacent nodes with a frequency value sum not greater than the frequency threshold are obtained for merging to generate a parent node of the multiple adjacent nodes.

[0099] Among them, the time stamp of the parent node is the median of the time stamps of the corresponding multiple adjacent nodes; the frequency value of the parent node is the sum of the frequency values of the corresponding multiple adjacent nodes; and the hash value of the parent node is the SM3 hash result after concatenating the hash values of the multiple adjacent nodes.

[0100] In a feasible implementation, the present application proposes a multi-fork Merkle tree support mechanism, specifically:

[0101] According to the size of the data, a frequency threshold (such as a merging frequency and a rule of less than or equal to 100) is set for multi-node merging determination.

[0102] Further, in the ordered queue of hash nodes, find nodes whose frequency value sum is less than or equal to the set threshold. If the condition is met, merge the nodes to generate a parent node.

[0103] Further, the hash value of the parent node is the SM3 hash result after concatenating the hash values of child nodes, i.e. The timestamp of the parent node is the median of the timestamps of the two child nodes, and the frequency value is the sum of the frequencies of the two child nodes.

[0104] Further, the parent node is inserted into the original queue to replace the original two child nodes, and the process is repeated until all nodes are merged and a Merkle tree root node is generated, forming a multi-way Merkle tree structure.

[0105] S4, the generated parent node replaces the corresponding multiple adjacent nodes in the ordered queue, and the merging and replacing process is recursively repeated until only one hash node remains in the ordered queue as the root node of the Merkle tree, obtaining the constructed Merkle tree structure.

[0106] In a feasible implementation, the above steps S1-S4 are the initialization of the Merkle tree: the data is grouped into a Merkle tree structure according to time and frequency, and a 4-node Merkle tree structure diagram is shown in Figure 3 .

[0107] Specifically, the generated parent node replaces the two child nodes in the original queue. The timestamp, frequency value and hash value of the parent node are updated according to the rules of step S3. The above merging process is recursively repeated until only one node remains in the queue, which is the root node of the Merkle tree.

[0108] S5, starting from the root node of the Merkle tree, recursively binary search the target node according to the timestamp of the hash node from top to bottom, and record the path from the target node to the root node as the Merkle path. The hash values of the hash nodes of the Merkle path are verified in reverse order from bottom to top, and the sibling nodes of the Merkle path are aggregated layer by layer to verify whether the hash value of the root node is consistent, and further verify the integrity and correctness of the target node.

[0109] In a feasible implementation, step S5 is data verification: from the Merkle root, use a recursive algorithm to do binary search, and increase the frequency of the node query, record the Merkle path, and do integrity verification.

[0110] Specifically, starting from the root node of the Merkle tree, recursively binary search according to the timestamp characteristic value from top to bottom. In the searching process, if a node with invalid hash value (i.e. the invalid timestamp of the node is earlier than the current time) is encountered, the node and its subtree are pruned to avoid repeated verification of the invalid node in the subsequent query process.

[0111] Further, the query path is recorded in the searching process to find the specified data node, and the path is recorded as the Merkle path, and the frequencies of the data node and the parent node are increased: ​​

[0112] (2)

[0113] wherein, denotes the Merkel path, denotes the timestamp of the th node on the Merkel path, denotes the hash value of the th node, denotes the number of nodes on the path.

[0114] Further, according to the Merkel path, the node hash values of the path are verified in reverse order from bottom to top, and the sibling nodes of the path are layer by layer to do hash aggregation:

[0115] (3)

[0116] wherein, denotes the Merkel root node, denotes the root node hash value, denotes the node hash value on the Merkel path, denotes the sibling node hash value of the corresponding node, both of which are hash aggregated and gradually go up to the root node.

[0117] The Merkel root hash value is verified for consistency. Through this bottom-up hash verification method, the integrity and correctness of the target node can be accurately verified, and it is ensured that the data is not tampered with during storage and transmission.

[0118] Optionally, the method for adding a new data node to the Merkel tree comprises:

[0119] locating and deleting the invalid node of the Merkel tree through binary search to obtain a Merkel tree retaining valid data.

[0120] According to the increasing order of the invalid timestamp, the insertion position of the new data node is determined through binary search, and the Merkel tree is reconstructed according to the frequency value.

[0121] In a feasible implementation, the application further comprises reconstruction of the Merkel: when a new node is added, the Merkel tree is reconstructed, the expired node is deleted, the node linked list is inserted in time sequence, and the Merkel tree is reconstructed from bottom to top according to the frequency.

[0122] Specifically, when a new node needs to be added, first, the invalid node is quickly deleted through binary search to ensure that only valid nodes are included in the queue, thereby improving the effectiveness of the data. Then, the new node is inserted into the ordered queue according to the timestamp order. Finally, the Merkle tree structure is reconstructed according to the frequency characteristics to form a kind of tilted Merkle ordered tree structure. This structure can better adapt to the timeliness and access frequency characteristics of the data nodes.

[0123] For the authentication process of large data integrity and consistency using the Merkle tree, the present application proposes a Merkle tree construction and verification method based on time-frequency dual feature perception, aiming to optimize the data verification efficiency of the Merkle tree by combining the timestamp and frequency value dual information, especially suitable for scenarios with timeliness and access frequency characteristics. In constructing the Merkle tree, first, all data nodes generate their own hash nodes upwards according to the SM3 algorithm, the hash nodes inherit the timestamp and frequency value of the data nodes, all hash nodes are arranged in increasing order of invalid timestamp to form an ordered queue, the invalid timestamp reflects the timeliness of the data nodes, the nodes that are invalid earlier are arranged in front, and the nodes that are invalid later are arranged in back. Then, the queue is traversed from left to right, and the two adjacent nodes with the smallest frequency sum are found, the frequency value reflects the access frequency or importance of the data nodes, the smaller the frequency sum, the lower the access frequency of the two nodes, which is suitable for priority merging. After finding the two nodes, the parent node is generated, and the parent node replaces the two child nodes in the original queue, the timestamp of the parent node takes the average of the timestamps of the child nodes, the frequency value takes the sum of the frequency values of the child nodes, and the hash value takes the sum of the hash values of the child nodes, this process ensures that the timestamp and frequency value of the parent node can comprehensively reflect the timeliness and access frequency of the child nodes. The above merging process is recursively repeated until only the last two nodes are left in the queue, the two nodes are merged to generate the Merkle root node, and finally a kind of tilted Merkle tree structure is formed, which can better adapt to the timeliness and frequency characteristics of the data nodes. In verifying the data nodes, the binary search algorithm is used to recursively locate the target node, and the path from the target node to the Merkle root, i.e. the Merkle path, is recorded, the integrity and correctness of the target node are verified by performing hash check on the nodes in the path. When a new data node is added, first, the current invalid node is quickly located and deleted through binary search to ensure that only valid data is left in the tree, then the insertion position of the new node is determined through binary search to arrange it in increasing order of invalid timestamp, and finally the Merkle tree is reconstructed according to the latest frequency value to ensure the dynamic update and optimization of the tree structure.

[0124] The application has the advantages that the construction process is simple, compared with the current Merkle tree structure, the time stamp and frequency double feature perception are introduced, the speed of verifying the data node is greatly improved, the inclined Merkle tree structure uses fewer hash nodes, the cost of space storage is reduced, the SM3 algorithm of the national secret is used, compared with the SHA hash algorithm, the security of the data node is improved, and the Merkle tree structure can be effectively applied to the blockchain large-scale data verification scene.

[0125] Optionally, in order to further improve the performance and timeliness of the Merkle tree structure in processing dynamic data, the method further comprises: automatic cleaning and reconstruction of the invalid nodes.

[0126] The automatic cleaning and reconstruction of the invalid nodes comprises:

[0127] Periodically (such as every minute) scan the hash node queue, use the skip list structure to find the nodes with invalid time stamps earlier than the current time, and mark them as invalid.

[0128] Further, after deleting the invalid nodes, if the parent node of the invalid nodes has only one valid child node, the sibling nodes are recursively found upwards and the merging or pruning operation is performed to avoid the residual of redundant structures.

[0129] Further, the remaining valid nodes are reordered according to the time stamps, and the merging operation is re-executed according to the frequency value to update the Merkle tree structure, so that the integrity and timeliness of the root hash value authentication path are maintained.

[0130] Optionally, the method further comprises cache optimization of time and frequency:

[0131] A path weight calculation formula is defined to calculate the cache priority weight for each Merkle path, which is used to preferentially retain the recent high-frequency access path in the cache:

[0132] (4)

[0133] In the formula, and are adjustable parameters, and the default , .

[0134] Further, the LRU-K algorithm (K=2) is used to record the latest two access time stamps of each path, and when the cache space reaches the upper limit, the paths with low weight and long interval between the two access times are preferentially eliminated.

[0135] Further, if the path in the cache contains an invalid node, the path is immediately removed from the cache, and the invalid node cleaning mechanism of the Merkle tree main structure is triggered to ensure that the cache data is consistent with the main structure and improve the verification performance.

[0136] In the embodiment of the present application, aiming at the difficulties faced in the field of data integrity verification and network security, a Merkle tree authentication method based on frequency-time dual feature perception is proposed. By innovatively integrating time stamp and frequency feature into the construction and authentication process of Merkle tree, the efficiency of data verification is improved, the storage resource consumption is reduced, and the security of data is strengthened, so as to adapt to the stringent requirements of digital era on data processing and security protection.

[0137] Figure 4 is a frequency-time dual feature perception MERKLE tree authentication device block diagram according to an exemplary embodiment, which is used for frequency-time dual feature perception MERKLE tree authentication method. Referring to Figure 4 , the device includes an acquisition module 310, an arrangement module 320, a merging module 330, a construction module 340 and an authentication module 350. Among them:

[0138] The acquisition module 310 is used for acquiring a data node set to be constructed by Merkle tree, and generating a corresponding hash node for each data node in the data node set by SM3 algorithm, the pointer of the generated hash node points to the corresponding data node, and the generated hash node inherits the time stamp and frequency value of the corresponding data node.

[0139] The arrangement module 320 is used for arranging all hash nodes in ascending order of invalid time stamp of hash node, to obtain an ordered queue.

[0140] The merging module 330 is used for traversing the ordered queue from left to right, finding a plurality of adjacent nodes for merging, and generating a parent node of the plurality of adjacent nodes.

[0141] The construction module 340 is used for replacing the corresponding plurality of adjacent nodes in the ordered queue with the generated parent node, recursively repeating the merging and replacing process until only one hash node is left in the ordered queue as the root node of the Merkle tree, to obtain the constructed Merkle tree structure.

[0142] The authentication module 350 is used for starting from the root node of the Merkle tree, recursively locating the target node according to the time stamp of the hash node from top to bottom, and recording the path from the target node to the root node as the Merkle path, verifying the hash value of the hash node of the Merkle path from bottom to top in reverse order, and performing hash aggregation on the sibling nodes of the Merkle path layer by layer, verifying whether the hash value of the root node is consistent, and then verifying the integrity and correctness of the target node.

[0143] In the embodiment of the present application, aiming at the difficulties faced in the field of data integrity verification and network security, a Merkle tree authentication method based on frequency-time dual feature perception is proposed. By innovatively integrating timestamp and frequency feature into the construction and authentication process of Merkle tree, the efficiency of data verification is improved, the storage resource consumption is reduced, and the security of data is strengthened, so as to adapt to the strict requirements of digital era on data processing and security protection.

[0144] Figure 5 is a structural schematic diagram of a Merkle tree authentication device provided by the embodiment of the present application, as shown in Figure 5 , the Merkle tree authentication device can include the frequency-time dual feature perception Merkle tree authentication apparatus shown in Figure 4 . Optionally, the Merkle tree authentication device 410 can include a first processor 2001.

[0145] Optionally, the Merkle tree authentication device 410 can further include a memory 2002 and a transceiver 2003.

[0146] Among them, the first processor 2001 and the memory 2002 and the transceiver 2003 can be connected through a communication bus.

[0147] The specific components of the Merkle tree authentication device 410 will be introduced below Figure 5 :

[0148] Among them, the first processor 2001 is the control center of the Merkle tree authentication device 410, which can be one processor or a plurality of processing elements. For example, the first processor 2001 is one or more central processing units (CPU), which can also be an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiment of the present application, such as one or more digital signal processors (DSP), or one or more field programmable gate arrays (FPGA).

[0149] Optionally, the first processor 2001 can execute the various functions of the Merkle tree authentication device 410 by running or executing the software program stored in the memory 2002 and calling the data stored in the memory 2002.

[0150] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 5 CPU0 and CPU1 are shown in FIG.

[0151] In a specific implementation, as an embodiment, the MERKLE tree authentication device 410 may also include multiple processors, such as Figure 5 1 and 2. The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). A processor herein can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0152] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled by the first processor 2001 for execution. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0153] Alternatively, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, a random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, an optical disc storage (including a compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or exist independently and be authenticated by the interface circuit ( Figure 5 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0154] The transceiver 2003 is used to communicate with a network device or a terminal device.

[0155] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 5 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.

[0156] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently and authenticate the device 410 through the interface circuit ( Figure 5 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.

[0157] It should be noted that Figure 5 The structure of the MERKLE tree authentication device 410 shown in the figure does not constitute a limitation on the router. The actual knowledge structure identification device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0158] In addition, the technical effects of the MERKLE tree authentication device 410 can refer to the technical effects of the frequency-time dual feature perception MERKLE tree authentication method described in the above method embodiment, and will not be repeated here.

[0159] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc.

[0160] It should also be understood that the memory in the embodiments of the present application can be volatile or nonvolatile memory, or can include both volatile and nonvolatile memory. Nonvolatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), electrically EPROM (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as external cache. By way of example, and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous dynamic RAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0161] The above-described embodiments can be implemented in whole or in part by software, hardware (such as a circuit), firmware, or any combination thereof. When implemented in software, the above-described embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are wholly or partially generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center through a wired (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. containing one or more available medium collections. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state disk.

[0162] It should be understood that the term "and / or" herein merely describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. In addition, the character " / " herein generally represents that the associated objects before and after it are in an "or" relationship, but it can also represent an "and / or" relationship, which can be understood according to the context before and after it.

[0163] In the present application, "at least one" means one or more, and "multiple" means two or more. "At least one of the following" or the like means any combination of the items, including any combination of single or multiple items. For example, at least one of a, b, or c can represent a, b, c, a-b, a-c, b-c, or a-b-c, where a, b, and c can be single or multiple.

[0164] It should be understood that in various embodiments of the present application, the size of the sequence number of the above-described processes does not mean the order of execution, and the execution order of the processes should be determined according to their functions and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0165] Those skilled in the art can clearly understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0166] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the devices, apparatuses and units described above can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0167] In several embodiments provided by the present application, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0168] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0169] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically independently, or two or more units can be integrated into one unit.

[0170] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the present application that essentially contribute to the prior art or the parts of the technical solutions can be embodied in the form of software products. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0171] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A frequency-time dual feature perception MERKLE tree authentication method, characterized in that: The method comprises: S1. Obtain a set of data nodes to be used for Merkle tree construction. For each data node in the set of data nodes, generate a corresponding hash node using the SM3 algorithm. The pointer of the generated hash node points to the corresponding data node, and the generated hash node inherits the timestamp and frequency value of the corresponding data node. S2. Arrange all hash nodes in ascending order of their expiration timestamps to obtain an ordered queue. S3. Traverse the ordered queue from left to right, find multiple adjacent nodes, merge them, and generate parent nodes of the multiple adjacent nodes; S4. Use the generated parent node to replace the corresponding multiple adjacent nodes in the ordered queue, and recursively repeat the merging and replacing process until only one hash node remains in the ordered queue as the root node of the Merkle tree, thereby obtaining the constructed Merkle tree structure. S5. Starting from the root node of the Merkle tree, perform a recursive binary search from top to bottom based on the timestamps of the hash nodes to locate the target node, and record the path from the target node to the root node as the Merkle path. Verify the hash values ​​of the hash nodes of the Merkle path in reverse order from bottom to top, hash the sibling nodes of the Merkle path layer by layer, verify whether the hash value of the root node is consistent, and then verify the integrity and correctness of the target node.

2. The frequency-time dual feature-aware MERKLE tree authentication method according to claim 1, characterized in that: The method for adding a new data node to the Merkle tree includes: Use binary search to locate and delete invalid nodes in the Merkle tree to obtain a Merkle tree that retains valid data; According to the increasing order of the expiration timestamps, the insertion position of the new data node is determined by binary search, and the Merkle tree is rebuilt according to the frequency value.

3. The frequency-time dual feature-aware MERKLE tree authentication method according to claim 1, characterized in that: The recursive binary search in S5 to locate the target node includes: In the process of performing recursive binary search to locate the target node, the nodes with invalid hash values ​​and the subtrees corresponding to the nodes with invalid hash values ​​are pruned to avoid repeated verification of invalid nodes in subsequent queries.

4. The frequency-time dual feature-aware MERKLE tree authentication method according to claim 1, characterized in that: The step of finding multiple adjacent nodes and merging them to generate parent nodes of the multiple adjacent nodes in S3 includes: Find the two adjacent nodes with the smallest sum of frequency values ​​and merge them to generate the parent node of multiple adjacent nodes; Alternatively, the weight of each hash node is calculated based on a dynamic weight calculation formula, and the two adjacent nodes with the smallest sum of weights are found and merged. If there are multiple adjacent node pairs whose sum of weights is less than a preset similarity threshold, the adjacent node pair with the smallest sum of timestamps is selected for merging to generate the parent node of the multiple adjacent nodes. The timestamp of the parent node is the average of the timestamps of the two corresponding adjacent nodes; the frequency value of the parent node is the sum of the frequency values ​​of the two corresponding adjacent nodes; the hash value of the parent node is the sum of the hash values ​​of the two corresponding adjacent nodes calculated by the SM3 algorithm; The dynamic weight calculation formula is as follows (1): （1）。 5. The frequency-time dual feature-aware MERKLE tree authentication method according to claim 1, characterized in that: The step of finding multiple adjacent nodes and merging them to generate a parent node of the multiple adjacent nodes in S3 further includes: A frequency threshold is set according to the size of the data node set, and multiple consecutive adjacent nodes whose sum of frequency values ​​is not greater than the frequency threshold are obtained and merged to generate a parent node of the multiple adjacent nodes; Among them, the timestamp of the parent node is the median of the timestamps of the corresponding multiple adjacent nodes; the frequency value of the parent node is the sum of the frequency values ​​of the corresponding multiple adjacent nodes; the hash value of the parent node is the SM3 hash result of the hash values ​​of multiple adjacent nodes.

6. The frequency-time dual feature-aware MERKLE tree authentication method according to claim 1, characterized in that: The method further includes: automatic cleaning and reconstruction of failed nodes; The automatic cleanup and reconstruction of failed nodes includes: Periodically scan the ordered queue, use the skip table structure to find the hash nodes whose invalid timestamps are earlier than the current time, and mark them as invalid nodes; After deleting the invalid node, if the parent node of the invalid node has only one valid child node left, recursively search for the sibling nodes and perform a merge; The hash nodes except the invalid nodes are rearranged in the ascending order of the invalid timestamps, and the merge operation is re-executed according to the frequency value, and the Merkle tree structure is updated to maintain the validity of the authentication.

7. The frequency-time dual feature-aware MERKLE tree authentication method according to claim 1, characterized in that: The method further comprises: The weight of each Merkle path is calculated based on the cache priority weight calculation formula. The timestamps of the two most recent accesses to each Merkle path are recorded using the LRU-K algorithm, and the access interval is calculated based on the timestamps of the two most recent accesses. When the cache space reaches the upper limit, Merkle paths with low weights and long access intervals are eliminated first. The cache priority weight calculation formula is as follows (2): （2） Where, and It is an adjustable parameter.

8. A MERKLE tree authentication device based on dual frequency and time features, wherein the MERKLE tree authentication device based on dual frequency and time features is used to implement the MERKLE tree authentication method based on dual frequency and time features as claimed in any one of claims 1 to 7, characterized in that: The device comprises: An acquisition module is used to obtain a set of data nodes to be used for Merkle tree construction, and to generate a corresponding hash node for each data node in the set of data nodes using the SM3 algorithm. The pointer of the generated hash node points to the corresponding data node, and the generated hash node inherits the timestamp and frequency value of the corresponding data node; The arrangement module is used to arrange all hash nodes in the increasing order of their expiration timestamps to obtain an ordered queue; A merging module is used to traverse the ordered queue from left to right, find multiple adjacent nodes to merge, and generate parent nodes of the multiple adjacent nodes; The construction module is used to replace the corresponding multiple adjacent nodes in the ordered queue with the generated parent node, and recursively repeat the merging and replacing process until only one hash node remains in the ordered queue as the root node of the Merkle tree, thereby obtaining the constructed Merkle tree structure; The authentication module is used to start from the root node of the Merkle tree, perform recursive binary search from top to bottom based on the timestamps of the hash nodes to locate the target node, and record the path from the target node to the root node as the Merkle path. The hash values ​​of the hash nodes of the Merkle path are verified in reverse order from bottom to top, and the sibling nodes of the Merkle path are hashed layer by layer to verify whether the hash value of the root node is consistent, thereby verifying the integrity and correctness of the target node.

9. A MERKLE tree authentication device, characterized in that: The MERKLE tree authentication device includes: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, which can be called by a processor to execute the method according to any one of claims 1 to 7.

Images