Block chain data rapid retrieval and storage method based on online learning machine

Abstract

The invention relates to a block chain data rapid retrieval and storage method based on an online learning machine, and belongs to the field of distributed computing. The method is divided into a hot spot block storage part and a hot spot block updating part. Firstly, in a hotspot block storage part, a hotspot block storage strategy architecture based on an online sequence extreme learning machine OS-ELM is provided, a new block is verified by a verification node and then input into a feature extraction module to extract feature values, then a classification module judges whether the new block is a hotspot block, and a storage scheme is determined. And secondly, in the hot block updating part, after a set time, the verification node recalculates the characteristic value, classifies the blocks according to the newest online learning machine model, deletes non-hot blocks on the storage node, and ensures the continuity of efficient query.

Description

Technical Field

[0001] The present invention belongs to the field of distributed computing and relates to a blockchain data fast retrieval and storage method suitable for the Internet of Things, in particular to a blockchain data fast retrieval and storage method for the Internet of Things when the resources of the Internet of Things devices are limited and the information is fragmented, and is used to improve the query performance and storage efficiency of the blockchain database in the Internet of Things. Background Art

[0002] In recent years, the widespread application of supply chain technology has enabled companies to gain more profits by optimizing resource allocation and improving efficiency. To ensure the normal operation of the supply chain system, collaboration between a large number of entities (such as factories, retailers, and logistics companies) is essential. During the operation of each node in the supply chain, a large amount of information is generated, which needs to be quickly transmitted between entities so that these entities can obtain relevant data in a timely manner and make effective decisions. Therefore, how to efficiently store and quickly query massive supply chain data is an important technical challenge for achieving real-time monitoring and management of the supply chain. The combination of blockchain technology and IoT technology provides a decentralized and secure data storage and management solution for the supply chain system. Blockchain can ensure the integrity and authenticity of data in the supply chain and prevent data from being tampered with. However, blockchain requires that each node must save a complete copy of the entire blockchain data. As the amount of data increases, the storage space demand also increases significantly. For IoT devices such as cameras, their computing and storage capabilities are limited and cannot effectively cope with this data growth, resulting in the decentralization and security of the blockchain system being weakened. Therefore, in IoT blockchain applications, developing an efficient data storage method is an urgent problem to be solved.

[0003] In order to solve the data storage problem in the supply chain system, sharding technology was introduced into the blockchain. Sharding technology can distribute data to multiple nodes to reduce the storage pressure of each node. The current sharding methods are mainly divided into two categories: one is that multiple IoT nodes jointly undertake the work of a complete node, and the other is to shard the block and store it only on some nodes instead of the entire network system. Both types of sharding methods can reduce the storage burden of a single node, while allowing resource-constrained IoT nodes to participate in the consensus process of the blockchain network, thereby enhancing the decentralization and security of the system.

[0004] However, the sharding-based blockchain supply chain system still has shortcomings in terms of data query response time. There are two main reasons for this: 1. The amount of supply chain data is huge; 2. The number of participating nodes is large. Since the data is stored in shards, the query target data may not be in the local node, and requests need to be made to other nodes, which increases the query latency. If the system cannot query the latest data of the supply chain in real time, it may cause significant losses. For example, in some scenarios in the supply chain, delays in obtaining key data may affect decision-making, causing economic losses or security risks.

[0005] In summary, how to adopt efficient data storage and query methods, especially to store and manage massive data in the supply chain, has become the focus and difficulty of distributed computing research. This paper aims to propose a blockchain data fast retrieval and storage method based on online learning machine to solve the query efficiency problem in the supply chain system. Summary of the invention

[0006] The purpose of the present invention is to provide a blockchain data fast retrieval and storage method based on an online learning machine to solve the technical problem of low query efficiency in the prior art.

[0007] In order to achieve the above purpose, the technical solution adopted by the invention is: a blockchain data fast retrieval and storage method based on an online learning machine, including two parts: hotspot block storage and hotspot block update;

[0008] The method for storing the hotspot block is:

[0009] A1) Feature extraction module: reads the newly generated verification block, the verification node determines the system's storage requirement SR for the block, and calculates the fixed feature FF, hidden feature HP, historical activity HISP and node performance NP of the new block;

[0010] In the feature extraction module, the verification node reads four knowledge points of the newly generated verification block: data in the block, historical query records of the block, node performance, and system security analysis; based on this knowledge, it calculates five features of a node's block: fixed features, historical activity, hidden features, node performance, and storage requirements;

[0011] A1.1: Fixed feature FF

[0012] First, define the fixed characteristics of the block. Once the fixed characteristic value of the block is calculated, it will not change. When a block contains N transactions, the calculation method of FF is as follows

[0013]

[0014] N is the number of transactions in the evaluation block. When the target node participates in the i-th transaction, Nti is 1; when the target node does not participate, Nti is 0; Nv is the maximum value of transactions in all blocks; Nvi is the transaction value of the target node participating in the i-th transaction of the block;

[0015] A1.2: Historical Activity HISP

[0016] When the data in a data block is read multiple times by the target node recently, the data block is regarded as the hot block of the node. The node saves the hot block locally. The calculation method of HISP is as follows

[0017]

[0018] Set a fixed update time and record the number of times the node accesses the data block in each fixed time. After the block is generated, M time periods have passed, Tk is the kth time period; Nk is the number of times the node accesses the block in the Tk time period;

[0019] A1.3: Hidden Features HP

[0020] Although the fixed feature value of the current block is not high, the transaction participants in the block have many other transactions associated with this node. In the future, this node will read the data in the data block to verify the authenticity of the new transaction. The calculation method of HP is as follows

[0021]

[0022] Set a fixed update time. The node generates J transactions within a fixed time. If the initiator of the jth transaction is also a participant of the evaluated block, then η = 1, otherwise η = 0; at the same time, if the recipient of the jth transaction is also a participant of the evaluated block, then ξ = 1, otherwise ξ = 0. Vj is the value of the jth transaction;

[0023] A1.4: Node Performance NP

[0024] Node performance includes storage performance and processing performance. The storage performance of a node is reflected in whether there is enough remaining storage space to save new block data. The processing performance of a node is reflected in whether it can quickly complete system consensus and respond to query requests from other nodes. The calculation method of NP is as follows

[0025]

[0026] Sa is the currently available storage space on the node; Sb is the average space occupied by the block; Nc is the number of times the node completes consensus; Nq is the number of times the node responds to query requests in a timely manner; γ is the coefficient for evaluating the processing performance of the node;

[0027] A1.5: Storage Requirements SR

[0028] In a sharding-based blockchain system, block data is only stored on certain nodes. In order to ensure the security of block data, the storage requirements of the blockchain are unified by the sharded blockchain system in which it is located.

[0029] A2) Based on the latest OS-ELM online learning machine classifier, the verification node determines whether the new block is a hot block of the storage node based on the features;

[0030] In the classifier module, the block features are used as the input of the latest classifier based on OS-ELM, and the output result is the hotspot block or non-hotspot block of the node. The node stores the hotspot block locally;

[0031] The training process of the OS-ELM classifier is as follows: based on a small amount of training data, an original classifier model is obtained. Then, as the blockchain system runs, new blocks are constantly added, which will generate new training data. Then, the OS-ELM method will train a new training set based on the existing classifier model to obtain the latest FF, HISP, HP, NP and SR.

[0032] A3) Determine the storage plan, read the classification results output by the classifier module, and if it is determined to be a hot block, add the new block to the storage node blockchain; otherwise, abandon the storage of the new block and do not add it to the storage node blockchain;

[0033] The method for updating the hotspot blocks is as follows: after a specified period of time, the verification node recalculates the characteristic values ​​of each block on the storage node blockchain in turn, classifies each block according to the latest online learning machine model, and deletes non-hotspot blocks on the storage node.

[0034] B1: Read the parameters of each block

[0035] As the blockchain system runs, nodes will generate new transmission data and respond to some new query requests. The hot blocks stored in the nodes may gradually become non-hot blocks. Therefore, the blockchain system based on ElasticChain needs to re-evaluate the characteristics of hot blocks in the storage nodes after a period of time. First, the verification node will read the parameters in each storage node, including: nk, Tk, M, Vj, η, ξ, Sa, Sb, Nc, Nq, and then the verification node will evaluate the storage requirements of each block.

[0036] B2: Recalculate eigenvalues

[0037] Based on the latest data, the verification node recalculates the feature values ​​of the block: HISP, HP, NP and SR are updated, while the fixed feature FF determined when the block is generated will not change. The feature value calculation method is the same as the hotspot block storage part;

[0038] B3: Classify and update storage node blockchain

[0039] Based on the latest OS-ELM classifier, the verification node divides the blocks into hotspot blocks and non-hotspot blocks according to the feature values ​​of FF, HISP, HP, NP, and SR. If the result is a hotspot block, the block will be retained in the storage node; if the result is a non-hotspot block, the storage node will delete these blocks.

[0040] The beneficial effects created by the present invention are as follows: the present invention relates to a method for rapid retrieval and storage of blockchain data based on an online learning machine, and belongs to the field of distributed computing. The method is divided into two parts: hotspot block storage and hotspot block update. First, in the hotspot block storage part, a hotspot block storage strategy architecture based on the online sequence extreme learning machine OS-ELM is proposed. After the new block is verified by the verification node, it is input into the feature extraction module to extract the feature value, and then the classification module determines whether it is a hotspot block and determines the storage plan. Secondly, in the hotspot block update part, after a specified time, the verification node recalculates the feature value, classifies the block according to the latest online learning machine model, deletes the non-hotspot blocks on the storage node, and ensures the continuity of efficient query. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a module diagram of an implementation mode of the present invention. DETAILED DESCRIPTION

[0042] The embodiments of the present invention will be further described below in conjunction with the accompanying drawings.

[0043] During the operation of each node in the supply chain, a large amount of information is generated and spread rapidly. Entities need to ensure that they can grasp the information and respond in time to achieve effective cooperation. Based on the advantages of supply chain decentralization and deep learning technology, the present invention makes full use of the information of each node in the supply chain and realizes a blockchain data fast retrieval and storage method based on online learning machine. Next, taking the chilled transportation supply chain of Sino-Ocean Group as an example, the process of adding a new block in the blockchain is demonstrated. For example, Figure 1 shown.

[0044] The main modules in the method include:

[0045] Blockchain module based on sharding: divided into user nodes, storage nodes, and verification nodes. Used for efficient storage and verification of supply chain information.

[0046] Feature extraction module: In the feature extraction module, the verification node will analyze the attributes of the block and obtain the knowledge of the block. Then, the verification node calculates the feature value of each block of the storage node based on the knowledge.

[0047] Classifier module: The block features are used as input to the state-of-the-art classifier based on OS-ELM, and the output result is the hotspot block or non-hotspot block of the node.

[0048] Here are the steps:

[0049] Step 1: When a batch of frozen goods is loaded from Sino-Ocean Group’s warehouse to a transport company’s vehicle, the IoT device will automatically generate and upload a block of the batch of goods. After verification by the verification node, the verification block is obtained and input into the feature extraction module. In the feature extraction module: First, the fixed feature value (FF) of the block is evaluated based on the quantity, value, and correlation between the goods in the block and each node. Then, based on the potential transaction relationship between the transaction parties involved in the block and other nodes, the possibility of the block being accessed in the future is predicted, and the hidden population (HP) is calculated. Then, based on the available storage space and computing power of the storage node, the node performance (NP) is calculated. Finally, based on the security requirements of the data during cold chain transportation, the storage requirement (SR) is calculated.

[0050]

[0051]

[0052]

[0053] Step 2: OS-ELM Online Learning Machine Classifier Module

[0054] Based on the result of feature extraction, the OS-ELM online learning machine classifier will determine whether the new block is a hotspot block. In the classifier module, the online learning machine model generated by the previous block classification training is stored. Based on the feature value and the latest online learning machine model, the classifier outputs the type of the new verification block, updates the OS-ELM parameters, and further optimizes the OS-ELM online learning machine classifier to prepare for this rapid classification.

[0055] Step 3: Determine the storage solution

[0056] If the block is identified as a hotspot block, such as a transportation company frequently accessing its data to track the location and status of the goods, the block will be preferentially stored in the local nodes of the transportation company, cold storage and retailer to speed up data query. For example, when a retailer queries the transportation status of the goods, since the block is a hotspot block, the data can be quickly obtained directly from the local node. If the block is identified as a non-hotspot block, such as the data of the block is not frequently queried, it will not be stored in the local node, but distributed and stored in a wider blockchain network to reduce local storage pressure.

[0057] Hotspot block storage part:

[0058] Step 1: Read the parameters of each block

[0059] As the cold chain transportation process continues, the node generates new transaction data. When the goods arrive at different storage and transportation nodes, the system will regularly evaluate the heat status of the blocks in the storage nodes. The verification node reads the following latest parameters: the number of node accesses to the block Nk, time period Tk, storage space Sa, and average occupied space Sb. Then, the verification node evaluates the storage requirements of each block.

[0060] Step 2: Recalculate the eigenvalues

[0061] According to the verification block, the verification node recalculates the feature value. First, the historical activity HISP is updated by counting the number of times each node has visited the block in the recent period of time. Nodes that frequently query data such as item numbers and locations will increase the popularity of the block. Next, the system updates the hidden feature HP and analyzes whether the transaction parties involved in the block have potential connections with other nodes. If the node may query this data in the future, the HP value will increase. Then, the system checks the node performance NP and calculates the current storage space and processing power of the node. If the storage space of a node becomes insufficient or the ability to respond to queries decreases, the NP value will decrease, indicating that the storage pressure of the node has increased. Finally, the system updates the storage requirement SR, and decides whether multiple nodes are needed to retain data copies to ensure the security of critical data based on the security and redundancy requirements of the cargo data. In addition, the fixed feature FF will not change and does not need to be updated again.

[0062] Step 3: Classify and update the storage node blockchain

[0063] Based on the latest OS-ELM classifier, the verification node divides the blocks into hotspot blocks and non-hotspot blocks according to the feature values ​​of FF, HISP, HP, NP, and SR, and deletes the non-hotspot blocks. For example, if the nodes of the transportation company or cold storage still frequently access the data of the block, such as querying the status of the goods, the system will continue to mark the block as a hotspot block and retain it in local storage. If a batch of goods has arrived at the destination and been consumed, and its block data is no longer frequently accessed, the system will mark the block as a non-hotspot block and delete it from the local node to free up storage space for storing new blocks.

Claims

1. A blockchain data fast retrieval and storage method based on an online learning machine, characterized in that: It includes two parts: hotspot block storage and hotspot block update; The method for storing the hotspot block is: A1) Feature extraction module: reads the newly generated verification block, the verification node determines the system's storage requirement SR for the block, and calculates the fixed feature FF, hidden feature HP, historical activity HISP and node performance NP of the new block; A2) Based on the latest OS-ELM online learning machine classifier, the verification node determines whether the new block is a hot block of the storage node based on the features; A3) Determine the storage plan, read the classification results output by the classifier module, and if it is determined to be a hot block, add the new block to the storage node blockchain; otherwise, abandon the storage of the new block and do not add it to the storage node blockchain; The method for updating the hotspot blocks is as follows: after a specified period of time, the verification node recalculates the characteristic values ​​of each block on the storage node blockchain in turn, classifies each block according to the latest online learning machine model, and deletes non-hotspot blocks on the storage node.

2. According to a blockchain data rapid retrieval and storage method based on an online learning machine according to claim 1, it is characterized in that: In the above A1), the specific method is: in the feature extraction module, the verification node reads four knowledge points of the newly generated verification block: data in the block, historical query records of the block, node performance and system security analysis; Based on this knowledge, five features of a node’s block are calculated: fixed features, historical activity, hidden features, node performance, and storage requirements; A1.1: Fixed feature FF First, define the fixed characteristics of the block. Once the fixed characteristic value of the block is calculated, it will not change. When a block contains N transactions, the calculation method of FF is as follows N is the number of transactions in the evaluation block. When the target node participates in the i-th transaction, Nti is 1; when the target node does not participate, Nti is 0; Nv is the maximum value of transactions in all blocks; Nvi is the transaction value of the target node participating in the i-th transaction of the block; A1.2: Historical Activity HISP When the data in a data block is read multiple times by the target node recently, the data block is regarded as the hot block of the node. The node saves the hot block locally. The calculation method of HISP is as follows Set a fixed update time and record the number of times the node accesses the data block in each fixed time. After the block is generated, M time periods have passed, Tk is the kth time period; Nk is the number of times the node accesses the block in the Tk time period; A1.3: Hidden Features HP Although the fixed feature value of the current block is not high, the transaction participants in the block have many other transactions associated with this node. In the future, this node will read the data in the data block to verify the authenticity of the new transaction. The calculation method of HP is as follows Set a fixed update time. The node generates J transactions within a fixed time. If the initiator of the jth transaction is also a participant of the evaluated block, then η = 1, otherwise η = 0; at the same time, if the recipient of the jth transaction is also a participant of the evaluated block, then ξ = 1, otherwise ξ = 0. Vj is the value of the jth transaction; A1.4: Node Performance NP Node performance includes storage performance and processing performance. The storage performance of a node is reflected in whether there is enough remaining storage space to save new block data. The processing performance of a node is reflected in whether it can quickly complete system consensus and respond to query requests from other nodes. The calculation method of NP is as follows Sa is the currently available storage space on the node; Sb is the average space occupied by the block; Nc is the number of times the node completes consensus; Nq is the number of times the node responds to query requests in a timely manner; γ is the coefficient for evaluating the processing performance of the node; A1.5: Storage Requirements SR In a sharding-based blockchain system, block data is only stored on certain nodes. In order to ensure the security of block data, the storage requirements of the blockchain are unified by the sharded blockchain system in which it is located.

3. According to a blockchain data rapid retrieval and storage method based on an online learning machine according to claim 1, it is characterized in that: In the above A2), the specific method is as follows: in the classifier module, the block feature is used as the input of the latest classifier based on OS-ELM, and the output result is the hotspot block or non-hotspot block of the node, and the node stores the hotspot block locally; The training process of the OS-ELM classifier is as follows: based on a small amount of training data, an original classifier model is obtained. Then, as the blockchain system runs, new blocks are constantly added, which will generate new training data. Then, the OS-ELM method will train a new training set based on the existing classifier model to obtain the latest FF, HISP, HP, NP and SR.

4. According to a blockchain data rapid retrieval and storage method based on an online learning machine according to claim 1, it is characterized in that: The specific method of the hotspot block update part is: B1: Read the parameters of each block As the blockchain system runs, nodes will generate new transmission data and respond to some new query requests. The hot blocks stored in the nodes may gradually become non-hot blocks. Therefore, the blockchain system based on ElasticChain needs to re-evaluate the characteristics of hot blocks in the storage nodes after a period of time. First, the verification node will read the parameters in each storage node, including: nk, Tk, M, Vj, η, ξ, Sa, Sb, Nc, Nq, and then the verification node will evaluate the storage requirements of each block. B2: Recalculate eigenvalues Based on the latest data, the verification node recalculates the feature values ​​of the block: HISP, HP, NP and SR are updated, while the fixed feature FF determined when the block is generated will not change. The feature value calculation method is the same as the hotspot block storage part; B3: Classify and update storage node blockchain Based on the latest OS-ELM classifier, the verification node divides the blocks into hotspot blocks and non-hotspot blocks according to the feature values ​​of FF, HISP, HP, NP, and SR. If the result is a hotspot block, the block will be retained in the storage node; if the result is a non-hotspot block, the storage node will delete these blocks.

Images