Block chain-based Internet of Things data storage adaptive synchronization method and system

Abstract

The invention discloses an internet of things data storage adaptive synchronization method and system based on a block chain, and belongs to the technical field of internet of things and block chains. According to the method, firstly, sensor information is collected by an Internet of Things edge node and data is cached locally, a block chain consensus node verifies and consensus the received sensor data, then a block chain network issues rewards to the consensus node based on stored data, and a reward score is obtained by weighted summation of influence factors; the weight of the influence factor is dynamically adjusted in the storage process. According to the method, the synchronization window is adaptively adjusted according to the load, the energy consumption and the hardware cost are reduced, a self-adaptive incentive mechanism is adopted in the Internet of Things to encourage nodes to participate in bookkeeping and data verification, and the security and the reliability of the network are improved; the self-adaptive synchronization and verification process of the method reduces the time consumption of the block chain for storing the Internet of Things data, reduces the risk of data loss, and improves the overall storage efficiency.

Description

Technical Field

[0001] The present invention relates to a blockchain-based Internet of Things data storage adaptive synchronization method and system, belonging to the technical field of the Internet of Things and blockchain. Background Art

[0002] The traditional IoT architecture adopts a client-server (CS) centralized communication model, where IoT devices exchange data through a central server. Once the central server fails, all IoT devices that rely on the central server will not be able to operate normally. At the same time, as the number of IoT devices increases, the amount of data that the central server needs to process will also increase dramatically, and the network will face scalability issues, which has led to the need for distributed storage of data. Traditional distributed storage generally consists of multiple distributed nodes with computing and storage capabilities that independently store data and then synchronize it to each node for backup. However, due to the large number of IoT devices, such as the presence of thousands of flow meters, pressure gauges, and smart water meters in smart water systems, the amount of data is huge.

[0003] In the process of synchronous accounting of IoT data, frequent synchronization will cause huge system overhead due to the large number of nodes. In addition, the data is relatively single time series data, and in many cases the time series data will not change much. The purpose of IoT data storage is to obtain accurate and efficient monitoring and prediction models based on the data. Therefore, a more efficient distributed storage and synchronization method can be considered based on the characteristics of the IoT system. Summary of the invention

[0004] Technical problem to be solved by the invention

[0005] In view of the problems of huge data volume and centralized storage in existing Internet of Things systems, the present invention proposes an Internet of Things data storage adaptive synchronization method and system based on blockchain.

[0006] Technical Solution

[0007] In order to achieve the above object, the technical solution provided by the present invention is:

[0008] The method for adaptive synchronization of IoT data storage based on blockchain includes the following steps:

[0009] Step 1: The IoT edge node collects sensor information and caches the data locally;

[0010] Step 2: The blockchain consensus node verifies and reaches consensus on the received sensor data. This process includes: verifying the integrity and source of the sensor data, checking whether the data complies with predefined rules, and using the blockchain consensus algorithm to reach consensus on the data, so that multiple nodes in the system can reach a consistent state on the same data.

[0011] Step 3: The blockchain network awards rewards to consensus nodes based on stored data. The reward score is obtained by weighted sum of data storage duration factor, data access frequency factor, data integrity factor, node storage capacity factor, node network bandwidth factor, data compression efficiency factor, node geographic location factor, and node reputation factor.

[0012] Step 4: During the storage process, the system dynamically adjusts the factor weights described in step 3.

[0013] Furthermore, the data stored in step 2 is divided into important data and general data based on the frequency of data changes. When consensus is reached on important data, all participating nodes in the blockchain network verify and confirm the data. When consensus is reached on general data, some blockchain nodes are randomly sampled to verify and confirm the data.

[0014] Furthermore, the system defines a time window. In step 2, when the time window is not reached, the edge node temporarily stores the collected data; when the time window is reached, the edge node pre-processes the collected data.

[0015] Furthermore, the length of the time window is determined by the blockchain system load. When the system load is light, a long time window is defined, and when the system load is heavy, a short time window is defined. The blockchain system load includes one or more of the parameters of CPU usage, memory usage, network bandwidth utilization, transaction processing speed, node response time, and length of the queue to be processed.

[0016] Furthermore, the data preprocessing process includes: deleting abnormal values ​​and obviously erroneous data, and compressing the data using a coding algorithm.

[0017] Furthermore, in step 4, the factor weights are dynamically adjusted based on the system load conditions, and the adjustment method is:

[0018] w i (t+1)=w i (t)+α·ΔN i (t)+β·ΔS i (t)

[0019] where w i (t) is the weight at the current moment, w i (t+1) is the weight at the next moment, α and β are adjustment coefficients, ΔN i (t) is the change in node performance index associated with this factor, ΔS i (t) is the change in system performance index associated with this factor, ΔN i (t), ΔS i The calculation method of (t) is:

[0020]

[0021]

[0022] Where N i (t),S i (t) is the current time performance index, N i (t-1),S i (t-1) is the performance index of the previous time.

[0023] Furthermore, the above node performance index changes ΔN i (t) includes changes in node transaction processing speed, user request response time, and data storage time; system performance indicator change ΔS i (t) Includes changes in system network delay and network bandwidth utilization.

[0024] Furthermore, in step 4, the factor weights are dynamically adjusted based on the characteristics of the stored data: when the amount of stored data is large, the weight of the data compression efficiency factor is increased, and for key business data, the weight of the data importance factor is increased.

[0025] Furthermore, in step 4, the factor weights are dynamically adjusted based on the storage service requirements: the corresponding weight factors are adjusted based on different types of storage service requirements of users.

[0026] A system for executing the above-mentioned blockchain-based IoT data storage adaptive synchronization method, the system consists of a number of edge nodes, an IoT control platform and a blockchain network, wherein the edge nodes are used to store neighboring sensor data; the consensus nodes in the blockchain network execute the method consensus and synchronization process, and the consensus nodes can be edge nodes of the sensor network or independent nodes located in the blockchain network; the IoT control platform is connected to each edge node and the blockchain network to realize data interaction between the blockchain network and the IoT.

[0027] Beneficial Effects

[0028] The present invention distributes data storage and processing to multiple nodes in a decentralized manner. The method adaptively adjusts the synchronization window according to the load, reduces the overall computing burden, and thus reduces energy consumption and hardware costs. In addition, the adaptive consensus mechanism of the blockchain can effectively reduce redundant calculations during synchronization and ensure data consistency and security.

[0029] The present invention adopts an adaptive incentive mechanism in the Internet of Things to encourage nodes to participate in accounting and data verification, which not only improves the security and reliability of the network, but also promotes collaboration and data sharing between Internet of Things devices with different capabilities, forming a virtuous circle;

[0030] The adaptive synchronization and verification of the method of the present invention reduces the time consumption of blockchain storage of IoT data. The decentralized nature of blockchain makes data storage no longer dependent on a single storage device, reduces the risk of data loss, and improves the overall storage efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a block diagram of the steps of the blockchain-based IoT data storage adaptive synchronization method of the present invention;

[0032] Figure 2 This is a block diagram of the Internet of Things data storage adaptive synchronization system based on blockchain in the present invention. DETAILED DESCRIPTION

[0033] In order to further understand the content of the present invention, the present invention is described in detail in conjunction with the accompanying drawings and specific implementation methods.

[0034] like Figure 1 As shown, the blockchain-based IoT data storage adaptive synchronization method includes the following steps:

[0035] Step 1: The system edge node collects nearby sensor information and caches the data locally.

[0036] Step 2: The blockchain consensus node performs the adaptive synchronization process, which mainly includes the following processes:

[0037] The sensor data received by each node is verified and consensus is reached. The process includes: verifying the integrity and source of the sensor data, checking whether the data complies with predefined rules, and using the blockchain consensus algorithm to ensure that multiple nodes in the system reach a consensus on the same data.

[0038] The time window is defined in the blockchain network. The length of the time window is determined by the blockchain system load. When the system load is light, a long time window is defined, and when the system load is heavy, a short time window is defined. When the time window is not reached, the edge node temporarily stores the collected data; when the time window is reached, the edge node pre-processes the data to be stored, including deleting outliers and obviously erroneous data, and compressing the data using an encoding algorithm.

[0039] The time window is set according to data characteristics and system requirements. The window size needs to strike a balance between real-time performance and efficiency. In this embodiment, the long time window is set to 10 minutes, the medium time window is set to 5 minutes, and the short time window is set to 1 minute: when the system load is light, a long time window (10 minutes) is set; when the system load is medium, a medium time window (5 minutes) is set; when the system load is heavy, a short time window (1 minute) is set.

[0040] The blockchain system load is defined by one or more of the following parameters: CPU usage, memory usage, network bandwidth utilization, transaction processing speed, node response time, and length of the pending queue. The system complexity is divided into three levels: low, medium, and high. Table 1 shows the corresponding relationship between the above parameters and the system load.

[0041] Table 1 Blockchain system load classification and corresponding parameters

[0042]

[0043] The stored data is classified into important data and general data. The data classification standard is determined based on the frequency of data changes. Data that changes frequently and requires high-frequency updates is defined as important data, and data with a low frequency of change is defined as general data. When reaching consensus on important data, the traditional consensus algorithm is used. All participating blockchain nodes in the consensus process verify and confirm the data. When reaching consensus on general data, the signature or hash verification algorithm is used. During the consensus process, some nodes are randomly sampled to verify and confirm the data.

[0044] Step 3: The blockchain network awards rewards to consensus nodes based on stored data. Node i is rewarded with a score S. i Defined as

[0045]

[0046] where s i,j is the score of a certain impact factor j, w i,j is the weight of the impact factor, satisfying

[0047] In this embodiment, the impact factor s i,j Including: data storage duration factor, data access frequency factor, data integrity factor, node storage capacity factor, node network bandwidth factor, data compression efficiency factor, node geographic location factor, and node reputation factor.

[0048] Step 4: During the storage process, the system dynamically adjusts the weights of the factors described in step 3 by:

[0049] w i (t+1)=w i (t)+α·ΔN i (t)+β·ΔS i (t)

[0050] where w i (t) is the weight at the current moment, w i (t+1) is the weight at the next moment, α and β are adjustment coefficients. In this embodiment, α=0.5, β=0.5, ΔN i(t) is the change in node performance index associated with the impact factor, ΔS i (t) is the change in system performance index associated with the impact factor, ΔN i (t), ΔS i The calculation method of (t) is:

[0051]

[0052]

[0053] Where N i (t),S i (t) is the current time performance index, N i (t-1),S i (t-1) is the performance index of the previous time.

[0054] In this embodiment, the node performance indicators include node transaction processing speed, user request response time, data storage time, etc., and the system performance indicators include system network delay, network bandwidth utilization, etc. Table 2 shows the corresponding relationship between the factor weight adjustment and the considered node performance indicators and system performance indicators.

[0055] Table 2 Relationship between factor weight adjustment and performance indicators

[0056]

[0057] In addition to adjusting the factor weights based on the above performance indicators, you can also adjust the factor weights based on the storage data characteristics and different storage service requirements, specifically:

[0058] Dynamically adjust the factor weights based on the characteristics of the stored data: when the amount of stored data is large, increase the weight of the data compression efficiency factor; for key business data, increase the weight of the data importance factor;

[0059] Dynamically adjust factor weights based on storage service needs: Adjust corresponding weight factors based on different types of storage service needs of users.

[0060] In addition to the above-mentioned adaptive adjustment method, machine learning algorithms can also be combined to learn factor weights, adjustment coefficients α, β, etc. based on historical data. A feedback mechanism can be introduced in the machine learning stage to fine-tune the above parameters based on the performance of the adjusted system.

[0061] The method of the present invention can be run on a blockchain network with or without IPFS (InterPlanetary File System). When IPFS is used, the IPFS system saves the data, and the content identifier of the IPFS file is recorded on the blockchain. The nodes verify and reach consensus on the content identifier corresponding to the received data. A complete consensus process is used for the content identifier corresponding to important data, and a simplified consensus process is used for the content identifier corresponding to general data.

[0062] Figure 2 The system composition diagram for executing the method of the present invention is shown in the figure. The system consists of several edge nodes for storing sensor data, an Internet of Things control platform, and a blockchain network including consensus nodes. The edge nodes are used to store adjacent sensor data, and the blockchain network consensus nodes execute the blockchain consensus and synchronization process. In this embodiment, the blockchain consensus node can be an edge node of the sensor network or an independent node in the blockchain network. The Internet of Things control platform is connected to each edge node and the blockchain network to realize data interaction between the blockchain network and the Internet of Things.

[0063] The present invention and its embodiments are described schematically above, and the description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. Therefore, if a person skilled in the art is inspired by it and designs a structural method and an embodiment similar to the technical solution without creativity without departing from the purpose of the invention, they shall all fall within the protection scope of the present invention.

Claims

1. An adaptive synchronization method for IoT data storage based on blockchain, characterized in that: The following steps are involved: Step S1, the IoT edge node collects sensor information and caches the data locally; Step S2, the blockchain consensus node verifies and reaches consensus on the received sensor data. The process includes: verifying the integrity and source of the sensor data, checking whether the data complies with predefined rules, and using the blockchain consensus algorithm to reach consensus on the data, so that multiple nodes in the system can reach a consistent state on the same data; Step S3, the blockchain network awards rewards to consensus nodes based on stored data, and the reward score is obtained by weighted summation of data storage duration factor, data access frequency factor, data integrity factor, node storage capacity factor, node network bandwidth factor, data compression efficiency factor, node geographic location factor, and node reputation factor; Step S4: During the storage process, the system dynamically adjusts the factor weights described in step S3.

2. The method for adaptive synchronization of IoT data storage based on blockchain as claimed in claim 1, characterized in that: The sensor data stored in step S2 is divided into important data and general data based on the frequency of data changes. When consensus is reached on important data, all participating nodes in the blockchain network verify and confirm the data. When consensus is reached on general data, some blockchain nodes are randomly sampled to verify and confirm the data.

3. The method for adaptive synchronization of IoT data storage based on blockchain as claimed in claim 1, characterized in that: The system defines a time window. In step 2, when the time window is not reached, the edge node temporarily stores the collected data; when the time window is reached, the edge node pre-processes the collected data.

4. The method for adaptive synchronization of IoT data storage based on blockchain as claimed in claim 3, characterized in that: The length of the time window is determined by the blockchain system load. When the system load is light, a long time window is defined, and when the system load is heavy, a short time window is defined. The blockchain system load includes one or more of the parameters of CPU usage, memory usage, network bandwidth utilization, transaction processing speed, node response time, and length of the queue to be processed.

5. The blockchain-based IoT data storage adaptive synchronization method as claimed in claim 3, characterized in that: The data preprocessing process includes: deleting abnormal values ​​and obviously erroneous data, and compressing the data using a coding algorithm.

6. The blockchain-based IoT data storage adaptive synchronization method according to claim 1, characterized in that: In step S4, the factor weights are dynamically adjusted based on the system load conditions, and the adjustment method is: w i (t+1)=w i (t)+α·ΔN i (t)+β·ΔS i (t) where w i (t) is the weight at the current moment, w i (t+1) is the weight at the next moment, α and β are adjustment coefficients, ΔN i (t) is the change in node performance index associated with this factor, ΔS i (t) is the change in system performance index associated with this factor, ΔN i (t), ΔS i The calculation method of (t) is: Where N i (t),S i (t) is the current time performance index, N i (t-1),S i (t-1) is the performance index of the previous time.

7. The method for adaptive synchronization of IoT data storage based on blockchain as claimed in claim 6, characterized in that: The node performance indicator changes ΔN i (t) includes changes in node transaction processing speed, user request response time, and data storage time; system performance indicator change ΔS i (t) Includes changes in system network delay and network bandwidth utilization.

8. The method for adaptive synchronization of IoT data storage based on blockchain as claimed in claim 6, characterized in that: In step S4, the factor weights are dynamically adjusted based on the characteristics of the stored data: when the amount of stored data is large, the weight of the data compression efficiency factor is increased, and for key business data, the weight of the data importance factor is increased.

9. The method for adaptive synchronization of IoT data storage based on blockchain as claimed in claim 6, characterized in that: In step S4, the factor weights are dynamically adjusted based on the storage service requirements: the corresponding weight factors are adjusted based on the different types of storage service requirements of the users.

10. A system for executing the blockchain-based IoT data storage adaptive synchronization method as described in any one of claims 1 to 9, characterized in that: The system consists of several edge nodes, an Internet of Things control platform and a blockchain network. The edge nodes are used to store neighboring sensor data. The consensus nodes in the blockchain network execute method consensus and synchronization processes. The consensus nodes can be edge nodes of the sensor network or independent nodes in the blockchain network. The Internet of Things control platform is connected to each edge node and the blockchain network to realize data interaction between the blockchain network and the Internet of Things.

Images