Data storage methods, apparatus, devices and computer-readable storage media
By leveraging the interaction between user nodes and target storage nodes, and using gateway nodes as data relays, the problem of limited storage capacity in the Ethereum blockchain is solved, achieving both large-capacity data storage and data security.
Patent Information
- Application Number
- CN202311694676.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-11
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2043-12-11
AI Technical Summary
The Ethereum blockchain has limited storage capacity, which cannot accommodate large amounts of encrypted data. This limits the storage capacity of ENS gateways that rely on the Ethereum blockchain, making it impossible to run a large number of centralized applications.
Through the interaction between user nodes, gateway nodes, and target storage nodes, the gateway node is used as a data relay node to obtain a list of candidate storage nodes, determine the node selection result, and send the data to be stored to the target storage node to achieve large-capacity data storage.
It enables the storage of unlimited large amounts of data, improves the efficiency and reliability of data storage, establishes a trust mechanism for storage services, and protects the data security and rights of user nodes.
Smart Images

Figure CN117725041B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of security technology, and in particular to data storage methods, apparatus, devices, and computer-readable storage media. Background Technology
[0002] Blockchain storage is a decentralized storage system based on blockchain technology, representing an effective combination of blockchain and storage systems.
[0003] In related technologies, the Ethereum Name Service (ENS) gateway is typically used for data storage. The ENS gateway is a scalable, distributed, and open naming system based on the Ethereum blockchain, primarily serving human-readable mapping names. The basic principle is as follows: register an ENS domain name on Ethereum through the ENS gateway; prepare the data to be stored, such as files, text, and images; create an ENS resolver to resolve the ENS domain name into the address where the data is stored; use the interface provided by the ENS gateway to store the data on the blockchain; associate the data's storage address with the ENS domain name, so that others can access and use the stored data by accessing the ENS domain name.
[0004] However, the Ethereum blockchain has a small storage capacity that cannot accommodate large amounts of encrypted data. This limits the storage capacity of ENS gateways that rely on the Ethereum blockchain, making it impossible to run a large number of centralized applications on ENS gateways. Summary of the Invention
[0005] The main objective of this invention is to provide a data storage method, apparatus, device, and computer-readable storage medium, aiming to solve the technical problem of limited data storage capacity.
[0006] To achieve the above objectives, the present invention provides a data storage method applied to a user node, the data storage method comprising the following steps:
[0007] Send a storage node list request to the gateway node;
[0008] Receive the candidate storage node list sent by the gateway node based on the storage node list request;
[0009] Output the candidate storage node list and determine the node selection result based on the received selection action;
[0010] Send the node selection result and the data to be stored to the gateway node.
[0011] Optionally, the step of sending the node selection result and the data to be stored to the gateway node includes:
[0012] Send a first transaction request to the gateway node for the node selection result;
[0013] Receive the first transaction information sent by the gateway node based on the first transaction request;
[0014] Based on the first transaction information, the first transaction certificate and the data to be stored are sent to the gateway node.
[0015] To achieve the above objectives, the present invention provides a data storage method applied to a gateway node, the data storage method comprising the following steps:
[0016] Upon receiving a storage node list request from a user node, a candidate storage node list is sent to the user node.
[0017] Receive the node selection result and data to be stored sent by the user node, and generate the encrypted storage order proposal based on the data to be stored;
[0018] The encrypted storage order proposal and the data to be stored are sent to the target storage node corresponding to the node selection result.
[0019] Optionally, the step of sending a candidate storage node list to the user node upon receiving the storage node list request includes:
[0020] Upon receiving the storage node list request sent by the user node, an access request is sent to the blockchain node;
[0021] Receive the list of candidate storage nodes from the blockchain node in response to the access request;
[0022] Send a list of candidate storage nodes to the user node.
[0023] Optionally, the step of receiving the node selection result and the data to be stored sent by the user node, and generating the encrypted storage order proposal based on the data to be stored includes:
[0024] After receiving the first transaction request sent by the user node, obtain the attribute information of each storage node corresponding to the node selection result;
[0025] The first transaction information is sent to the user node according to the attribute information corresponding to the storage node;
[0026] Receive the first transaction certificate and the data to be stored sent by the user node, and send the first transaction certificate to the settlement state machine module;
[0027] Upon receiving confirmation from the settlement state machine module, the encrypted storage order proposal is generated based on the data to be stored.
[0028] Optionally, the data to be stored includes a data tree, and the step of generating the encrypted storage order proposal based on the data to be stored includes:
[0029] Determine the root of the data tree in the data to be stored;
[0030] Based on the storage requirements associated with the candidate storage node list, the gateway data sharding algorithm is used to construct the data to be stored into a data tree that satisfies the data tree root.
[0031] The encrypted storage order proposal is generated based on the root of the data tree, the parameters of the data tree, and the storage requirements.
[0032] Optionally, the step of sending the encrypted storage order proposal and the data to be stored to the target storage node corresponding to the node selection result includes:
[0033] Send the encrypted storage order proposal to the target storage node;
[0034] Upon receiving confirmation of the encrypted storage order proposal from the target storage node, the data to be stored is sent to the target storage node.
[0035] Optionally, the step of sending the data to be stored to the target storage node upon receiving confirmation of the encrypted storage order proposal from the target storage node includes:
[0036] Upon receiving confirmation of the encrypted storage order proposal from the target storage node, a data push request is sent to the data transmission module;
[0037] Upon receiving confirmation of the data push request sent by the data transmission module, the data to be stored is sent to the target storage node.
[0038] To achieve the above objectives, the present invention provides a data storage method for a target storage node, the data storage method comprising the following steps:
[0039] Receive encrypted storage order proposals and data to be stored from the gateway node;
[0040] Based on the proposed encrypted storage order, the data to be stored is stored to a blockchain node.
[0041] Optionally, the step of receiving the encrypted storage order proposal and the data to be stored sent by the gateway node includes:
[0042] Receive the encrypted storage order proposal sent by the gateway node;
[0043] When the storage parameters corresponding to the encrypted storage order proposal pass the verification, the attribute information of the user node associated with the encrypted storage order proposal is confirmed by the settlement state machine module;
[0044] If the attribute information of the user node meets the total storage conditions, then send an encrypted storage order proposal confirmation to the gateway node;
[0045] Receive the data to be stored sent by the gateway node.
[0046] Optionally, the data to be stored includes a data tree, and the step of receiving the encrypted storage order proposal and the data to be stored sent by the gateway node includes:
[0047] Receive the encrypted storage order proposal sent by the gateway node;
[0048] After receiving a data push request from the data transmission module, the data transmission module sends a confirmation request to the gateway node.
[0049] Receive the data tree root sent by the gateway node, and send a data transmission request for the data tree corresponding to the data tree root to the gateway node;
[0050] The gateway node receives the data tree sent block by block based on the data transmission request and signs the data tree.
[0051] Optionally, after the step of receiving the encrypted storage order proposal and the data to be stored sent by the gateway node, the following steps are included:
[0052] Send the second transaction certificate to the settlement state machine module;
[0053] Receive transaction confirmation information sent by the settlement state machine module based on the second transaction certificate.
[0054] Optionally, after the step of storing the data to be stored to the blockchain node based on the encrypted storage order proposal, the method further includes:
[0055] Based on the proposed encrypted storage order, a corresponding encrypted storage order is generated;
[0056] The encrypted storage order is signed, and the signed encrypted storage order is sent to the blockchain node;
[0057] Receive order release information sent by the blockchain node;
[0058] Based on the order release information, send the order acceptance information to the gateway node.
[0059] Furthermore, to achieve the above objectives, the present invention also provides a data storage device, the device comprising:
[0060] The sending module is used to send a storage node list request to the gateway node;
[0061] A receiving module is used to receive a list of candidate storage nodes sent by the gateway node based on the storage node list request;
[0062] The determination module is used to output the candidate storage node list and determine the node selection result based on the received selection action;
[0063] The sending module is used to send the node selection result and the data to be stored to the gateway node.
[0064] Furthermore, to achieve the above objectives, the present invention also provides a data storage system, which includes user nodes, gateway nodes, storage nodes, and blockchain nodes, wherein the data storage system is configured to implement the steps of the data storage method described above.
[0065] In addition, to achieve the above objectives, the present invention also provides a data storage device, the data storage device comprising: a memory, a processor, and a data storage program stored on the memory and executable on the processor, the data storage program being configured to implement the steps of the data storage method described above.
[0066] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing a data storage program, which, when executed by a processor, implements the steps of the data storage method.
[0067] In one technical solution provided by this invention, the user node obtains a list of candidate storage nodes, determines the node selection result based on the received selection action, and then sends the node selection result and the data to be stored to the gateway node. At this point, the gateway node can send the data to be stored to the corresponding target storage node, so that the target storage node can complete the corresponding data storage. In this process, the gateway node acts as a data relay node, capable of receiving and sending an infinite amount of data, thereby achieving large-capacity data storage. Attached Figure Description
[0068] Figure 1 This is a diagram of the encrypted storage network architecture of the data storage method of the present invention;
[0069] Figure 2 This is a flowchart illustrating the first embodiment of the data storage method of the present invention;
[0070] Figure 3 This is an interactive schematic diagram of the first embodiment of the data storage method of the present invention;
[0071] Figure 4 This is a flowchart illustrating step S14 in the first embodiment of the data storage method of the present invention;
[0072] Figure 5 This is an interactive schematic diagram of step S14 in the first embodiment of the data storage method of the present invention;
[0073] Figure 6 This is a flowchart illustrating the second embodiment of the data storage method of the present invention;
[0074] Figure 7 This is an interactive schematic diagram of the second embodiment of the data storage method of the present invention;
[0075] Figure 8 This is a flowchart illustrating step S21 in the second embodiment of the data storage method of the present invention;
[0076] Figure 9 This is an interactive schematic diagram of step S21 in the second embodiment of the data storage method of the present invention;
[0077] Figure 10 This is a flowchart illustrating step S22 in the second embodiment of the data storage method of the present invention;
[0078] Figure 11 This is an interactive schematic diagram of step S22 in the second embodiment of the data storage method of the present invention;
[0079] Figure 12 This is a flowchart illustrating step S22 in the second embodiment of the data storage method of the present invention;
[0080] Figure 13 This is a schematic diagram of a data tree in the second embodiment of the data storage method of the present invention;
[0081] Figure 14 This is a flowchart illustrating step S23 in the second embodiment of the data storage method of the present invention;
[0082] Figure 15 This is an interactive schematic diagram of step S23 in the second embodiment of the data storage method of the present invention;
[0083] Figure 16 This is a flowchart illustrating step S232 in the second embodiment of the data storage method of the present invention;
[0084] Figure 17 This is an interactive schematic diagram of step S222 in the second embodiment of the data storage method of the present invention;
[0085] Figure 18This is a flowchart illustrating the third embodiment of the data storage method of the present invention;
[0086] Figure 19 This is an interactive schematic diagram of the third embodiment of the data storage method of the present invention;
[0087] Figure 20 This is a flowchart illustrating step S31 in the third embodiment of the data storage method of the present invention;
[0088] Figure 21 This is an interactive schematic diagram of step S31 in the third embodiment of the data storage method of the present invention;
[0089] Figure 22 This is a flowchart illustrating step S31 in the third embodiment of the data storage method of the present invention;
[0090] Figure 23 This is a flowchart illustrating the process after step S31 in the third embodiment of the data storage method of the present invention.
[0091] Figure 24 This is a schematic diagram of the interaction after step S31 in the third embodiment of the data storage method of the present invention;
[0092] Figure 25 This is a flowchart illustrating the process after step S33 in the third embodiment of the data storage method of the present invention.
[0093] Figure 26 This is a schematic diagram of the interaction after step S33 in the third embodiment of the data storage method of the present invention;
[0094] Figure 27 This is a schematic diagram of the first part of the data storage method of the present invention;
[0095] Figure 28 This is a schematic diagram of the second part of the data storage method of the present invention;
[0096] Figure 29 This is a flowchart illustrating the third part of the data storage method of the present invention;
[0097] Figure 30 This is a schematic diagram of the data storage device in the hardware operating environment involved in the embodiments of the present invention.
[0098] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0099] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0100] The ENS gateway runs on the Ethereum blockchain, resolving .eth domain names to Ethereum addresses. However, the Ethereum blockchain has limited storage capacity, insufficient to handle large amounts of encrypted data. Furthermore, Ethereum's Transactions Per Second (TPS) is also low, around 1000, unable to handle high-concurrency client access to the server, and the server's data capacity is also limited. Therefore, it is impossible to run a large number of centralized applications on the ENS gateway.
[0101] This application proposes a data storage method that enables large-capacity data storage through the interaction between user nodes, network nodes, target storage nodes, and the blockchain.
[0102] To better understand the above technical solutions, exemplary embodiments of this application will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of this application and to fully convey the scope of this application to those skilled in the art.
[0103] Web3.0 is an improvement on Web2.0. In this environment, users do not need to create multiple identities on different centralized platforms. Instead, they can create a decentralized universal digital identity system that works across various platforms.
[0104] Web3 is used to describe the potential next stage of the internet: a decentralized internet running on blockchain technology. Web3.0 enables information within a website to interact directly with information on other websites, and allows for the simultaneous integration and use of information from multiple websites through third-party information platforms; users own their own data on the internet and can access it across different websites; it is entirely web-based, allowing system functions that would otherwise require complex programs to achieve with just a browser; and user data is audited and synchronized with network data.
[0105] Reference Figure 1This application uses a Web3.0 network as an example to construct a data storage system for realizing data storage transactions. This system consists of web3 user nodes, web3 gateway nodes, web3 storage nodes, and blockchain nodes. The web3 user nodes correspond to web3 storage user A, web3 storage user B, and web3 storage user C, and all use a Super SIM card (Subscriber Identity Module), a special SIM card with larger storage capacity and more functions, capable of storing more personal information and data. The web3 encrypted storage network includes several web3 gateway nodes and web3 storage nodes. The blockchain nodes interact with the web3 encrypted storage network to realize the on-chain recording of encrypted storage orders and daily settlement information.
[0106] In addition, the data transmission module, settlement state machine module, and encrypted storage module serve as components responsible for related functions. Web3 user nodes, web3 gateway nodes, and web3 storage nodes contain data transmission modules; web3 user nodes, web3 gateway nodes, web3 storage nodes, and blockchain nodes contain settlement state machine modules; and web3 storage nodes contain encrypted storage modules.
[0107] This invention provides a data storage method applied to user nodes, as described in the following embodiments. Figure 2 , Figure 2 This is a flowchart illustrating a first embodiment of a data storage method according to the present invention.
[0108] In this embodiment, the data storage method includes:
[0109] Step S11: Send a storage node list request to the gateway node;
[0110] Step S12: Receive the candidate storage node list sent by the gateway node based on the storage node list request;
[0111] Understandably, when each web3 storage node joins the web3 network, it registers {web3 storage node name, web3 storage node IP address, port, and margin account} in the storage node list of the blockchain database.
[0112] Optionally, such as Figure 3 During the storage node discovery phase, the user node sends a storage node list request to the gateway node, i.e., it queries the list of candidate storage nodes. This candidate storage node list contains information about all available, unfiltered storage nodes.
[0113] For example, a web3 user node sends a JSON request {StorgeNode: ?} to a web3 gateway node. The web3 gateway node forwards the JSON request {StorgeNode: ?} to a blockchain node. The blockchain node reads the list of storage nodes in the blockchain database and then sends a JSON response {StorgeNode: Node A; Node B; Node C; Node D} to the web3 gateway node, which is the candidate list of storage nodes. At this point, the web3 gateway node can send the candidate list of storage nodes to the web3 user node.
[0114] Step S13: Output the candidate storage node list and determine the node selection result based on the received selection action;
[0115] Step S14: Send the node selection result and the data to be stored to the gateway node.
[0116] Optionally, the user node outputs a list of candidate storage nodes, allowing the user to select several suitable nodes from the list based on storage needs, such as the size of the data to be stored, the storage period, and the unit price of storage. For example, selecting storage nodes with better performance and more reasonable prices improves the efficiency and reliability of data storage.
[0117] Accordingly, after receiving the user's selection action, the user node determines the node selection result, that is, the target storage node selected by the user. For example... Figure 3 During the storage order matching phase, the user node sends the node selection result and the data to be stored to the gateway node so that the target storage node can perform the corresponding storage operation on the data to be stored.
[0118] Reference Figure 4 Step S14 includes:
[0119] Step S141: Send a first transaction request to the gateway node for the node selection result;
[0120] Step S142: Receive the first transaction information sent by the gateway node based on the first transaction request;
[0121] Step S143: Send the first transaction certificate and the data to be stored to the gateway node based on the first transaction information.
[0122] It is understandable that, such as Figure 5During the encrypted storage pricing phase, before officially commencing storage, user nodes need to understand the transaction information regarding the target storage node. For ease of understanding, price information will be used as an example below. This setup ensures that both user nodes and the target storage node fulfill the storage agreement, helps establish a trust mechanism for the storage service, and protects the data security and rights of user nodes.
[0123] Optionally, the user node sends a first transaction request, i.e. a price inquiry request, to the gateway node for the node selection result. Accordingly, the gateway node queries the attribute information of each storage node, i.e. the asking price information, based on the price inquiry request, and then sends the corresponding first transaction information, i.e. the price information, to the user node.
[0124] Furthermore, the user node provides the data to be stored and the price information to provide the corresponding first transaction certificate, namely the margin transaction certificate. The margin transaction certificate is a document generated by the user node when conducting a margin transaction. The specific transaction method can be paying margin to a third-party institution or directly submitting margin to the gateway node. The margin will be deposited into the gateway node's account in the settlement state machine module, specifically reflected in corresponding changes to the deposit data of the user and the gateway node. This embodiment does not specifically limit the specific method.
[0125] It is important to note that since the margin transaction certificate records the transaction object, i.e. the target storage node, the user node only needs to send the margin transaction certificate to the gateway node. Accordingly, the gateway node can determine the target storage node based on the margin transaction certificate, i.e., the node selection result.
[0126] In one technical solution provided in this embodiment, the user node obtains a list of candidate storage nodes, determines the node selection result based on the received selection action, and then sends the node selection result and the data to be stored to the gateway node. At this point, the gateway node can send the data to be stored to the corresponding storage node, so that the storage node can complete the corresponding data storage. In this process, the gateway node acts as a data relay node, capable of receiving and sending an unlimited amount of data, thereby achieving large-capacity data storage.
[0127] This invention provides a data storage method applied to a gateway node, as described in the following embodiments. Figure 6 , Figure 6 This is a flowchart illustrating a second embodiment of a data storage method according to the present invention.
[0128] In this embodiment, the data storage method includes:
[0129] Step S21: Upon receiving a storage node list request from a user node, send a candidate storage node list to the user node;
[0130] Optionally, such as Figure 7 During the storage node discovery phase, when the gateway node receives a storage node list request from the user node, it sends a list of candidate storage nodes to the user node.
[0131] As for the process of obtaining the candidate storage node list, it can be obtained by accessing the state data of the blockchain, and this embodiment does not make specific limitations.
[0132] In addition, gateway nodes can obtain the account attribute information of each storage node by cyclically accessing the state data of the blockchain, so as to subsequently pay the corresponding account a deposit.
[0133] Step S22: Receive the node selection result and data to be stored sent by the user node, and generate the encrypted storage order proposal based on the data to be stored;
[0134] Step S23: Send the encrypted storage order proposal and the data to be stored to the target storage node corresponding to the node selection result.
[0135] Optionally, such as Figure 7 During the storage order matching phase, after receiving the node selection result and the data to be stored from the user node, the gateway node can directly generate an encrypted storage order proposal based on the data to be stored. Then, the encrypted storage order proposal and the data to be stored are sent to the target storage node corresponding to the node selection result.
[0136] Reference Figure 8 Step S21 includes:
[0137] Step S211: Upon receiving the storage node list request sent by the user node, send an access request to the blockchain node;
[0138] Step S212: Receive the list of candidate storage nodes from the blockchain node in response to the access request;
[0139] Step S213: Send the candidate storage node list to the user node.
[0140] Optionally, such as Figure 9 During the storage order matching phase, when each web3 storage node joins the web3 network, it registers in the storage node list of the blockchain database of the blockchain node. Therefore, when the gateway node receives a storage node list request from the user node, it needs to send an access request to the blockchain node. This access request is to access the storage node list of the blockchain database of the blockchain node to obtain the candidate storage node list responded by the blockchain node and send it to the user node.
[0141] This solution provides interactive operations between the gateway node and the data chain node involving the candidate storage node list. This solution adopts a method of directly accessing the blockchain node's blockchain database. Because the data is stored in a local database, the access speed is faster, and the data reliability is higher, unaffected by network instability or other node failures.
[0142] Reference Figure 10 Step S22 includes:
[0143] Step S221: After receiving the first transaction request sent by the user node, obtain the attribute information of each storage node corresponding to the node selection result;
[0144] Step S222: Send the first transaction information to the user node according to the attribute information corresponding to the storage node;
[0145] For example, such as Figure 11 In the encrypted storage price inquiry phase, after receiving the first transaction request (price inquiry) from the user node, the gateway node iteratively visits each storage node corresponding to the selected result (i.e., the target storage node) and obtains the corresponding attribute information, i.e., the asking price information, such as the storage unit price nPrice, which is how much per MB per day. Then, based on the obtained asking price information, it sends the first transaction information, i.e., the price information, to the user node.
[0146] Step S223: Receive the first transaction certificate and the data to be stored sent by the user node, and send the first transaction certificate to the settlement state machine module;
[0147] Step S224: After receiving the confirmation information from the settlement state machine module, generate the encrypted storage order proposal based on the data to be stored.
[0148] Optionally, such as Figure 11 During the storage order matching phase, after receiving the transaction voucher and the data to be stored from the user node, the gateway node sends the transaction voucher to the settlement state machine module. Upon receiving confirmation from the settlement state machine module, it indicates that the module has confirmed the target storage node's account has received the required margin. At this point, the gateway node further generates a cryptographic storage order proposal.
[0149] Setting up a deposit mechanism helps establish a trust mechanism for storage services and protects the data security and rights of user nodes.
[0150] Reference Figure 12 The data to be stored includes a data tree, and the step of generating the encrypted storage order proposal based on the data to be stored includes:
[0151] Step S225: Determine the root of the data tree in the data to be stored;
[0152] Step S226: Based on the storage requirements associated with the candidate storage node list, use the gateway data sharding algorithm to construct the data to be stored into a data tree that satisfies the data tree root.
[0153] Step S227: Generate the encrypted storage order proposal based on the data tree root and the storage requirements.
[0154] Optionally, in Figure 11 During the storage order matching stage, after receiving the data to be stored sent by the user node, the gateway node will determine the root of the data tree in the data to be stored.
[0155] Furthermore, based on the storage requirements associated with the candidate storage node list, such as the size of the data to be stored, the storage period, and the unit price of storage, the gateway data sharding algorithm is used to construct a data tree that satisfies the data tree root.
[0156] In addition, the gateway node generates encrypted storage order proposals based on the data tree root and storage requirements, namely the data tree root, the size of the data to be stored, the storage period, and the storage unit price, and sends these proposals to the target storage node. After the data to be stored has completely moved in and out of the target storage node, the gateway node writes the storage node information into the blockchain and periodically checks whether the storage order has been processed correctly.
[0157] The gateway data sharding algorithm is as follows:
[0158] The data to be stored, DATA, is divided into data fragments of 8MB units, padded with zeros if the size is less than αMB. Each data fragment has a Content Identifier (CID), typically 128 bits, and is located by constructing an n-level Web3 data tree (DATA-tree). The root is the CID of the data to be stored, representing the root of the DATA-tree. The bottom of the Web3 data tree consists of data fragments of αMB units. Every two bottom-level nodes form the next level node, and this process continues, forming an n-level DATA-tree. Each data fragment can be described by the path from the root of the DATA-tree to that fragment. For example, data fragment 2 can be described by the path ced, such as... Figure 13 As shown.
[0159] It's important to note that user nodes and gateway nodes synchronize data via a web3 data tree. The Graph Synchronization Protocol (Graphsync) is a protocol for synchronizing data trees between different nodes. This protocol allows a local node to send a request to a remote node, which then searches that node's data tree using a selector and returns the desired result matching the request—the data tree itself.
[0160] Specifically, the data transmission process uses the libp2p protocol, and the protocol name is / fil / datatransfer / 1.0.
[0161] Both parties involved in the data transmission have registered the protocol / fil / datatransfer / 1.0. Data sender A sends the CID1 string of data tree DATA-tree1 to data receiver B. After receiving CID1, data receiver B agrees to receive the data. The data in data tree DATA-tree1 will be automatically transmitted to data receiver B according to the protocol / fil / datatransfer / 1.0, and the data synchronization is complete.
[0162] This solution uses a data sharding algorithm to divide large datasets into smaller chunks, which are then transmitted via a data tree. This improves data transmission efficiency and reduces transmission time. Furthermore, the data sharding algorithm can evenly distribute data across different nodes, achieving load balancing, reducing the pressure on individual nodes, and improving overall transmission efficiency.
[0163] Reference Figure 14 Step S23 includes:
[0164] Step S231: Send the encrypted storage order proposal to the target storage node;
[0165] Step S232: Upon receiving confirmation of the encrypted storage order proposal from the target storage node, send the data to be stored to the target storage node.
[0166] Optionally, such as Figure 15During the data transmission phase, after sending an encrypted storage order proposal to the target storage node, the gateway node waits for confirmation from the target storage node. That is, it only sends the data to be stored to the target storage node upon receiving confirmation of the encrypted storage order proposal. By waiting for the storage node to send a confirmation signal, the gateway node can confirm that the target storage node is ready to receive data, thereby improving the reliability and integrity of data transmission and ensuring that data can be transmitted securely and completely to the target storage node, avoiding data loss or errors during transmission.
[0167] Reference Figure 16 Step S232 includes:
[0168] Step S2321: Send a data push request to the data transmission module;
[0169] Step S2322: Upon receiving confirmation of the data push request sent by the data transmission module, send the data to be stored to the target storage node.
[0170] Optionally, such as Figure 17 During the data transmission phase, the gateway node sends a data push request to the data transmission module, which then sends a push request to the storage node. After the storage node receives and requests the data, it sends a data push request confirmation to the gateway node. Only then will the gateway node send the data to be stored to the target storage node.
[0171] Furthermore, after the gateway node completes data transmission, it can notify the data storage module that the data transmission is complete, and then the data storage module can notify the target storage node that the data transmission is complete.
[0172] During data transmission, an additional data storage module is set up between the gateway node and the target storage node to effectively save the data transmission records between the two, thereby providing a more comprehensive and complete data management and protection mechanism to ensure the reliability and integrity of data transmission.
[0173] In one technical solution provided in this embodiment, the gateway node realizes the proxy function through interaction with the user node, pays a certain storage fee in the transaction mode of margin / deposit, and stores the data to the storage node through interaction with the user node and the storage node. In the whole process, the gateway node, as a data relay node, can receive and send an infinite amount of data, thereby realizing large-capacity data storage.
[0174] This invention provides a data storage method for a target storage node, as described in the following embodiments. Figure 18 , Figure 18 This is a flowchart illustrating a third embodiment of a data storage method according to the present invention.
[0175] In this embodiment, the data storage method includes:
[0176] Step S31: Receive the encrypted storage order proposal and the data to be stored sent by the gateway node;
[0177] Step S32: Based on the encrypted storage order proposal, store the data to be stored to the blockchain node.
[0178] like Figure 19 During the data transmission phase, the target storage node receives the encrypted storage order proposal and the data to be stored sent by the gateway node, and then enters the storage order publishing and on-chain phase. Based on the encrypted storage order proposal, the target storage node stores the data to be stored to the blockchain node.
[0179] Optionally, see Figure 20 Step S31 includes:
[0180] Step S311: Receive the encrypted storage order proposal sent by the gateway node;
[0181] Step S312: When the storage parameters corresponding to the encrypted storage order proposal pass the verification, the attribute information of the user node associated with the encrypted storage order proposal is confirmed through the settlement state machine module;
[0182] Step S313: If the attribute information of the user node meets the total storage conditions, then send an encrypted storage order proposal confirmation to the gateway node;
[0183] Step S314: Receive the data to be stored sent by the gateway node.
[0184] like Figure 21 During the storage order verification phase, after receiving an encrypted storage order proposal, the target storage node verifies the storage parameters corresponding to that proposal, such as sharding and size. After successful verification, the settlement state machine module confirms the attribute information of the user node associated with the encrypted storage order proposal, thereby determining whether the user node's attributes meet the total storage requirements. For example, it checks if the user node's margin is sufficient. If the margin is greater than the total storage quote, it indicates that the margin is adequate, and the target storage node accepts the encrypted storage order proposal and sends confirmation to the gateway node. Only after completing these steps will it receive the data to be stored from the gateway node.
[0185] For example, when each storage node joins the network, it registers {web3 storage node name, web3 storage node IP address, port, and margin account} in the storage node list of the blockchain database. Based on this, the current margin balance can be determined by querying the margin account in the storage node list.
[0186] This solution includes storage parameter verification and margin verification. Firstly, it ensures that the target storage node has sufficient capacity and performance to receive and process the data to be stored. This prevents data loss or corruption due to improper storage node configuration. Secondly, it establishes a trust mechanism for the storage service, protecting the data security and rights of user nodes.
[0187] Optionally, see Figure 22 Step S31 includes:
[0188] Step S315: Receive the encrypted storage order proposal sent by the gateway node;
[0189] Step S316: After receiving the data push request sent by the data transmission module, send a data push request confirmation to the gateway node through the data transmission module;
[0190] Step S317: Receive the data tree root sent by the gateway node, and send a data transmission request for the data tree corresponding to the data tree root to the gateway node;
[0191] Step S318: Receive the data tree sent block by block by the gateway node based on the data transmission request.
[0192] In the data transmission phase of this scheme, after receiving the data push request from the data transmission module, the target storage node sends a data push request confirmation to the gateway node through the data transmission module. It then receives the data tree root sent by the gateway node, sends a data transmission request for the data tree corresponding to the data tree root to the gateway node, and finally receives the data tree sent block by block by the gateway node based on the data transmission request. The specific process is the same as described above. Figure 17 This will not be elaborated upon here.
[0193] The transmission principle is as follows:
[0194] Data sender A sends the CID1 string of the data tree (DATA-tree1) to data receiver B. After receiving the CID1, data receiver B agrees to receive the data. The data in the data tree (DATA-tree1) will be automatically transferred to data receiver B according to the protocol / fil / datatransfer / 1.0, and the data synchronization is completed.
[0195] See Figure 23After step S31, the following steps are included:
[0196] Step S33: Send the second transaction certificate to the settlement state machine module;
[0197] Step S34: Receive transaction confirmation information sent by the settlement state machine module based on the second transaction certificate.
[0198] like Figure 24 During the data transmission phase, the target data node sends a second transaction voucher to the settlement state machine module to further ensure transaction security. For ease of understanding, a deposit transaction voucher will be used as an example below. This deposit transaction voucher can be obtained after paying a deposit to the settlement state machine module or after paying a deposit to a third-party institution; there are no restrictions here. Correspondingly, the target storage node receives transaction confirmation information sent by the settlement state machine module based on the deposit transaction voucher. After data transmission is complete, the target storage node can also sign the encrypted storage order.
[0199] In the settlement status module, once a deposit is deposited into the target storage node, the deposit will be automatically deducted by the system if the target storage node goes offline. Therefore, setting a deposit helps ensure that the target storage node complies with the agreement, provides reliable service, and reduces malicious behavior, thereby maintaining network security and data reliability.
[0200] See Figure 25 After step S33, the following steps are included:
[0201] Step S35: Generate a corresponding encrypted storage order based on the encrypted storage order proposal;
[0202] Step S36: Sign the encrypted storage order and send the signed encrypted storage order to the blockchain node;
[0203] Step S37: Receive the order release information sent by the blockchain node;
[0204] Step S38: Based on the order release information, send the order acceptance information to the gateway node.
[0205] like Figure 26 During the on-chain storage order publication phase, the target storage node will generate a corresponding encrypted storage order based on the encrypted storage order proposal, sign it, and then publish the signed encrypted storage order to the blockchain node to request the on-chain publication of the encrypted storage order.
[0206] Furthermore, once a crypto storage order has been published, the blockchain node will send the order publication information to the target storage node, and then the target storage node will send the order acceptance information to the gateway node, such as sending the CID of the storage order that has been uploaded to the blockchain to the gateway node.
[0207] Furthermore, the gateway node will confirm with the blockchain node whether the encrypted storage order message has appeared on the blockchain. In response, the blockchain node will inform the user node that the encrypted storage order has been published. At this point, the gateway node will notify the user node that the order has been put on the blockchain.
[0208] It is important to note that during the automatic processing phase, once a cryptographic storage order on the blockchain is activated, the system will make linear payments according to the data storage time in the storage order and according to the block epoch. That is, a certain amount of storage fee will be paid to the storage node in each epoch (the client's storage fee will be deducted from the client's storage market system account margin and paid to the storage node) until the storage order is completed.
[0209] In one technical solution provided in this embodiment, the target storage node receives an encrypted storage order proposal and the data to be stored sent by the gateway node, and stores the data to be stored to the blockchain node based on the encrypted storage order proposal. Since the gateway node acts as a data relay node, it can receive and send an infinite amount of data. Therefore, the target storage node can also continuously receive data from the gateway node and continuously store this data in the blockchain, thereby achieving large-capacity data storage.
[0210] Reference Figure 27 , Figure 28 and Figure 29 To complete the process, encrypted storage orders are processed according to the following steps: storage node discovery, encrypted storage price inquiry, storage order matching, storage order verification, data transmission, storage order on-chain release, and automatic processing.
[0211] During the storage node discovery phase, web3 user nodes obtain a list of candidate web3 encrypted storage nodes that meet the requirements in the network.
[0212] During the encrypted storage inquiry phase, web3 user nodes inquire about the price of specific web3 encrypted storage nodes.
[0213] During the storage order matching phase, web3 user nodes pay a deposit and the data to be stored to the web3 gateway node, and the web3 gateway node initiates a proposal for an encrypted storage order.
[0214] During the storage order verification phase, the web3 gateway node pays a deposit to the web3 storage node, and the web3 storage node confirms the encrypted storage order.
[0215] During the data transmission phase, the web3 gateway node transmits the data of the web3 user node to the web3 storage node through the data transmission module.
[0216] During the on-chain release of storage orders, web3 storage nodes publish encrypted storage orders on the blockchain, and web3 gateway nodes confirm them on the blockchain, at which point the encrypted storage orders become effective.
[0217] During the automatic processing phase, once a cryptographic storage order on the blockchain is activated, the system will make linear payments according to the data storage time in the storage order and according to the block epoch. That is, a certain amount of storage fee will be paid to the storage node in each epoch. The storage fee of the client will be deducted from the client's storage market system account margin and paid to the storage node until the storage order is completed.
[0218] This invention provides a data storage device, the device comprising:
[0219] The sending module is used to send a storage node list request to the gateway node;
[0220] A receiving module is used to receive a list of candidate storage nodes sent by the gateway node based on the storage node list request;
[0221] The determination module is used to output the candidate storage node list and determine the node selection result based on the received selection action;
[0222] The sending module is used to send the node selection result and the data to be stored to the gateway node.
[0223] Since the embodiments of the apparatus and the embodiments of the method correspond to each other, please refer to the description of the embodiments of the method for the embodiments of the apparatus, and they will not be repeated here.
[0224] Additionally, the device may also include:
[0225] The sending module is used to send the encrypted storage order proposal to the target storage node;
[0226] The sending module is used to send the data to be stored to the target storage node when it receives the confirmation of the encrypted storage order proposal sent by the target storage node.
[0227] Additionally, the device may also include:
[0228] The receiving module is used to receive the encrypted storage order proposal sent by the gateway node;
[0229] The determination module is used to confirm the margin of the user node associated with the encrypted storage order proposal through the settlement state machine module when the storage parameters corresponding to the encrypted storage order proposal are verified.
[0230] The sending module is used to send a cryptographic storage order proposal confirmation to the gateway node if the deposit is greater than the total storage quote;
[0231] A receiving module is used to receive the data to be stored sent by the gateway node.
[0232] Reference Figure 30 , Figure 30 This is a schematic diagram of the data storage device structure of the hardware operating environment involved in the embodiments of the present invention.
[0233] like Figure 30 As shown, the data storage device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.
[0234] Those skilled in the art will understand that Figure 30 The structure shown does not constitute a limitation on the data storage device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0235] like Figure 30 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a data storage module, a network communication module, a user interface module, and a data storage program.
[0236] exist Figure 30In the data storage device shown, the network interface 1004 is mainly used for data communication with other devices; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the data storage device of the present invention can be set in the data storage device, and the data storage device calls the data storage program stored in the memory 1005 through the processor 1001 and executes the data storage method provided in the embodiment of the present invention.
[0237] This invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described data storage methods.
[0238] Since the embodiments of the computer-readable storage medium portion correspond to the embodiments of the method portion, the embodiments of the computer-readable storage medium portion are described in the description of the embodiments of the method portion, and will not be repeated here.
[0239] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.
[0240] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0241] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.
[0242] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.
Claims
1. A data storage method, characterized in that, Applied to gateway nodes, the data storage method includes the following steps: Upon receiving a storage node list request from a user node, a candidate storage node list is sent to the user node. The process of receiving node selection results and data to be stored from the user node, and generating an encrypted storage order proposal based on the data to be stored, includes: after receiving a first transaction request from the user node, obtaining attribute information corresponding to each storage node in the node selection result; sending first transaction information to the user node according to the attribute information corresponding to the storage nodes; receiving a first transaction voucher and the data to be stored from the user node, and sending the first transaction voucher to the settlement state machine module; after receiving confirmation information from the settlement state machine module, determining the root of the data tree in the data to be stored; constructing a data tree that satisfies the root of the data tree using a gateway data sharding algorithm based on the storage requirements associated with the candidate storage node list; and generating the encrypted storage order proposal based on the root of the data tree, the parameters of the data tree, and the storage requirements. Sending the encrypted storage order proposal and the data to be stored to the target storage node corresponding to the node selection result includes: sending the encrypted storage order proposal to the target storage node; and sending the data to be stored to the target storage node upon receiving confirmation of the encrypted storage order proposal from the target storage node.
2. The data storage method as described in claim 1, characterized in that, The step of sending a candidate storage node list to the user node upon receiving the storage node list request includes: Upon receiving the storage node list request sent by the user node, an access request is sent to the blockchain node; Receive the list of candidate storage nodes from the blockchain node in response to the access request; Send a list of candidate storage nodes to the user node.
3. The data storage method as described in claim 1, characterized in that, The step of sending the data to be stored to the target storage node upon receiving confirmation of the encrypted storage order proposal from the target storage node includes: Upon receiving confirmation of the encrypted storage order proposal from the target storage node, a data push request is sent to the data transmission module; Upon receiving confirmation of the data push request sent by the data transmission module, the data to be stored is sent to the target storage node.
4. A data storage method, characterized in that, When applied to user nodes, the data storage method includes the following steps: Send a storage node list request to the gateway node according to any one of claims 1-3; Receive the candidate storage node list sent by the gateway node based on the storage node list request; Output the candidate storage node list and determine the node selection result based on the received selection action; Send the node selection result and the data to be stored to the gateway node.
5. The data storage method as described in claim 4, characterized in that, The step of sending the node selection result and the data to be stored to the gateway node includes: Send a first transaction request to the gateway node for the node selection result; Receive the first transaction information sent by the gateway node based on the first transaction request; Based on the first transaction information, the first transaction certificate and the data to be stored are sent to the gateway node.
6. A data storage method, characterized in that, For a target storage node, the data storage method includes the following steps: Receive the encrypted storage order proposal and the data to be stored sent by the gateway node according to any one of claims 1-3; Based on the proposed encrypted storage order, the data to be stored is stored to a blockchain node.
7. The data storage method as described in claim 6, characterized in that, The steps of receiving the encrypted storage order proposal and the data to be stored sent by the gateway node include: Receive the encrypted storage order proposal sent by the gateway node; When the storage parameters corresponding to the encrypted storage order proposal pass the verification, the attribute information of the user node associated with the encrypted storage order proposal is confirmed by the settlement state machine module; If the attribute information of the user node meets the total storage conditions, then send an encrypted storage order proposal confirmation to the gateway node; Receive the data to be stored sent by the gateway node.
8. The data storage method as described in claim 6, characterized in that, The data to be stored includes a data tree, and the step of receiving the encrypted storage order proposal and the data to be stored sent by the gateway node includes: Receive the encrypted storage order proposal sent by the gateway node; After receiving a data push request from the data transmission module, the data transmission module sends a confirmation request to the gateway node. Receive the data tree root sent by the gateway node, and send a data transmission request for the data tree corresponding to the data tree root to the gateway node; The gateway node receives the data tree sent block by block based on the data transmission request and signs the data tree.
9. The data storage method as described in claim 6, characterized in that, After the step of receiving the encrypted storage order proposal and the data to be stored sent by the gateway node, the following steps are included: Send the second transaction certificate to the settlement state machine module; Receive transaction confirmation information sent by the settlement state machine module based on the second transaction certificate.
10. The data storage method as described in claim 6, characterized in that, Following the step of storing the data to be stored to the blockchain node based on the encrypted storage order proposal, the following steps are included: Based on the proposed encrypted storage order, a corresponding encrypted storage order is generated; The encrypted storage order is signed, and the signed encrypted storage order is sent to the blockchain node; Receive order release information sent by the blockchain node; Based on the order release information, send the order acceptance information to the gateway node.
11. A data storage system, characterized in that, The data storage system includes user nodes, gateway nodes, storage nodes, and blockchain nodes, wherein the data storage system is configured to implement the steps of the data storage method as described in any one of claims 1 to 10.
12. A data storage device, characterized in that, The data storage device includes: a memory, a processor, and a data storage program stored on the memory and executable on the processor, the data storage program being configured to implement the steps of the data storage method as described in any one of claims 1 to 10.
13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a data storage program, which, when executed by a processor, implements the steps of the data storage method as described in any one of claims 1 to 10.
Citation Information
Patent Citations
Intelligent terminal storage sharing method and device based on block chain, medium and equipment
CN112416881A
Internet of Things edge computing system and method based on block chain
CN113065866A
