Decentralized storage system, method, device and storage medium for multi-version files

Abstract

The application belongs to the technical field of network storage, and particularly relates to a decentralized storage system, method and device for multi-version files and a storage medium, comprising a client, a verification node and a blockchain, the client is responsible for sending a file storage, retrieval or download request; the verification node provides file storage and file acquisition services to the client and maintains a blockchain ledger; the blockchain is a double-layer DAG structure, comprising upper and lower layers, the lower layer supports file creation, update, merging and forking operations, and the upper layer guarantees the consistency of the ledger, and the two layers share the same group of blocks. The advantage lies in that the double-layer DAG blockchain structure is adopted, the lower layer supports file creation, update, merging and forking operations, and the upper layer guarantees the consistency of the ledger. The structure saves additional storage space for file indexing.

Description

Technical Field

[0001] This application belongs to the field of network storage technology, specifically relating to a decentralized storage system, method, device, and storage medium for multi-version files. Background Technology

[0002] Decentralized Storage Networks (DSNs) are emerging technologies that aggregate storage space from independent storage providers and self-coordinate data storage and retrieval services. Compared to traditional storage networks, DSNs operate on a blockchain system as an incentive layer. The blockchain rewards validating nodes that provide reliable storage to customers, thus establishing an open and manageable storage marketplace. Furthermore, the blockchain can act as a state machine replication protocol, ensuring file consistency across Byzantine nodes.

[0003] Supporting the storage of multiple versions of files is essential in a Data Subsystem for Networking (DSN) because files are often dynamically modified or edited, and users need to query different versions of files from time to time. Since files on current DSNs are not editable, users must upload all versions of a file, leading to high redundancy. Even though some DSNs have made efforts to support directory-level deduplication, they cannot avoid more granular, file-level redundancy. For example, Filecoin uses Merkle DAGs for directory-level deduplication, where objects including files, file blocks, and directories are organized into a Merkle DAG structure based on their nesting relationships to eliminate duplicate objects between different directories, but redundancy between different file versions is still unavoidable. The lack of file-level deduplication leads to wasted storage space and bandwidth. However, implementing file-level deduplication is challenging. Due to the application of encryption and obfuscation to files, the correlation between different versions of files in a DSN is implicit, making it very difficult to find duplicate content and establish relationships between multiple versions of files. Furthermore, traditional version control systems typically use version graphs based on Directed Acyclic Graphs (DAGs) to describe the derivation relationships between multiple versions and help build file indexes. However, such a graph should be maintained by a centralized server, such as GitHub. In current DSNs, centralized servers are unavailable, and the blockchain database merely stores file information as serialized transactions, without considering the derivation relationships between file versions. Therefore, each user must maintain a separate local database to store additional metadata (such as a version graph) to facilitate file operations like creation, modification, merging, and forking. This deficiency hinders blockchain from serving many data-intensive applications because it consumes significant storage space and, in many cases, cannot directly answer file queries.

[0004] In summary, current DSNs only implement directory-level deduplication, and duplicate content of different versions of files is still redundantly stored. The lack of file-level deduplication leads to a waste of storage and bandwidth. Furthermore, the lack of a complete description of the derivation relationship between file versions prevents these systems from providing effective file indexing. Summary of the Invention

[0005] The purpose of this invention is to propose a decentralized storage network system that can achieve file-level deduplication and efficient file indexing. When storing multi-version files, an incremental mechanism is used to achieve file-level deduplication. Specifically, an editing script is implemented using an incremental generation algorithm, which calculates the increment to convert the file from its previous version to the current version. Storing the increment enables fine-grained file-level deduplication and saves storage space. Furthermore, a two-layer DAG blockchain ledger is adopted, which can organize transactions according to derivation relationships. With the help of this ledger, people can record the derivation relationships between files without building an additional database. Moreover, compared with chained ledgers, DAG-based ledgers can provide higher concurrency and can process queries simultaneously. The technical solution is as follows:

[0006] A decentralized storage system for multi-version files, comprising a client, verification nodes, and a blockchain;

[0007] The client is responsible for sending file storage, retrieval, or download requests;

[0008] Verification nodes provide file storage or file retrieval services to clients;

[0009] Blockchain is a two-layer DAG structure. One layer supports file creation, updating, merging, and forking operations, while the other layer ensures the consistency of the ledger. Both layers share the same set of blocks.

[0010] Preferably, the client compares the new file version with the previous file version to calculate the file increment, and sends the text increment to the verification node for storage.

[0011] Preferably, the verification node includes a storage verification node and a retrieval verification node; the storage verification node selectively performs the following processing according to different needs:

[0012] During the file storage process, the storage verification node is responsible for processing file storage requests sent by clients and receiving the initial file... Alternatively, files can be stored incrementally, or file version branches can be merged or forked in the storage space, with storage proofs generated periodically and stored on the blockchain; or...

[0013] During the file retrieval process, the storage verification node is responsible for processing the download requests sent by the client and sending the initial file or file increment stored in its storage space to the client.

[0014] The retrieval and verification node is responsible for retrieving the list of identifiers of the initial file and file increment required for file recovery from the blockchain in a breadth-first search manner according to the retrieval request sent by the client during the file retrieval process, and then sending it to the client.

[0015] Preferably, the client sends different requests during file storage, updating, and retrieval, and generates corresponding transactions to be sent to the blockchain after each request is processed; the client sends file storage requests according to different needs during file storage:

[0016] When there is a need to store the initial file, a file creation request is sent, and the initial file is uploaded to the storage verification node; or

[0017] When there is a need to update a file, a file update request is sent, the new file version is compared with the previous file version to calculate the file increment, and the file increment is uploaded to the storage verification node.

[0018] or

[0019] When there is a need to merge or fork a file version branch, send a file version merge or fork request without attaching any other content; or

[0020] During the file acquisition process, the client is responsible for sending retrieval requests and download requests for the required file version to the retrieval verification node and the storage verification node, respectively. After obtaining the list of identifiers for the initial file and file increment required to restore the file from the retrieval verification node, the client downloads the initial file and file increment from the storage verification node according to the list, and is responsible for restoring the file version based on the downloaded initial file and file increment.

[0021] Preferably, blockchain can be represented as ,in and These are the edge sets of the lower and upper layers of the blockchain, respectively. It is a set of vertices shared by the upper and lower layers; each vertex represents a transaction, and the transaction type is either a create transaction, update transaction, merge transaction, or fork transaction. Different vertices correspond to specific file versions and contain identifiers for the initial file or file increment. .

[0022] Preferred, lower-layer edge set of blockchain Each edge in the blockchain represents the derivation relationship between the update, merge, or fork of the file versions corresponding to the two vertices it connects; the upper-layer edge set of the blockchain It is constructed by modifying the DAG-Rider consensus algorithm, where each validator node broadcasts one transaction per round, i.e., creates a vertex, and each vertex references at least one transaction from the previous round. There are vertices, among which It is the maximum number of Byzantine nodes that can be tolerated.

[0023] Preferably, the blockchain construction process is as follows:

[0024] In any round, transactions that have passed consensus first point to the confirmed transactions they depend on based on the file version derivation relationship. The edges constructed at this time are included in the lower-level edge set of the blockchain. Then, the incoming transaction is directed to other confirmed transactions from the previous round according to the consensus algorithm. The edges constructed at this point are included in the upper-level edge set of the blockchain. middle.

[0025] A decentralized storage method for multi-version files includes the following four steps: creation, updating, retrieval, downloading, and restoration;

[0026] Creation: The client stores the initial file at the storage verification node and sends the file to create a transaction on the blockchain;

[0027] Update: The client calculates the file increment of the new file version compared to the previous file version, stores it at the storage verification node, and sends the file update transaction to the blockchain;

[0028] Retrieval: The retrieval verification node retrieves the list of file identifiers required to recover the file from the blockchain;

[0029] Download and restore: The client downloads the corresponding file fragments from the corresponding storage verification node based on the file identifier list and restores the files.

[0030] Preferably, the creation steps are as follows:

[0031] S11. Client calculates initial file Unique Identifier The initial file is about to be generated. The data is divided into blocks and hashed. Then, the hashes of the blocks are concatenated and hashed again to obtain the identifier. ;

[0032] S12. The client selects a storage verification node from the system and includes... File creation request Send to the storage verification node;

[0033] S13. The storage verification node processes file creation requests sent by clients and simultaneously receives initial files sent by clients. And store it;

[0034] S14. Initial File After the transmission is complete, the client sends a request to create a transaction. Upon receiving the transaction on the blockchain, the verification node broadcasts the transaction, and after consensus is reached, the transaction is added to the blockchain according to the blockchain construction process.

[0035] S15. The storage verification node periodically initializes the file. Generate storage proofs and send them to the blockchain to support other nodes in verifying the integrity and reliability of the files.

[0036] Preferably, the update steps are as follows:

[0037] S21. The client uses an incremental mechanism to calculate file versions. With the initial file File increment And calculate file increment. identifier ;like Size greater than or equal to version Then, the file version will be stored according to the steps of file creation. ;like Smaller than version Then store according to the steps of updating the file. ;

[0038] S22. The client selects a storage verification node from the system and includes... File update request Sending to the storage verification node means in the initial file Update based on;

[0039] S23. The storage verification node processes file update requests sent by the client and simultaneously receives file increments sent by the client. And store it;

[0040] S24. File Increment After the transmission is complete, the client sends an update transaction. Upon receiving the transaction on the blockchain, the verification node broadcasts the transaction, and after consensus is reached, the transaction is added to the blockchain according to the blockchain construction process.

[0041] S25. Storage verification nodes periodically increment file sizes. Generate a storage certificate and send it to the blockchain.

[0042] Preferably, the retrieval steps are as follows:

[0043] S31. The client sends a message containing [the following information] to the retrieval verification node. Search request To restore file version ;

[0044] S32. After receiving the retrieval request, the retrieval verification node first finds the file version in the blockchain. The corresponding transaction, and then the retrieval function is called. Get the recovery file version A list of all required files and increment identifiers;

[0045] S33. The retrieval and verification node sends the obtained list of identifiers to the client. This list contains identifiers for the initial file and file increments required for file recovery, such as... and ;

[0046] S34. After obtaining the list of identifiers, the client sends the search record. To blockchain.

[0047] The preferred download and restore steps are as follows:

[0048] S41. The client sends download requests to the corresponding storage verification nodes according to the identifier list, such as... and ;

[0049] S42. After receiving the client's download request, the storage verification node sends the stored file fragments, i.e., the initial file or the file increment, to the client;

[0050] S43. After the client has finished downloading the required files, it will record the download history. and Send to the blockchain;

[0051] S44. Client calls recovery function The downloaded file fragments were used to ultimately recover the file.

[0052] An electronic device includes at least one processor and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the storage method described in this application.

[0053] A computer storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the storage method described in this application.

[0054] Compared with the prior art, the beneficial effects of this application are as follows:

[0055] (1) Supports multi-version file-level deduplication DSN. An incremental generation method is introduced to calculate and store the increment between adjacent versions instead of storing the entire new version, which significantly reduces the storage cost and bandwidth usage caused by dynamic file changes.

[0056] (2) To support file indexing, a two-layer DAG blockchain structure is adopted. The lower layer supports operations such as creation, update, merging, and forking, while the upper layer ensures the consistency of the ledger. This structure saves additional storage space for file indexing. Attached Figure Description

[0057] Figure 1 This is a system architecture diagram.

[0058] Figure 2 It serves as the underlying structure of the blockchain.

[0059] Figure 3 It is the upper layer structure of the blockchain.

[0060] Figure 4 This describes the system workflow. Detailed Implementation

[0061] The following detailed descriptions are illustrative and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application.

[0062] like Figure 1 As shown, a decentralized storage system for multi-version files includes a client, a verification node, and a blockchain.

[0063] Verification nodes include storage verification nodes and retrieval verification nodes (of course, it can be understood that a certain verification node is called a storage verification node or a retrieval verification node only when it has storage or retrieval functions at a certain time. The name of the verification node can be adjusted according to its function. That is, when the storage task of the storage verification node is completed, it can act as a retrieval verification node according to the needs of the next task. Similarly, the retrieval verification node also has this attribute).

[0064] The client is responsible for sending file storage, retrieval, or download requests.

[0065] Specifically, the client offers several advantages in improving storage, transmission, and retrieval efficiency, while reducing storage and network transmission costs. During file storage, the client sends file storage requests based on different needs: when there's a need to store the initial file, it sends a file creation request and uploads the initial file to the storage verification node; when there's a need to update the file, it sends a file update request, compares the new file version with the previous version to calculate the file increment, and uploads the file increment to the storage verification node. Compared to a system that uploads the entire file, this significantly reduces the amount of data transmitted over the network, improves transmission efficiency, and reduces storage costs. (For example, if the client needs to upload multiple versions of the same file, i.e....) ,in, This is the initial version, that is, the first version uploaded. , , ..., These are the file versions updated from the previous file version. The client first sends the initial version... Uploaded to any verification node in the system for storage. When upload is required. At this time, the client first calculates the text increment: Then increment the text. Proceed with the upload. At this point, Not stored in its complete form, but rather in... The data is stored in various formats on any selected verification node, thus reducing duplicate saving of the same content and lowering redundancy. Similarly, when uploading... At that time, due to Therefore It is stored in the form of [data], therefore the client calculates [data]. Compared to Text increment: At this point, you only need to upload the text increment. That's it. Similarly, multiple different versions of the same file can be stored as text increments. When there's a need to merge or fork file versions, a file version merge or fork request is sent without any additional content. During file retrieval, the client is responsible for sending retrieval and download requests for the required file version to the retrieval verification node and storage verification node, respectively. After obtaining the list of identifiers for the initial file and file increment needed to recover the file from the retrieval verification node, the client downloads the initial file and file increment from the storage verification node according to the list, and recovers the file version based on the downloaded initial file and file increment. This avoids unnecessary downloads and transmissions, improving the efficiency of file recovery.

[0066] Verification nodes provide file storage and retrieval services to clients and maintain the blockchain ledger.

[0067] Specifically, storage and retrieval functions are handled by different verification nodes, improving the system's scalability and maintainability. The collaboration between storage and retrieval verification nodes also enables the entire system to handle file storage and retrieval tasks more efficiently and reliably, thus providing better service to users. During file storage, the storage verification node is responsible for processing file storage requests sent by clients, storing received files or file increments, or performing file version branch merging or branching operations in the storage space, providing clients with efficient file storage services. Simultaneously, it ensures the security and immutability of file storage by periodically generating storage proofs and storing them on the blockchain. During file retrieval, the storage verification node is responsible for processing download requests sent by clients, sending the initial file or file increment stored in its storage space to the client. During file retrieval, the retrieval verification node is responsible for retrieving the identifier list of the initial file and file increment required for file recovery from the blockchain using a breadth-first search based on the client's retrieval request, and sending it to the client. The breadth-first search method ensures that the obtained identifier list is generated according to file derivation relationships, providing clients with reliable file retrieval services.

[0068] Blockchain uses a two-layer DAG structure, consisting of two layers that share the same set of blocks. Each layer's blocks form a different directed acyclic graph (DAG) structure. The lower layer supports operations such as file creation, updating, merging, and forking, while the upper layer ensures ledger consistency, preventing malicious nodes from tampering with historical records. This structure improves the concurrency of the blockchain and the throughput of the entire network, effectively supporting large-scale transactions and queries, increasing the efficiency of file storage and retrieval, and ensuring blockchain consistency and security. It effectively solves the data security and reliability problems of traditional centralized storage methods.

[0069] Specifically, blockchain can be represented as . and These are the lower and upper edge sets, respectively, and the two layers share the same set of vertices. Each vertex represents a transaction, or a file version. The edges at the lower level are used to describe the derivation relationships between adjacent file versions, and more edges are added at the upper level to ensure consistency.

[0070] Figure 2 This demonstrates an example of the lower layer of the blockchain, where each vertex represents a transaction corresponding to a specific file version, containing an identifier for the initial file or file increment. Each edge represents the derivation relationship between file versions represented by two vertices. The blockchain contains four different types of transactions: create, update, merge, and fork. The latter three transactions describe the derivation relationships between different file versions. The edges in the diagram represent update, merge, or fork operations, respectively.

[0071] When the client stores a new initial file, a creation transaction must be constructed simultaneously. And broadcast it to the blockchain, where This is the initial file. The generated identifier. When the client updates the file, an update transaction must be constructed simultaneously. And broadcast it to the blockchain, where This means that the update transaction is in the file version. Based on, This is the identifier for the file increment. When a client merges files stored on the network, a merge transaction needs to be constructed. And broadcast to the blockchain, where and This represents the two file version branches to be merged. The merge operation does not produce file increments, and only two version branches can be merged at a time. Multiple version branches can be merged by calling the merge operation multiple times. When a client performs a fork operation on files stored on the network, a fork transaction must be constructed. And broadcast it to the blockchain, where This represents the file version branch that will be forked. This represents a set of new file version branches resulting from a fork operation on the file. Each new version contains an empty increment and a new identifier. By forking, different clients can manage their own branches without creating new copies.

[0072] Figure 3 An example of the upper layer of the blockchain is shown. To ensure storage consistency, It is constructed by modifying the DAG-Rider consensus algorithm, where each vertex is associated with a round number. Each validator node broadcasts one transaction per round, i.e., creates a vertex, and each vertex references at least one transaction from the previous round. There are vertices, among which This is the maximum tolerable number of Byzantine nodes. In other words, to enter the [number of nodes]... In the first round, the verification node needs to identify the different verification nodes in the second round. wheel construction The DAG structure constructed in this way has vertices. It has been proven that it can achieve Byzantine atomic broadcasting with strong consistency guarantees.

[0073] Therefore, the entire construction process of a two-layer DAG blockchain with strong consistency guarantees can be summarized as follows: In any round, the incoming transaction first points to the confirmed transaction corresponding to the file version that the file version corresponding to the transaction depends on, according to the derivation relationship (i.e., update, merge, or fork). The constructed edges are included in the lower layer edge set of the blockchain. Then, the incoming transactions are directed to other confirmed transactions from the previous round according to the consensus algorithm, and the constructed edges are included in the upper-layer edge set of the blockchain. middle.

[0074] A decentralized storage method for multi-version files mainly includes four steps: creation, updating, retrieval, downloading, and restoration. Figure 4 As shown, the specific steps are as follows:

[0075] S1, Create:

[0076] S11. Client calculates initial file Unique Identifier The initial file is about to be generated. The data is divided into blocks and hashed. Then, the hashes of the blocks are concatenated and hashed again to obtain the identifier. The identifier, also known as a digital fingerprint, can quickly verify data integrity and prevent file tampering.

[0077] S12. The client selects a storage verification node from the system and includes... File creation request Send it to the storage verification node.

[0078] S13. The storage verification node processes file creation requests sent by clients and simultaneously receives initial files sent by clients. And store it.

[0079] S14. Initial File After the transmission is complete, the client sends a request to create a transaction. Once the transaction is received on the blockchain, the verification node broadcasts the transaction, and after consensus is reached, the transaction is added to the blockchain according to the blockchain construction process.

[0080] S15. The storage verification node periodically initializes the file. Generate storage proofs and send them to the blockchain to support other nodes in verifying the integrity and reliability of the files, preventing files from being tampered with or lost, ensuring secure and reliable file storage, and providing users with a better experience.

[0081] S2, First Update:

[0082] S21. The client uses an incremental mechanism to calculate file versions. With the initial file File increment And calculate file increment. identifier If Size greater than or equal to version Then, the file version will be stored according to the steps of file creation. ;like Smaller than version Then store according to the steps of updating the file. .

[0083] The incremental mechanism performs file-level deduplication; that is, when a file is updated in the system, the unmodified parts of the file are not stored repeatedly, only the modified parts, i.e., the file increment, are stored. Different incremental generation algorithms are used for different types of files in the system: the Myers algorithm is used for incremental calculation of text files, and the BSDiff algorithm is used for incremental calculation of binary files. Because different algorithms are used for incremental calculation of text files and binary files, it is necessary to determine the file type before calculation, that is, to check the contents of the first byte of the file to determine whether it meets the encoding format of a text file or a binary file.

[0084] Myers algorithm: An algorithm for comparing the differences between two files, used in the implementation of the Linux system's `diff` command. It is used when processing two text files. , At this time, the algorithm generates a minimal editing script, which is a sequence of operations consisting of deletion and insertion operations. By executing this sequence, the file can be edited. Convert to file The expected time complexity of this algorithm is... ,in This is the minimum size for editing scripts.

[0085] BSDiff Algorithm: The BSDiff algorithm is a differential update algorithm. This algorithm is widely used to generate patch files for mobile applications. The time complexity of this algorithm is O(n log n). .

[0086] S22. The client selects a storage verification node from the system and includes... File update request Sending to the storage verification node means in the initial file Updated based on the existing data.

[0087] S23. The storage verification node processes file update requests sent by the client and simultaneously receives file increments sent by the client. And store it.

[0088] S24. File Increment After the transmission is complete, the client sends an update transaction. Once the transaction is received on the blockchain, the verification node broadcasts the transaction, and after consensus is reached, the transaction is added to the blockchain according to the blockchain construction process.

[0089] S25. Storage verification nodes periodically increment file sizes. Generate a storage certificate and send it to the blockchain.

[0090] S26. If the number of updates is greater than 1, repeat S21-25. Assuming there are n updates, then at this time... .

[0091] The main difference between updating and creating files is that updating files requires storing file increments instead of the entire file. This mechanism offers several advantages: First, since file increments are typically much smaller than the entire file, less network bandwidth is required for update operations, improving efficiency. Second, using increments reduces storage space requirements, which is particularly beneficial for devices or systems with limited storage capacity. Third, because increments are calculated using a reliable increment generation algorithm based on the previous file version and each increment has its own identifier, the integrity and correctness of each update version are ensured, thus providing more reliable tamper protection. Finally, storing increments facilitates file version tracking and management; users can view previous versions and their corresponding increments at any time, enabling quick and easy rollback and file version comparison.

[0092] S3, Search:

[0093] S31. The client sends a message containing [the following information] to the retrieval verification node. Search request To restore file version .

[0094] S32. Upon receiving the retrieval request, the retrieval verification node first searches in the blockchain. Then call the retrieve function. Get the recovery file version A list of identifiers for all required files and increments.

[0095] Among them, the retrieve function The algorithm process is as follows: from file version Begin breadth-first search, traversing... The edges, all traversed nodes are placed In the array; stop the breadth-first search when a complete version of the file is found, and reverse the search. The order of the array yields a list of identifiers needed to recover the file version.

[0096] S33. Send the list of identifiers to the client. For example... Figure 4 In the example, the list of identifiers contains and .

[0097] S34. After obtaining the list of identifiers, the client sends the search record. To blockchain.

[0098] S4. Download and restore:

[0099] S41. The client sends a download request to the corresponding storage verification node based on the identifier list. For example... Figure 4 In the example, the client sends a download request to the storage verification node. and Download the initial file and file increment .

[0100] S42. After receiving a download request from a client, the storage verification node sends the stored file fragments, i.e., the initial file or the file increment, to the client. The client downloads the required file fragments from the corresponding storage verification node based on the identifier list. Compared to directly downloading the complete file, this reduces download time and network bandwidth usage.

[0101] S43. After the client has finished downloading the required files, it sends the download record. and To blockchain.

[0102] S44. Client calls recovery function The downloaded file fragments were used to ultimately recover the file, ensuring the integrity and correctness of the recovered file.

[0103] Among them, the recovery function The algorithm process is as follows: determine the initial file according to the order of the identifier list; according to the file format, use Myers or BSDiff incremental generation algorithm to sequentially supplement the file increment into the initial file; finally output the complete file.

[0104] This invention tested the storage cost and runtime of the system during the file storage and file retrieval phases, and compared it with two market-tested DSNs, Filecoin and Sia. Filecoin includes two implementations, Venus and Lotus. The tests used three file types: text, multimedia, and binary files. Text files were selected from four GitHub repositories: IPLD, go-ipfs, ccf-deadline, and Git. Multimedia files were selected from multiple versions of a research report's presentation slides. Binary files were selected from the APKs of three popular applications: Minecraft, WeChat, and Netflix.

[0105] The average test results from 100 trials across the eight datasets mentioned above are summarized. It can be seen that compared to Filecoin, this invention saves up to 25% to 99% of storage space because it stores incremental updates rather than complete versions of multiple file types. For text and multimedia files, the file storage and retrieval times of this invention are significantly shorter than other DSNs. While the file storage time may be longer than other DSNs due to limitations in the performance of the incremental generation algorithm for binary files, the file retrieval time is still significantly shorter. Therefore, in practice, the longer file storage time of this invention can be easily offset by the shorter time required for multiple file retrievals.

[0106] Table 1

[0107] An electronic device includes at least one processor and a memory communicatively connected to the processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to execute a decentralized storage method for multi-version files.

[0108] A processor can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processors include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor performs the various methods described above.

[0109] A computer storage medium storing a computer program that, when executed by a processor, implements a decentralized storage method for multiple versions of files.

[0110] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A decentralized storage system for multi-versioned files, characterized in that, The system comprises a client, a verification node and a blockchain; The client is responsible for sending a file storage, retrieval or download request; the client compares a new file version with a previous file version to calculate a file increment, and sends the text increment to the verification node for storage; The verification node provides file storage or file acquisition services to the client; the verification node comprises a storage verification node and a retrieval verification node; The blockchain is a double-layer DAG structure, one layer supports file creation, update, merge and fork operations, and the other layer ensures the consistency of the ledger, and the two layers share the same group of blocks; The blockchain can be represented as wherein and are the edge sets of the lower and upper layers of the blockchain, respectively, is the vertex set shared by the two layers; each vertex represents a transaction, and the transaction type is a create transaction, an update transaction, a merge transaction, or a fork transaction, and different vertices correspond to a specific file version, containing an identifier of an initial file or a file increment .

2. The decentralized storage system of multiple versions of files according to claim 1, characterized in that, The storage verification node selectively performs the following processing according to different needs: The storage verification node is responsible for processing the file storage request sent by the client during the file storage process, storing the received initial file or file increment, or performing file version branch merging or file version branching operation in the storage space, and generating storage proof and storing it on the blockchain at regular intervals; or, The storage verification node is responsible for processing the download request sent by the client in the file acquisition process, and sending the initial file or file increment stored in its storage space to the client; The retrieval verification node is responsible for obtaining the identifier list of the initial file and file increment required for file recovery in the blockchain according to the retrieval request sent by the client in the file acquisition process, and sending it to the client.

3. The decentralized storage system of multiple versions of files according to claim 1, characterized in that, The client sends different requests in the file storage, update and acquisition processes, and generates corresponding transactions after the requests are processed and sends them to the blockchain; the client sends a file storage request according to different needs in the file storage process: When there is a need to store an initial file, a file creation request is sent, and the initial file is uploaded to the storage verification node; or When there is a file update requirement, a file update request is sent, a file increment is calculated by comparing a new file version with a previous file version, and the file increment is uploaded to the storage verification node; Or When there is a file version branch merge or file version fork requirement, a file version merge or file version fork request is sent without additional content; Or The client is responsible for sending a retrieval request and a download request for the required file version to the retrieval verification node and the storage verification node respectively in the file acquisition process, obtains the identifier list of the initial file and file increment required for file recovery from the retrieval verification node, and then downloads the initial file and file increment from the storage verification node according to the list, and is responsible for recovering the file version according to the downloaded initial file and file increment.

4. The decentralized storage system of multiple versions of files according to claim 1, characterized in that, Blockchain lower edge set Each edge in the blockchain lower edge set represents the derivative relationship between the updates, merges or forks of the two file versions corresponding to the two connected vertices; Blockchain upper edge set is constructed by modifying the DAG-Rider consensus algorithm, each verification node broadcasts a transaction in each round, that is, creates a vertex, and each vertex references at least vertices in the previous round, where is the maximum number of Byzantine nodes that can be tolerated.

5. The decentralized storage system of multiple versions of files according to claim 1, characterized in that, The construction process of the blockchain comprises the following four steps: creation, update, retrieval, download and recovery. In any round, the transactions after consensus are first directed to the confirmed transactions they depend on according to the derivation relationship of file versions, at this time the constructed edges are included in the lower layer edge set of the blockchain ; then the incoming transactions are directed to other confirmed transactions in the previous round according to the consensus algorithm, at this time the constructed edges are included in the upper layer edge set of the blockchain .

6. A method for decentralized storage of multi-versioned files, adapted to the system of any of claims 1-5, characterized in that, The creation step is as follows: The update step is as follows: The retrieval step is as follows: The download and recovery step is as follows: S42. The storage verification node receives the download request from the client and sends the stored file segment, i.e. the initial file or file increment, to the client; 7. The method of claim 6, wherein, ​ S11. The client computes an initial file's unique identifier i.e. the initial file is chunked and hashed, and the hashes of the chunks are concatenated and hashed to obtain the identifier ;​​ S12. The client selects a storage verification node from the system and sends a file creation request containing to the storage verification node. ​ S13. The storage verification node processes the file creation request sent by the client, while receiving the initial file sent by the client and stores; S14. Initial file After the transmission is completed, the client sends a creation transaction to the blockchain, the verification node receiving the transaction broadcasts the transaction, and after consensus, the transaction is added to the blockchain according to the blockchain construction process; S15. Store the verification node timing for the initial file A storage proof is generated and sent to the blockchain to support the verification of the integrity and reliability of the file by other nodes.

8. The method of claim 6, wherein, ​ S21. The client computes the file version using the delta mechanism with the initial file and computes the identifier of the file delta ; wherein if the size of the file is greater than or equal to the version , the file version is stored according to the step of creating a file; if the size of the file is less than the version , the file is stored according to the step of updating a file;​​ S22. The client selects a storage verification node from the system and sends a file update request containing to the storage verification node, meaning to update the initial file . ​ S23. The storage verification node processes the file update request sent by the client while receiving the file increment sent by the client and stores; S24. File Increment After the transmission is completed, the client sends an update transaction to the blockchain, the verification node receiving the transaction broadcasts the transaction, and after consensus, the transaction is added to the blockchain according to the blockchain construction process; S25. Store the validation node timing for the file increment A storage proof is generated and sent to the blockchain. 9.The method of claim 6, wherein, ​ S31. The client sends a retrieval request containing to the retrieval validation node to recover a file version ; S32. After the retrieval request is received by the verification node, first find the file version in the blockchain corresponding transaction, then call the retrieval function Get the recovery file version The identifier list of all files and increments required; S33. The retrieval validation node sends the list of identifiers it has acquired to the client, the list of identifiers containing the identifiers of the initial file and file increments needed to recover the file, and ; S34. After the client obtains the list of identifiers, it sends a search record to the blockchain.

10. The method of claim 6, wherein, ​ S41. The client respectively sends a download request to the corresponding storage verification node according to the identifier list, and ; ​ S43. The client sends the download record to the blockchain upon completion of the download of the required file and to the blockchain; S44. The client invokes the resume function with the downloaded file segments to ultimately recover the file.

11. An electronic device, comprising: The apparatus comprises at least one processor, and a memory connected with the processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the storage method of any one of claims 6-10.

12. A computer storage medium having stored thereon a computer program, characterized in that The program is executed by the processor to implement the storage method of any one of claims 6-10.

Images