A file storage method and device, electronic equipment and storage medium

Abstract

This application provides a file storage method, apparatus, electronic device, and storage medium. The method includes: receiving chaincode parameters and a file to be processed through a conversion layer of a blockchain server; identifying the chaincode parameters; after confirming that the chaincode parameters include a file identifier, establishing a connection between the conversion layer and the blockchain server, and transmitting the file to be processed to the blockchain server; performing fragmentation on the file to be processed to obtain file fragments; calculating a digest of the file fragments to obtain a fragment hash value for each file fragment; and distributively storing the fragment hash values ​​on the blockchain server. By receiving the chaincode parameters and the file to be processed through the conversion layer of the blockchain server, identifying the chaincode parameters, and establishing a connection with the blockchain server after confirming that the transmitted file is a file, the method enables file storage on the blockchain server, improving file transmission efficiency, eliminating reliance on third parties, and enhancing data security.

Description

Technical Field

[0001] This application relates to the field of blockchain technology, and more specifically, to a file storage method, apparatus, electronic device, and storage medium. Background Technology

[0002] Blockchain, as an enabling technology for value transfer, possesses numerous characteristics such as decentralization, immutability, transparency, full traceability, and security and privacy. Based on openness, blockchains can be categorized into public blockchains, consortium blockchains, and private blockchains. Public blockchain systems are the most open, allowing anyone to participate in the maintenance and retrieval of blockchain data, and applications are easily deployed. Consortium blockchains require registration and permission for access; from a user perspective, participation is limited to consortium members. Private blockchains are the most closed, suitable for internal use by individuals.

[0003] Smart contracts are digital contracts stored on the blockchain that are automatically executed when predetermined terms and conditions are met. Currently, smart contracts cannot upload files; they can only transfer character data of a certain size. This necessitates relying on third-party storage systems, BitTorrent nodes, or third-party file distribution systems to solve the file storage problem, resulting in low file transfer efficiency. Summary of the Invention

[0004] The purpose of this application is to provide a file storage method, apparatus, electronic device, and storage medium. A conversion layer is added to the blockchain server. The conversion layer receives chaincode parameters and the file to be processed, identifies the chaincode parameters, and after determining that the file to be transmitted is a file, establishes a connection with the blockchain server and transmits the file to the blockchain server, thereby improving the file transmission efficiency.

[0005] In a first aspect, embodiments of this application provide a file storage method, comprising: receiving chaincode parameters and a file to be processed through a conversion layer of a blockchain server; identifying the chaincode parameters, and after confirming that the chaincode parameters include a file identifier, establishing a connection between the conversion layer and the blockchain server, and transmitting the file to be processed to the blockchain server; performing fragmentation processing on the file to be processed to obtain file fragments; performing digest calculation on the file fragments to obtain a fragment hash value for each file fragment; and distributing the fragment hash values ​​in the blockchain server.

[0006] In the above implementation process, the blockchain server's conversion layer receives chaincode parameters and the file to be processed, identifies the chaincode parameters, and after confirming that the transmitted file is a file, establishes a connection with the blockchain server and transmits the file to the blockchain server, thus realizing the storage of the file on the blockchain server without relying on a third party and improving the efficiency of file transmission.

[0007] Optionally, in this embodiment of the application, the shard hash value is distributed and stored on the blockchain server, including: storing the shard hash value on the blockchain server through a Merkle tree data structure; the Merkle tree includes leaf nodes and non-leaf nodes; each leaf node stores a shard hash value; the non-leaf nodes store the hash values ​​of the child nodes corresponding to the non-leaf nodes; the child node hash value is obtained by calculating the shard hash value stored in the child nodes corresponding to the non-leaf nodes.

[0008] In the above implementation process, the sharded hash values ​​are stored in the blockchain server using a Merkle tree data structure. The Merkle tree stores the hash values ​​in a hierarchical manner, which can greatly reduce the storage space requirements and achieve efficient storage.

[0009] Optionally, in this embodiment, non-leaf nodes include root nodes; after storing the shard hash value in the blockchain server through a Merkle tree data structure, the method further includes: obtaining the root node hash value stored in the root node of the Merkle tree based on the shard hash value stored in each leaf node of the Merkle tree; and writing the root node hash value into the blockchain server by calling chaincode, so that the file to be processed can be processed on the chain.

[0010] In the above implementation, Merkle trees can verify data integrity solely through the root hash value without needing to know the actual data content, thus protecting data privacy. Furthermore, uploading the data's hash value to the blockchain ensures the data's immutability and traceability.

[0011] Optionally, in this embodiment, after obtaining the root node hash value stored in the Merkle tree, the method further includes: obtaining a pre-set verification hash value; the verification hash value is used to verify the file fragment corresponding to the file to be processed; comparing and verifying the verification hash value and the root node hash value to obtain a file verification result; the file verification result is used to indicate whether the file fragment corresponding to the file to be processed has been tampered with; and writing the root node hash value to the blockchain server by calling the chaincode, including: if the file verification result indicates that the file fragment corresponding to the file to be processed has not been tampered with, writing the root node hash value to the blockchain server by calling the chaincode.

[0012] In the above implementation, the integrity of a large amount of data is verified by comparing a small number of hash values. If the data has not been tampered with, the root node hash value will match the verification hash value, thus confirming data integrity. This eliminates the need to verify the entire dataset, thereby saving computational resources.

[0013] Optionally, in this embodiment of the application, after the file to be processed is segmented to obtain file segments, the method further includes: encoding each file segment to obtain a redundant data block corresponding to each file segment; storing each redundant data block in different storage nodes; the storage nodes include disks or hard disks.

[0014] In the above implementation process, erasure coding technology is used to protect the data, and each redundant data block is stored in different storage nodes to provide redundancy and fault tolerance for the data.

[0015] Optionally, in this embodiment of the application, after storing each redundant data block in different storage nodes, the method further includes: obtaining a data read request; reading redundant data blocks from the storage nodes based on the data read request; if a preset number of redundant data blocks fail to be read, then using the successfully read redundant data blocks to perform data repair processing on the failed redundant data blocks to obtain the repaired data blocks corresponding to the failed redundant data blocks; and using a decoder to decode the successfully read redundant data blocks and the repaired data blocks to obtain the fragmented file corresponding to the file to be processed.

[0016] In the above implementation process, erasure coding technology is used to protect the data. Redundant data blocks are used for data redundancy protection and fault tolerance recovery, which improves data reliability and saves more storage space.

[0017] Optionally, in this embodiment of the application, the process of splitting the file to be processed into file fragments includes: obtaining a preset fragmentation configuration; the fragmentation configuration includes file fragmentation parameters and fragmentation rules; and splitting the file to be processed into file fragments based on the fragmentation parameters and fragmentation rules.

[0018] In the above implementation process, the file to be processed is split into fragments. File fragmentation improves the parallelism of data execution, provides better fault tolerance, and offers more efficient transmission. This is beneficial for data processing and distribution in distributed storage systems.

[0019] Secondly, embodiments of this application also provide a file storage device, comprising: a receiving module, configured to receive chaincode parameters and a file to be processed through a conversion layer of a blockchain server; a transmission module, configured to identify the chaincode parameters, and after confirming that the chaincode parameters include a file identifier, establish a connection between the conversion layer and the blockchain server, and transmit the file to be processed to the blockchain server; a file sharding module, configured to shard the file to be processed to obtain file shards; a digest calculation module, configured to perform digest calculation on the file shards to obtain a shard hash value for each file shard; and a storage module, configured to distribute and store the shard hash values ​​on the blockchain server.

[0020] Thirdly, embodiments of this application also provide an electronic device, including: a processor and a memory, the memory storing machine-readable instructions executable by the processor, which, when executed by the processor, perform the method described above.

[0021] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the methods described above.

[0022] The file storage method, apparatus, electronic device, and storage medium provided in this application receive chaincode parameters and the file to be processed through the conversion layer of the blockchain server, identify the chaincode parameters, and after determining that the transmitted file is a file, establish a connection with the blockchain server and transmit the file to the blockchain server, thereby realizing the storage of the file on the blockchain server, improving file transmission efficiency, eliminating the need for third parties, and improving data security. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A flowchart illustrating a file storage method provided in an embodiment of this application;

[0025] Figure 2 This is a schematic diagram of a Merkle tree structure provided in an embodiment of this application;

[0026] Figure 3 A schematic diagram of the verification Merkle tree provided in the embodiments of this application;

[0027] Figure 4 The consortium blockchain data storage process provided in the embodiments of this application;

[0028] Figure 5 This is a schematic diagram of the structure of a file storage device provided in an embodiment of this application;

[0029] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0030] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this application.

[0032] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0033] Blockchains can be categorized into consortium blockchains, public blockchains, and private blockchains based on access and management permissions. Public blockchains are completely open, giving them the following characteristics: high decentralization due to the absence of a centralized management body, resulting in faster transaction speeds; and transparent transaction records that anyone can view. Private blockchains are accessed only by specific groups, with access managed by a centralized institution. Private blockchains are typically used within enterprises or organizations, such as in finance and healthcare. Consortium blockchains are jointly managed by multiple organizations, with access also managed by a centralized institution. Consortium blockchains are commonly used for collaboration between different organizations, such as in supply chain management and financial services. Because consortium blockchains offer a level of openness between public and private blockchains, they possess the functionality of a distributed ledger while offering higher security than public blockchains, making them widely used.

[0034] Chaincode, also known as smart contracts, is an event-driven program in blockchain technology. It stores state and runs on the blockchain. By pre-setting conditions and rules, it triggers execution under certain events. The ultimate goal of smart contracts is to generate ledger data on the blockchain. Essentially, a smart contract is an automated protocol between the contract creator and recipient, written into the blockchain as code, making it immutable and irreversible.

[0035] Current smart contracts cannot upload files via SDKs (Software Development Kits) or API calls; they can only transfer character data of a certain size via chaincode. Existing methods rely on third-party storage systems, BitTorrent nodes, or third-party file distribution systems for file transfer, resulting in low file transfer efficiency. Furthermore, the need to maintain third-party systems for storage support increases system maintenance costs and complexity.

[0036] The file storage method provided in this application adds a conversion layer to the blockchain server. The conversion layer receives chaincode parameters and the file to be processed, identifies the chaincode parameters, and after determining that the transmitted file is a file, establishes a connection with the blockchain server, transmits the file to the blockchain server, and stores the file. This enables the blockchain system to natively support file storage and distribution, improves file transmission efficiency, eliminates reliance on third parties, enhances data security, and reduces system maintenance costs and complexity.

[0037] Please see Figure 1 The illustrated diagram shows a flowchart of a file storage method provided in an embodiment of this application. The file storage method provided in this embodiment can be applied to a blockchain server, which can be an electronic device within a blockchain system for processing and storing data in the blockchain. The electronic device can include a terminal and a server; the terminal can specifically be a smartphone, tablet, computer, personal digital assistant (PDA), etc.; the server can specifically be an application server or a web server. The file storage method can include:

[0038] Step S110: Receive chaincode parameters and files to be processed through the conversion layer of the blockchain server.

[0039] Step S120: Identify chaincode parameters. After confirming that the chaincode parameters include the file identifier, the conversion layer establishes a connection with the blockchain server and transmits the file to be processed to the blockchain server.

[0040] Step S130: Perform file segmentation on the file to be processed to obtain file segments.

[0041] Step S140: Calculate the digest of the file fragments to obtain the fragment hash value of each file fragment.

[0042] Step S150: Distribute the sharded hash values ​​in the blockchain server.

[0043] In step S110, the blockchain server may include a public blockchain server, a private blockchain server, or a consortium blockchain server. This embodiment uses a consortium blockchain server as an example. The user calls the chaincode through the SDK and simultaneously uploads a file to be processed. The chaincode parameters and the file to be processed are received by the conversion layer of the consortium blockchain server.

[0044] In blockchain, chaincode (smart contracts) is an event-driven program that stores state and runs on the blockchain. Chaincode can be understood as the entry point for data to be uploaded to the chain. Chaincode parameters can include name, version, smart contract code, input parameters, output parameters, state variables, and / or event handling functions. Among these, name and version are the basic attributes of the chaincode, the smart contract code is the core of the chaincode, input parameters, output parameters, and state variables are the inputs and outputs of the chaincode, and event handling functions are the business logic of the chaincode. The file to be processed is the file that needs to be uploaded and stored on the blockchain server.

[0045] Before storing the file to be processed, a conversion layer is added to the consortium blockchain server. The conversion layer receives chaincode parameters and the file to be processed. It identifies the received chaincode parameters and, after recognizing that the pushed data is a file, establishes a connection with the consortium blockchain server and pushes the file to be processed to the consortium blockchain server.

[0046] Adding a transformation layer to the consortium blockchain server can also be done by adding the corresponding functional module code to the consortium blockchain system through patches or plugins. For example, based on the function of the transformation layer, functional code can be written and deployed to the blockchain using smart contracts. The functional module code is then executed by calling the smart contracts.

[0047] In step S120, the file identifier is a character pre-specified by the user to identify a file in the data. If the chain code parameters include the file identifier, the current content is considered a file. For example, the file identifier can be "bigfile". If the file identifier bit "bigfile" in the chain code parameters is 1, it is confirmed that the chain code parameters include a file identifier; if the file identifier bit "bigfile" in the chain code parameters is 0, it is confirmed that the chain code parameters do not include a file identifier.

[0048] After receiving the chaincode parameters, the conversion layer identifies the chaincode parameters and confirms whether the chaincode parameters include a file identifier. If the chaincode parameters include a file identifier, a communication connection is established with the consortium blockchain server through the conversion layer, and the file to be processed is transmitted to the consortium blockchain server.

[0049] The process of establishing a connection between the conversion layer and the consortium blockchain server is as follows: After the conversion layer recognizes that the chaincode parameters include the file identifier, it confirms that what has been received is a file. It then uses a communication protocol to establish a separate connection with the consortium blockchain server to push the file to be processed. The communication protocol may include TCP protocol, SCTP protocol, etc.

[0050] In step S130, the consortium blockchain server uses file fragmentation technology to fragment the file to be processed. File fragmentation is a technique that divides a large file into multiple smaller blocks for transmission over the network. The fragmented file can be fixed-size or variable-size. File fragments can be specifically bytes or characters, etc.

[0051] In step S140, after obtaining the file fragments, the consortium blockchain server uses a digest algorithm to calculate the digest of each file fragment, obtaining the fragment hash value corresponding to each file fragment. Digest algorithms include SecureHash Algorithm or Message-Digest Algorithm 5 (MD5), etc.

[0052] In step S150, the distribution interaction module in the consortium blockchain server stores data using a Distributed Hash Table (DHT). The DHT is used for data notification and retrieval. The distribution interaction module in the consortium blockchain server can be understood as a decentralized storage system. A DHT is a distributed hash table, a distributed storage method. It consists of a series of nodes, each storing a portion of the data. This data is mapped to a key-value pair using a hash function, and then the key-value pair and its corresponding data are stored on the node. When data needs to be retrieved, it is only necessary to search on the corresponding node.

[0053] Decentralized storage systems are a type of distributed storage technology that distributes data across multiple network nodes, similar to the distributed ledger technology of blockchain. Users can contribute their spare hard drive space to form a decentralized network, and these spare hard drive spaces become nodes in the decentralized network. Data stored on the decentralized network is divided into small blocks, encrypted, and then distributed across numerous nodes.

[0054] In the above implementation process, the blockchain server's conversion layer receives chaincode parameters and the file to be processed, identifies the chaincode parameters, and after confirming that the transmitted file is a file, establishes a connection with the blockchain server and transmits the file to the blockchain server, thereby storing the file on the blockchain server, improving file transmission efficiency, eliminating reliance on third parties, and enhancing data security.

[0055] Optionally, in this embodiment of the application, the shard hash value is distributed and stored on the blockchain server, including: storing the shard hash value on the blockchain server through a Merkle tree data structure; the Merkle tree includes leaf nodes and non-leaf nodes; each leaf node stores a shard hash value; the non-leaf nodes store the hash values ​​of the child nodes corresponding to the non-leaf nodes; the child node hash value is obtained by calculating the shard hash value stored in the child nodes corresponding to the non-leaf nodes.

[0056] In its implementation: A Merkle Tree is a data structure used in consortium blockchains for storing and verifying data and transactions. Each node in a Merkle Tree is labeled with the cryptographic hash value of a data block. In a consortium blockchain, a Merkle Tree can be used to verify any type of data stored, processed, and transmitted within and between computers.

[0057] The sharded hash values ​​are stored on the blockchain server using a Merkle tree data structure. A Merkle tree is a hash tree that includes leaf nodes and non-leaf nodes. Each leaf node is labeled with the cryptographic hash value of the data block, while each non-leaf node is labeled with the cryptographic hash value of its child nodes.

[0058] Please see Figure 2 The diagram shown is a schematic diagram of the Merkle tree structure provided in an embodiment of this application.

[0059] Each leaf node in a Merkle tree stores a shard hash value, such as... Figure 2 As shown, the Merkle tree includes four leaf nodes: the first leaf node stores the hash value of the first shard, HashA; the second leaf node stores the hash value of the second shard, HashB; the third leaf node stores the hash value of the third shard, HashC; and the fourth leaf node stores the hash value of the third shard, HashD.

[0060] The non-leaf nodes store the hash values ​​of the child nodes corresponding to the non-leaf nodes; the child node hash values ​​are calculated by taking the shard hash values ​​stored in the child nodes corresponding to the non-leaf nodes. Please continue reading. Figure 2 The fifth and sixth nodes are non-leaf nodes. The fifth node's child nodes include the first and second nodes. The hash value of the child nodes stored in the fifth node is calculated from the first shard hash value HashA and the second shard hash value HashB, denoted as HashAB. The sixth node's child nodes include the third and fourth nodes. The hash value of the child nodes stored in the sixth node is calculated from the third shard hash value HashC and the fourth shard hash value HashD, denoted as HashCD.

[0061] In the above implementation process, the sharded hash values ​​are stored in the blockchain server using a Merkle tree data structure. The Merkle tree stores the hash values ​​in a hierarchical manner, which can greatly reduce the storage space requirements and achieve efficient storage.

[0062] Optionally, in this embodiment, non-leaf nodes include root nodes; after storing the shard hash value in the blockchain server through a Merkle tree data structure, the method further includes: obtaining the root node hash value stored in the root node of the Merkle tree based on the shard hash value stored in each leaf node of the Merkle tree; and writing the root node hash value into the blockchain server by calling chaincode, so that the file to be processed can be processed on the chain.

[0063] For details regarding the implementation process, please refer to: Figure 2 The seventh node is the root node of the Merkle tree, and the root node is a non-leaf node. The root node includes two child nodes, the fifth node and the sixth node. The root node hash value is calculated from the child node hash values ​​HashAB stored in the fifth node and HashCD stored in the sixth node, and is denoted as HashABCD.

[0064] By invoking chaincode, the root node hash value is written to the blockchain server, enabling the file to be processed on-chain. As one implementation, a transaction can be created on a chosen blockchain platform, incorporating the root node hash value as part of the transaction. The transaction can include other information, such as the sender and receiver. The transaction is signed to prove its validity and then broadcast to the blockchain network. The signature ensures the integrity and authenticity of the transaction.

[0065] The transaction is broadcast to the blockchain network, where blockchain nodes verify and package it. The transaction is then packaged into a block, and the block is added to the blockchain, confirming the transaction and completing the on-chain processing of the file to be processed.

[0066] In the above implementation, Merkle trees can verify data integrity solely through the root hash value without needing to know the actual data content, thus protecting data privacy. Furthermore, uploading the data's hash value to the blockchain ensures the data's immutability and traceability.

[0067] Optionally, in this embodiment, after obtaining the root node hash value stored in the Merkle tree, the method further includes: obtaining a pre-set verification hash value; the verification hash value is used to verify the file fragment corresponding to the file to be processed; comparing and verifying the verification hash value and the root node hash value to obtain a file verification result; the file verification result is used to indicate whether the file fragment corresponding to the file to be processed has been tampered with; and writing the root node hash value to the blockchain server by calling the chaincode, including: if the file verification result indicates that the file fragment corresponding to the file to be processed has not been tampered with, writing the root node hash value to the blockchain server by calling the chaincode.

[0068] In the specific implementation process: the verification hash value is pre-generated based on the file to be processed, and it is used to verify the file fragments corresponding to the file to be processed. Since the root node hash value represents the integrity of the entire dataset in a Merkle tree, the file verification result can be obtained by comparing the verification hash value with the root node hash value. Specifically, if the verification hash value and the root node hash value match after comparison, it is considered that the file fragments corresponding to the file to be processed have not been tampered with; if the verification hash value and the root node hash value do not match after comparison, it is considered that the file fragments corresponding to the file to be processed have been tampered with.

[0069] Please see Figure 3 The diagram shown illustrates the verification Merkle tree provided in an embodiment of this application.

[0070] As shown in Figure 3, the leaf nodes of the check Merkle tree store the shard hash values ​​HashA, HashB, HashC, and HashE, respectively. The hash values ​​of the two non-leaf nodes are then HashAB and HashCE, respectively. The check hash value is calculated as HashABCE based on these two non-leaf node hash values.

[0071] As can be seen, the leaf node in the verified Merkle tree and the generated Merkle tree has a different fragment hash value, which leads to the final verification hash value being different from the root node hash value, thus confirming that the file fragment corresponding to the file to be processed has been tampered with.

[0072] If the file verification result indicates that the file fragment corresponding to the file to be processed has not been tampered with, the root node hash value is written to the blockchain server by calling the chaincode, that is, the on-chain processing is carried out while ensuring that the data is complete and has not been tampered with.

[0073] In the above implementation, the integrity of a large amount of data is verified by comparing a small number of hash values. If the data has not been tampered with, the root node hash value will match the verification hash value, thus confirming data integrity. This eliminates the need to verify the entire dataset, thereby saving computational resources.

[0074] Optionally, in this embodiment of the application, after the file to be processed is segmented to obtain file segments, the method further includes: encoding each file segment to obtain a redundant data block corresponding to each file segment; storing each redundant data block in different storage nodes; the storage nodes include disks or hard disks.

[0075] In the specific implementation process: erasure coding technology is used to protect the data. For example, the erasure coding algorithm is used to encode each file fragment to obtain a redundant data block corresponding to each file fragment. Each redundant data block is stored in a different storage node; the storage node includes a disk or hard disk, thus realizing the persistence of the file to be processed.

[0076] In the above implementation process, erasure coding technology is used to protect the data, and each redundant data block is stored in different storage nodes to provide redundancy and fault tolerance for the data.

[0077] Optionally, in this embodiment of the application, after storing each redundant data block in different storage nodes, the method further includes: obtaining a data read request; reading redundant data blocks from the storage nodes based on the data read request; if a preset number of redundant data blocks fail to be read, then using the successfully read redundant data blocks to perform data repair processing on the failed redundant data blocks to obtain the repaired data blocks corresponding to the failed redundant data blocks; and using a decoder to decode the successfully read redundant data blocks and the repaired data blocks to obtain the fragmented file corresponding to the file to be processed.

[0078] In the specific implementation process: when data needs to be read, the original data blocks and redundant data blocks are read from the disk. If a predetermined number of redundant data blocks fail to be read, those redundant data blocks may be faulty or corrupted. Erasure coding decoding algorithms can be used to process these failed data blocks to recover the original data blocks.

[0079] For example, erasure coding creates a mathematical function to describe a set of data, allowing its accuracy to be checked, and if one piece of data is missing, the missing data can be repaired using algorithms such as polynomial interpolation or oversampling.

[0080] In the above implementation process, erasure coding technology is used to protect the data. Redundant data blocks are used for data redundancy protection and fault-tolerant recovery, providing higher data reliability while saving more storage space.

[0081] Optionally, in this embodiment of the application, the process of splitting the file to be processed into file fragments includes: obtaining a preset fragmentation configuration; the fragmentation configuration includes file fragmentation parameters and fragmentation rules; and splitting the file to be processed into file fragments based on the fragmentation parameters and fragmentation rules.

[0082] In the specific implementation process: A preset fragmentation configuration is obtained; the fragmentation configuration includes file fragmentation parameters and fragmentation rules; fragmentation parameters include the size limit of each fragment, which can be determined based on factors such as file size, network bandwidth, or transmission time. Fragmentation rules can include fragments based on fixed size or fragments based on variable size.

[0083] Based on the partitioning parameters and rules, the file to be processed is divided into partitions according to the determined partition sizes. The partitioning process can be implemented using file processing tools or file operation functions provided by programming languages.

[0084] After obtaining the file fragments, a unique identifier or index can be generated for each fragment. The identifier or index can be a number, a hash value, or other unique identifier. The index can be used to identify and retrieve a specific file fragment.

[0085] In the above implementation process, the file to be processed is split into fragments. File fragmentation improves the parallelism of data execution, provides better fault tolerance, and offers more efficient transmission. This is beneficial for data processing and distribution in distributed storage systems.

[0086] Please see Figure 4 The illustrated embodiment of this application provides a consortium blockchain data storage process.

[0087] Users upload a file stream (the file to be processed) while calling the chaincode via the SDK. The conversion layer of the consortium blockchain server receives the chaincode parameters and the file stream. After recognizing that the pushed data is a file, it establishes a connection with the consortium blockchain server and pushes the file to be processed to the consortium blockchain server.

[0088] The file to be processed is segmented, and a hash algorithm is used to calculate the hash value of each segment. The hash algorithm can be SHA256. After data segmentation, the data is distributed through the distribution and interaction module of the consortium blockchain system. When the number of segments reaches a certain scale, a DHT distributed hash table is used to notify and retrieve data. When the scale is not too large, flooding notification is used to improve the speed of interactive distribution.

[0089] The sharded hash values ​​are stored on the blockchain server using a Merkle tree data structure. The root node hash value of the Merkle tree is obtained, and the root node hash value is written to the blockchain server by calling the chaincode.

[0090] Erasure coding technology is used to protect the data, and each redundant data block is stored on a different disk or hard drive to achieve data persistence.

[0091] Please see Figure 5 The diagram shown is a structural schematic of a file storage device provided in an embodiment of this application; this application provides a file storage device 200, including:

[0092] The receiving module 210 is used to receive chaincode parameters and files to be processed through the conversion layer of the blockchain server;

[0093] The transmission module 220 is used to identify chaincode parameters. After confirming that the chaincode parameters include the file identifier, the conversion layer establishes a connection with the blockchain server and transmits the file to be processed to the blockchain server.

[0094] The file fragmentation module 230 is used to perform fragmentation processing on the file to be processed to obtain file fragments;

[0095] The digest calculation module 240 is used to perform digest calculation on file fragments to obtain the fragment hash value of each file fragment;

[0096] Storage module 250 is used to distribute the sharded hash values ​​on the blockchain server.

[0097] Optionally, in this embodiment of the application, the file storage device and storage module are specifically used to store the sharded hash value in the blockchain server through a Merkle tree data structure; the Merkle tree includes leaf nodes and non-leaf nodes; each leaf node stores a sharded hash value; the non-leaf nodes store the child node hash values ​​corresponding to the non-leaf nodes; the child node hash value is obtained by calculating the sharded hash value stored in the child node corresponding to the non-leaf node.

[0098] Optionally, in this embodiment of the application, non-leaf nodes include root nodes; the file storage device further includes: an on-chain module, used to obtain the root node hash value stored in the root node of the Merkle tree based on the shard hash value stored in each leaf node of the Merkle tree; and to write the root node hash value to the blockchain server by calling chaincode, so that the file to be processed can be processed on the chain.

[0099] Optionally, in this embodiment of the application, the file storage device further includes: a verification module, used to obtain a pre-set verification hash value; the verification hash value is used to verify the file fragment corresponding to the file to be processed; the verification hash value and the root node hash value are compared and verified to obtain a file verification result; the file verification result is used to indicate whether the file fragment corresponding to the file to be processed has been tampered with; and the root node hash value is written to the blockchain server by calling the chaincode, including: if the file verification result indicates that the file fragment corresponding to the file to be processed has not been tampered with, the root node hash value is written to the blockchain server by calling the chaincode.

[0100] Optionally, in this embodiment of the application, the file storage device includes a persistent storage module that encodes each file segment to obtain a redundant data block corresponding to each file segment; and stores each redundant data block in different storage nodes; the storage nodes include disks or hard disks.

[0101] Optionally, in this embodiment of the application, the file storage device includes a data repair module, which is used to obtain a data read request; read redundant data blocks from the storage node based on the data read request; if a preset number of redundant data blocks fail to be read, then the successfully read redundant data blocks are used to perform data repair processing on the failed redundant data blocks to obtain the repaired data blocks corresponding to the failed redundant data blocks; and the successfully read redundant data blocks and the repaired data blocks are decoded using a decoder to obtain the fragmented file corresponding to the file to be processed.

[0102] Optionally, in this embodiment of the application, the file storage device includes a file fragmentation module, which is used to obtain a preset fragmentation configuration; the fragmentation configuration includes file fragmentation parameters and fragmentation rules; based on the fragmentation parameters and fragmentation rules, the file to be processed is fragmented to obtain file fragments.

[0103] It should be understood that this device corresponds to the file storage method embodiment described above and is capable of performing the various steps involved in the above method embodiment. The specific functions of this device can be found in the description above, and detailed descriptions are omitted here to avoid repetition. The device includes at least one software functional module that can be stored in memory or embedded in the device's operating system (OS) in the form of software or firmware.

[0104] Please see Figure 6 The diagram shows a structural schematic of an electronic device provided in an embodiment of this application. An electronic device 300 provided in this application includes a processor 310 and a memory 320. The memory 320 stores machine-readable instructions executable by the processor 310. When the machine-readable instructions are executed by the processor 310, the method described above is performed.

[0105] This application also provides a storage medium storing a computer program, which is executed by a processor to perform the above-described method.

[0106] The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0107] It should be understood that the disclosed apparatus and methods can also be implemented in other ways, given the several embodiments provided in this application. The apparatus embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0108] In addition, the functional modules in the various embodiments of this application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0109] The above description is only an optional implementation of the embodiments of this application, but the protection scope of the embodiments of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the embodiments of this application should be covered within the protection scope of the embodiments of this application.

Claims

1. A file storage method, characterized in that, Applications on blockchain server-side, including: The blockchain server receives chaincode parameters and files to be processed through its conversion layer. The chaincode parameters are identified. After confirming that the chaincode parameters include a file identifier, the conversion layer establishes a connection with the blockchain server and transmits the file to be processed to the blockchain server. The file to be processed is segmented to obtain file fragments; Digests are calculated for each file fragment to obtain the fragment hash value of each file fragment; The sharded hash values ​​are distributed and stored on the blockchain server. The step of distributing the sharded hash values ​​in the blockchain server includes: The shard hash value is stored in the blockchain server using a Merkle tree data structure; the Merkle tree includes leaf nodes and non-leaf nodes; each leaf node stores one shard hash value; the non-leaf nodes store the child node hash values ​​corresponding to the non-leaf nodes; the child node hash value is calculated from the shard hash value stored in the child node corresponding to the non-leaf node. The non-leaf nodes include the root node; after storing the sharded hash value in the blockchain server using a Merkle tree data structure, the method further includes: Based on the shard hash value stored in each leaf node of the Merkle tree, obtain the root node hash value stored in the root node of the Merkle tree; By calling the chaincode, the root node hash value is written to the blockchain server so that the file to be processed can be processed on the chain. After obtaining the root node hash value stored in the root node of the Merkle tree, the method further includes: Obtain a pre-set verification hash value; the verification hash value is used to verify the file fragment corresponding to the file to be processed; The verification hash value and the root node hash value are compared and verified to obtain the file verification result; the file verification result is used to indicate whether the file fragment corresponding to the file to be processed has been tampered with. The step of writing the root node hash value to the blockchain server by calling the chaincode includes: If the file verification result indicates that the file fragment corresponding to the file to be processed has not been tampered with, the root node hash value is written to the blockchain server by calling the chaincode; The method further includes: after the conversion layer recognizes that the chaincode parameters include the file identifier, it uses a communication protocol to establish a separate connection with the blockchain server to push the file to be processed.

2. The method according to claim 1, characterized in that, After performing file fragmentation on the file to be processed to obtain file fragments, the method further includes: Each file segment is encoded to obtain a redundant data block corresponding to each file segment; Each of the redundant data blocks is stored in a different storage node; the storage node includes a disk or hard disk.

3. The method according to claim 2, characterized in that, After storing each of the redundant data blocks to different storage nodes, the method further includes: Get data read request; The redundant data block is read from the storage node based on the data read request; If a preset number of redundant data blocks fail to be read, then the successfully read redundant data blocks are used to perform data repair processing on the redundant data blocks that failed to be read, so as to obtain the repaired data blocks corresponding to the redundant data blocks that failed to be read. The decoder is used to decode the successfully read redundant data blocks and the repaired data blocks to obtain the fragmented file corresponding to the file to be processed.

4. The method according to any one of claims 1-3, characterized in that, The step of segmenting the file to be processed to obtain file segments includes: Obtain the preset fragmentation configuration; the fragmentation configuration includes file fragmentation parameters and fragmentation rules; Based on the fragmentation parameters and the fragmentation rules, the file to be processed is fragmented to obtain the file fragments.

5. A file storage device, characterized in that, Applications on blockchain server-side, including: The receiving module is used to receive chaincode parameters and files to be processed through the conversion layer of the blockchain server; The transmission module is used to identify the chaincode parameters. After confirming that the chaincode parameters include a file identifier, the conversion layer establishes a connection with the blockchain server and transmits the file to be processed to the blockchain server. The file sharding module is used to shard the file to be processed to obtain file shards; The digest calculation module is used to perform digest calculation on the file fragments to obtain the fragment hash value of each file fragment; A storage module is used to distribute the sharded hash values ​​in the blockchain server. The storage module is specifically used to store the shard hash value in the blockchain server using a Merkle tree data structure; the Merkle tree includes leaf nodes and non-leaf nodes; each leaf node stores one shard hash value; the non-leaf nodes store the child node hash values ​​corresponding to the non-leaf nodes; the child node hash value is calculated from the shard hash value stored in the child node corresponding to the non-leaf node. The on-chain module is used to obtain the root node hash value stored in the root node of the Merkle tree based on the shard hash value stored in each leaf node of the Merkle tree; and to write the root node hash value into the blockchain server by calling the chaincode, so as to perform on-chain processing on the file to be processed. A verification module is used to obtain a pre-set verification hash value; the verification hash value is used to verify the file fragment corresponding to the file to be processed; the verification hash value and the root node hash value are compared and verified to obtain a file verification result; the file verification result is used to indicate whether the file fragment corresponding to the file to be processed has been tampered with; the step of writing the root node hash value to the blockchain server by calling chaincode includes: if the file verification result indicates that the file fragment corresponding to the file to be processed has not been tampered with, writing the root node hash value to the blockchain server by calling chaincode; After recognizing that the chaincode parameters include the file identifier, the conversion layer uses a communication protocol to establish a separate connection with the blockchain server to push the file to be processed.

6. An electronic device, characterized in that, include: A processor and a memory, the memory storing machine-readable instructions executable by the processor, which, when executed by the processor, perform the method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the method as described in any one of claims 1 to 4.

Images