Distributed parallel high-speed read-write system and method based on IPFS

A read-write system and distributed technology, applied in file systems, special data processing applications, instruments, etc., can solve problems such as write operation blocking, system availability impact, and system availability cannot be satisfied

Pending Publication Date: 2020-06-12
廊坊嘉杨鸣科技有限公司
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

If strong consistency synchronization is used, the consistency of the storage system is guaranteed. However, when a network or other failure occurs between the primary and backup copies, the write operation will be blocked, and the availability of the system cannot be satisfied.
If asynchronous replication is used, the availability of the storage system is guaranteed, but strong consistency cannot be achieved
[0004] When the data is accessed, the primary copy prov

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed parallel high-speed read-write system and method based on IPFS
  • Distributed parallel high-speed read-write system and method based on IPFS
  • Distributed parallel high-speed read-write system and method based on IPFS

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0027] Such as figure 1 and figure 2 As shown, a distributed parallel high-speed reading and writing system based on IPFS includes: a client, a service control terminal, and several peer-to-peer distributed copy nodes; the service control terminal calculates each copy node according to the operating parameters of each copy node The recommended value of the node, according to the calculated recommended value, establishes the connection between the corresponding replica node with the highest recommended value and the client; after the client establishes a connection with the replica node, the data reading process and the data writing process are completed; The service control terminal calculates the recommended value of each replica node according to the operating parameters of each replica node, and according to the calculated recommended value, establishes the connection method between the replica node with the highest recommended value and the client to perform the following...

Embodiment 2

[0031] On the basis of the previous embodiment, after the client establishes a connection with the replica node, the process of writing data is completed and the following steps are performed: when the client sends data to the node, the node performs fixed-size slice processing on the received data , and generate a unique Hash value for each data block, and at the same time connect these data blocks with the data structure of Merkle directed acyclic graph, and generate a root Hash as the Hash identifier of the file; generate a data block Hash The root Hash algorithm is generated according to the actual content of the file data, and different files will generate different Hash values; after any node completes the writing of the file, it will return to the client, prompting that the data is written successfully; When , multiple nodes will write the task synchronously, and the nodes performing the writing task will slice the file and store it in the local data warehouse with the s...

Embodiment 3

[0036] On the basis of the previous embodiment, after the client establishes a connection with the replica node, the process of reading data is completed and the following steps are performed: the client initiates a read request through the data Hash value, and any node in the distributed storage system Can respond, when the response node has the required data, it will send the data to the client in units of data blocks of the file; when the response node executes the reading task, it will generate a list at the same time and broadcast it to the whole network; The form of the content hash list displays all the read task lists sent by the responding node client. After the node storing the data in the system receives the broadcast, it will directly transmit data with the client; The broadcast task is carried out at the same time, so in the process of the user reading the data, the nodes responding to the reading task gradually increase until the N value is reached, that is, all t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention belongs to the technical field of distributed storage, and particularly relates to a distributed parallel high-speed read-write system and method based on an IPFS, and the system comprises a client, a service control end, and a plurality of peer-to-peer distributed replica nodes. The service control terminal calculates a recommendation value of each copy node according to the operation parameter of each copy node, and establishes connection between the corresponding copy node with the highest recommendation value and the client according to the calculated recommendation value; completing a data reading process and a data writing process after the connection between the client and the copy node is established; the system has the advantages of being high in intelligent degree,high in efficiency and high in safety.

Description

technical field [0001] The invention belongs to the technical field of distributed storage, and in particular relates to a distributed parallel high-speed reading and writing system and method based on IPFS. Background technique [0002] Distributed data processing has become the basis for realizing cloud computing and big data technology. The distributed storage system will deploy multiple clusters. Each cluster generally has a master control node, and the other nodes are working nodes. The master control node is based on the global load. information for overall scheduling. The data in the distributed storage system will be saved as multiple copies. Generally speaking, one of the copies is the primary copy, and the other copies are backup copies. [0003] When data is written, the client sends a write request to the primary copy, and the primary copy performs data backup through a replication protocol (strongly consistent synchronous or asynchronous replication), and notif...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/182G06F16/13G06F16/172
CPCG06F16/137G06F16/172G06F16/1834
Inventor 张兴旺
Owner 廊坊嘉杨鸣科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap