Unlock instant, AI-driven research and patent intelligence for your innovation.

A pre-reading method and device for distributed storage

A technology of distributed storage, reading and writing methods, applied in the field of pre-reading and writing methods and devices of distributed storage, can solve the problem of increasing the frequency of reading data in a distributed storage system, the system cannot predict where the data is placed, and the efficiency of pre-writing and reading Low and other problems, to achieve the effect of reducing operating pressure, reducing frequent reading frequency, and improving pre-reading efficiency

Active Publication Date: 2020-10-16
GUANGDONG EFLYCLOUD COMPUTING CO LTD
View PDF8 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, after the data uses distributed storage, the continuous data is split into different small data blocks and scattered on different devices, the system often cannot predict where the data that may be read or written next is placed
Therefore, during data pre-reading, distributed storage will continuously access different small data blocks, and then merge small data blocks to form continuous data. This process will greatly increase the frequency of reading data in the distributed storage system, and pre-reading low efficiency

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A pre-reading method and device for distributed storage
  • A pre-reading method and device for distributed storage

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0024] Such as figure 1 As shown, the present invention provides a pre-reading method for distributed storage, the steps include:

[0025] S10, the storage client writes data into the distributed storage system, and the distributed storage system divides the data block into several data blocks, and then dispersely stores the data blocks in different storage devices;

[0026] S11. The storage client continuously reads data blocks from the distributed storage system;

[0027] S12. The statistical prediction module makes statistics on a single data block, counts the data blocks that need to be read thereafter, and calculates the number of times each data block in these data blocks is read, and sorts these data blocks according to the number of times from large to small put in order;

[0028] S13. Step S12 is repeated until the statistics of the data blocks to be read by the storage client are completed, and after the statistical results are formed, they are stored in the statis...

Embodiment 2

[0035] The present invention provides a pre-reading and writing device for distributed storage on the basis of the method in Embodiment 1, such as figure 2 As shown, it includes a storage client, a distributed storage system and a statistical prediction module, and the distributed storage system includes several storage devices;

[0036] The storage client is used to write and read data to the distributed storage system;

[0037] The distributed storage system is used to divide the data into several data blocks, and store the data blocks in different storage devices;

[0038] The statistical prediction module is used for: when the storage client continuously reads data blocks from the distributed storage system, the statistical prediction module performs statistics on a single data block, counts the data blocks that need to be read thereafter, and calculates these The number of times each data block in the data block is read, and these data blocks are sorted according to the...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a pre-reading method and device for distributed storage. The steps of the pre-reading method include: a storage client continuously reads data blocks; The fetched data blocks, calculate the number of times these data blocks are read, and sort them; repeat the calculation of data blocks, complete the statistics of the data blocks that need to be read, and form statistical results; when the storage client needs to read the data blocks next time, For each data block read by the storage client, the subsequent data blocks to be read are predicted by the statistical prediction module; according to the statistical results, the most frequently read data block after each data block, It is predicted that it is the next data block that needs to be read. The present invention improves the pre-reading efficiency of scattered data and reduces the frequent reading frequency of data blocks by effectively predicting the next data block to be read.

Description

technical field [0001] The invention relates to the technical field of distributed storage, in particular to a method and device for pre-reading and writing of distributed storage. Background technique [0002] Distributed storage is a common storage method. Usually, a piece of data content is divided into different small pieces and stored on multiple storage devices. The obvious difference from the centralized storage method is distributed storage. Data is stored in different storage devices. superior. [0003] At present, distributed storage divides a piece of data into several parts according to a fixed size, and then saves these small data blocks scatteredly on the entire cluster device. Usually, in order to reduce the impact of equipment failure, these small data blocks are distributed as widely as possible. However, this decentralized data storage method has a disadvantage, that is, the pre-reading function cannot be well realized. [0004] At present, when data is ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F3/06
CPCG06F3/061G06F3/064G06F3/067
Inventor 麦剑史伟闵宇
Owner GUANGDONG EFLYCLOUD COMPUTING CO LTD