Unlock instant, AI-driven research and patent intelligence for your innovation.

Data storage method and data reading method

A technology of data storage and data reading, applied in the direction of electrical digital data processing, input/output process of data processing, instruments, etc., can solve the problem of low reliability of data storage, and achieve the effect of improving the reliability of data storage

Pending Publication Date: 2021-08-27
ALIBABA SINGAPORE HLDG PTE LTD
View PDF7 Cites 4 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] Embodiments of the present invention provide a data storage method and a data reading method to at least solve the problem of data storage reliability by using data nodes and metadata nodes to respectively store data and data metadata in online erasure code technology in related technologies. low technical issues

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data storage method and data reading method
  • Data storage method and data reading method
  • Data storage method and data reading method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0030] According to an embodiment of the present invention, an embodiment of a data storage method is provided. It should be noted that the steps shown in the flowcharts of the drawings can be executed in a computer system such as a set of computer-executable instructions, and, although A logical order is shown in the flowcharts, but in some cases the steps shown or described may be performed in an order different from that shown or described herein.

[0031] The method embodiment provided in Embodiment 1 of the present application may be executed in a mobile terminal, a computer terminal, or a similar computing device. figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for realizing the data storage method is shown. Such as figure 1 As shown, the computer terminal 10 (or mobile device 10) may include one or more (only one is shown in the figure) processor 102 (the processor 102 may include but not limited to microprocessor MCU or programmab...

Embodiment 2

[0068] According to an embodiment of the present invention, there is also provided a data reading method, which is applied to reading data from a plurality of stripe groups in any of the above items, such as Figure 4 shown, including:

[0069] Step S41 , determining a reading position of the data to be read, and determining a reading offset based on the reading position.

[0070] Specifically, the data to be read is stored in multiple data nodes of the distributed storage system by using the data storage method in any one of the above embodiments. Determine the read position of the data to be read, that is, to determine where to read data in multiple stripe groups, for example, to determine to read data at the 13k position of multiple stripe groups, then determine the read offset Can be 13k.

[0071] Step S42, calculating the quotient of the read offset and the size of the first storage area of ​​each stripe group, and determining the sequence number of the second target st...

Embodiment 3

[0093] According to an embodiment of the present invention, a device for implementing the above data storage method is also provided, such as Figure 5 As shown, the device includes:

[0094] The first storage unit 51 is configured to sequentially store the data to be stored in the first storage areas of a plurality of stripe groups, and store the verification data of the stored data in each first storage area to the first storage area of ​​the corresponding stripe group. In the two storage areas, the storage space of each stripe group includes the first storage area and the second storage area.

[0095] The second storage unit 52 is configured to use multiple invalid data stores to fill the remaining space when there is remaining space in the first storage area of ​​the first target stripe group among the multiple stripe groups, and store the multiple invalid data The data amount information of the data is stored in the second storage area in the first target stripe group, w...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a data storage method and a data reading method. The method comprises the steps of storing data to be stored sequentially in first storage areas of a plurality of stripe groups, storing verification data of the stored data in each first storage area in a second storage area of the corresponding stripe group, wherein the storage space of each stripe group comprises the first storage area and the second storage area; and under the condition that residual space exists in the first storage area of the first target stripe group in the plurality of stripe groups, filling the residual space with a plurality of invalid data storages, and storing data volume information of the plurality of invalid data into a second storage area in the first target stripe group. According to the method and the device, the technical problem of low data storage reliability due to the fact that data nodes and metadata nodes are adopted to store data and meta-information of the data respectively in an online erasure code technology in related technologies is solved.

Description

technical field [0001] The present invention relates to the technical field of data access, in particular to a data storage method and a data reading method. Background technique [0002] In the distributed storage system, in order to avoid data damage or loss in the process of data access, online erasure code technology is produced. Compared with multi-copy technology, online erasure code technology can be used without affecting data reliability. Greatly reduce data storage redundancy, thereby greatly reducing storage costs. [0003] Online erasure coding technology is to arrange the storage space of multiple data storage sections in stripes before writing data. Usually, a stripe sets a smaller storage space (such as 64K or 1M), and consists of m original data stripes Encoding generates k verification stripes to form a stripe group, and the client writes data to different data nodes in units of stripe groups. However, in the scenario where the processing file length is no...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F3/06
CPCG06F3/0614G06F3/0619G06F3/0644G06F3/067
Inventor 庄灿伟董元元赵亚飞魏舒展
Owner ALIBABA SINGAPORE HLDG PTE LTD