A cluster-based erasure code archiving method based on data caching

Abstract

The invention discloses an erasure code archiving method based on data cache under a cluster, which comprises the following steps: (1) according to a user access request, read the required data blocks from the production node where it is located to the node buffer, and The data block is sent to the access node, and forwarded to the archiving node according to the archive data status table; (2) the archiving node updates the number of data blocks of each data node according to the received data block; judges whether the archiving node is All the data blocks of the current stripe are received, if so, the current stripe is archived; if not, the cold data block is read from the production cluster, and the stripes are assembled and then archived; the correction provided by the present invention Compared with the common erasure code archiving method, the erasure code archiving method can greatly reduce the number of times the archiving node reads data blocks from the production cluster, without affecting user access. Perform erasure code archiving operations under certain circumstances to improve archiving efficiency.

Description

technical field [0001] The invention belongs to the technical field of computer storage, and more specifically relates to an erasure code archiving method based on data caching under a cluster. Background technique [0002] In today's information age, the amount of data is increasing rapidly, and large-scale storage clusters have been more and more widely used. According to the frequency of data access, the data in the storage cluster can be divided into hot data (Hot Data), warm data (Warm Data) and cold data (Cold Data) in turn. As the system runs longer, the access frequency of hot data will decrease, become warm data, and finally become cold data. Hot data is usually stored in the production cluster in three or more copies. A method to improve the utilization rate of the production cluster system is to migrate the copy data with low access frequency to the erasure code archival cluster (Archival Cluster), which uses RS code to organize the data stored in it; -coded Arc...

AI Technical Summary

Problems solved by technology

[0005] (1) The archiving request is directly sent to the production cluster, and the production cluster is also providing access services for users at this time; the archiving request brings additional external memory access, memory cache occupation, and network resource occupation, resulting in response time for user requests greatly increase;

[0006] (2) For archiving tasks, because nodes in the production cluster need to process user access requests and data archiving requests at the same time, there is resource competition between the two; when a storage cluster node has high frequency of user access requests, the node's The archiving request will not be responded for a long time, causing the archiving node to fail to obtain data blocks in time, which will affect the entire archiving efficiency

Images