Method and system for abnormal reconnection and multiplexing of nodes based on erasure code distributed storage

Abstract

The invention provides a node exception reconnection multiplexing method and system based on erasure code distributed storage. The node exception reconnection multiplexing method comprises the steps:obtaining a data writing state, and recognizing an exception node according to the data writing state; performing state monitoring on the abnormal nodes in the identification result; when it is detected that the abnormal node returns to normal, updating the node state, multiplexing the data writing process of the abnormal node, and sending and storing the position information of the abnormal datasegment in the abnormal period to the corresponding node; and calling corresponding data on other nodes to perform local data restoration on the abnormal node, and writing the restored data into the abnormal region in a covering manner. According to the node exception reconnection multiplexing invention, node equipment is used for multiplexing, so as to enable the local data to be recovered; and under the condition that the node is recovered to be normal after being temporarily abnormal, the data can be continuously written into the node, and the erasure characteristic of the cluster k + m isrecovered, and the influence caused by node abnormality is eliminated through a small amount of data interaction, and the subsequent data recovery pressure is reduced, and the risk of storage space waste is reduced.

Description

technical field [0001] The invention relates to the field of computers and their applications, in particular to a method and system for node abnormal reconnection and multiplexing based on erasure code distributed storage. Background technique [0002] As the market's demand for data scale continues to increase, storage technology is also innovating. In the field of mass data storage, compared to traditional SAN (Storage Attached Network) and NAS (Network-Attached Storage), distributed storage has the advantages of high performance, low cost and strong scalability. The erasure code technology cuts the complete original data into k data strips, and obtains m test strips after the erasure correction algorithm, and the original data can be recovered when the missing strips do not exceed m. Using erasure code fault-tolerant technology instead of multi-copy technology in distributed storage improves the utilization of storage space and further reduces costs while ensuring data s...

AI Technical Summary

Problems solved by technology

[0003] However, during the writing of a group of files, after some node devices are abnormal, even if these node devices can be restored in a short period of time, the corresponding files are often not reused, and the actual storage stripes in the subsequent writing process of this group of files are less than k+m, it is easier to exceed the erasure limit during the period. After exceeding the erasure limit, the group of files will not be writable, and a new file group needs to be applied for, resulting in waste of space

Images