Distributed storage system and data read-write method for same

Abstract

The present application discloses a distributed storage system. The distributed storage system comprises an agent module, a metadata service module and a plurality of storage service modules, wherein each storage service module manages at least one storage node; and the metadata service module is used for maintaining a global first level mapping table, and the storage service module is used for maintaining a second level mapping table. As adoption of a two-level metadata organization, when data is written, if same data exists, actual writing is not carried out, thereby reducing the data writing operation and fulfilling the aim of deleting the global duplicated data. The present application further discloses a corresponding data read-write method. The application scheme can achieve the following technical effects: achieving an online global duplicate data delete in the distributed storage system, thereby reducing the actual storage space; and when writing the data, as the duplicate data does not need the actual writing operation, thereby saving the data communication and bandwidth of the application systems and the storage systems, and improving the storage efficiency.

Description

technical field [0001] The present application relates to the technical field of computer storage, in particular to a distributed storage system and a data reading and writing method thereof. Background technique [0002] With the vigorous development of various applications such as mobile devices, social networks, and the Internet of Things, the data generated by human society has exploded. Traditional disk arrays are increasingly difficult to meet the storage requirements of data-intensive applications based on massive data in terms of capacity, performance and bandwidth. Therefore, a distributed cluster storage system with a scale-out architecture, capacity and performance that can increase linearly with the number of nodes came into being, which can provide higher read and write (IO) operations per second (IOPS, Input / OutputOperationsPerSecond) performance Solid-state drives have gradually replaced traditional disks as the first choice for IO-intensive applications. In...

AI Technical Summary

Problems solved by technology

However, for a distributed cluster storage system, the existing data deduplication technology in a single device cannot achieve the goal of global deduplication, and the data reduction ratio cannot achieve the best effect; in addition, the deduplication method of background processing The operation of data writing cannot be reduced. For storage systems using solid-state drives, the goal of reducing the number of erasures and prolonging the life of solid-state drives cannot be achieved.

Images