Method for deleting duplicated data in file system in real time

Abstract

The invention relates to a method for deleting duplicated data in a file system in real time, and belongs to the technical field of computer data storage. In a file system establishment stage, a file metadata table, a data block index and a file constitution table are set in an embedded database; and in a file system operation stage, operating commands initiated to the file system by applications are received and responded through a file system driver, and include establishing a new file, writing data to an existing file, reading data from the existing file and deleting the existing data. The method simultaneously supports fixed-length and variable-length file blocking methods, and can delete the duplicated data in the file system in real time, save storage space and improve the utilization efficiency of storage equipment; and the process of deleting the duplicated data is completely transparent to the applications and a user, the file operation of various conventional applications is seamlessly compatible, and almost all negative effects on the user experience are avoided.

Description

technical field [0001] The invention relates to a method for deleting duplicate data in real time in a file system, belonging to the technical field of computer data storage. Background technique [0002] With the rapid development of digital devices, human society is entering the digital age in an all-round way, and the amount of data that needs to be stored is also showing an explosive growth trend. In this context, in order to reduce storage costs and improve the scalability of storage systems, how to store as much data as possible with as little space as possible has become the hottest issue in the storage field. [0003] Data deduplication technology emerged at the beginning of this century and has been widely popularized and applied in recent years. The basic idea of ​​data deduplication can be summarized as follows: Firstly, the files in the storage system are divided into several data blocks, and the hash value of the data block content is used to build an index for...

AI Technical Summary

Problems solved by technology

In the DataDomain system, since each deduplication needs to scan the entire disk, it can only be performed periodically at a lower frequency.

It is not difficult to understand that this non-real-time data deduplication method has the following obvious disadvantages: 1. Data deduplication and data reading and writing operations are performed independently at different stages. In order to support random reading and writing of data in files, the storage system must Save all complete data in the first-level storage stage, but can only play a role in saving storage space in the second-level storage stage of backup and archiving; second, the efficiency of slicing is low, even if only a small part of the data is modified, it is necessary to modify a piece of data. All data in the file or even the disk is re-cut into chunks

Images