A stream data-oriented distributed file system and stream data writing method

Abstract

The invention provides a streaming data oriented distributed file system and a streaming data writing method. The streaming data oriented distributed file system comprises a storage unit, a directorystructure and a file structure, Wherein the directory structure is of a three-layer structure and comprises a file system root directory NetRoot, a file system folder NetDir and a file system file NetFile; The file NetFile structure of the file system is characterized in that the file system is divided into T layers from top to bottom according to a storage day period T; And for any layer, partitioning is carried out according to hours, one partition is carried out every hour, and for any partition, partitioning is carried out according to the number of nodes contained in the function group, and N data blocks are contained. The streaming data oriented distributed file system has the advantages that a relatively simple directory structure and a relatively simple file structure are provided,the functional requirements of streaming data storage, backup and circular coverage can be met at low cost and high efficiency, and retrieval and reading are convenient and rapid.

Description

technical field [0001] The invention belongs to the technical field of data storage, and in particular relates to a stream data-oriented distributed file system and a stream data writing method. Background technique [0002] With the development of information technology and network technology, the scale of data generated is getting larger and larger, a large part of which is streaming data, such as audio data, video data, and Internet of Things acquisition data. This type of streaming data has been continuously generated in industrial and commercial application environments. [0003] At present, the storage of streaming data is mostly in the form of ordinary files. However, due to the continuous generation of streaming data, limited by the limitation of storage space, it needs to be backed up, deleted or overwritten regularly, and is limited by the limitations of ordinary file systems. The file structure requires continuous segmentation of stored files according to time or...

AI Technical Summary

Problems solved by technology

[0003] At present, the storage of streaming data is mostly in the form of ordinary files. However, due to the continuous generation of streaming data, limited by the limitation of storage space, it needs to be backed up, deleted or overwritten regularly, and is limited by the limitations of ordinary file systems. The file structure requires continuous segmentation of stored files according to time or size

Due to the above problems, the efficiency of streaming data storage is low, and it is difficult to effectively manage streaming data

Images