File processing method, device and system based on MongoDB and server

Abstract

The invention provides a file processing method, device and system based on MongoDB and a server. The method includes: presetting a plurality of data document sets and at least one metadata document set in a MongoDB; after a to-be-stored file is received, detecting that the plurality of data document sets really do not store the to-be-stored file; storing preset attribute information based on theto-be-stored file into a corresponding data document set; updating the current metadata information of the to-be-stored file to a metadata document set. The metadata information is not stored in the memory of the Name Node, the technical problem that the mass small files cannot be stored due to the fact that a Name Node memory cannot store metadata information of the mass small files is solved, meanwhile, through classified storage of the files, storage of the large files can be achieved, and system level expansion can be conveniently achieved by adding a data file set.

Description

technical field [0001] The present invention relates to the field of database application, and more specifically relates to a MongoDB-based file processing method, device, system and server. Background technique [0002] Today, with the rapid development of Internet technology, the data that users can receive during the use of computer equipment has gradually increased exponentially. Simply expanding the storage capacity of computer equipment file systems by increasing the number of hard disks has been unable to meet the current massive file data. storage requirements. [0003] In response to this situation, the distributed file system (Distributed File System, DFS) can effectively solve the problem of data storage. The distributed file system will be fixed in a certain file system in a certain place, and it will be extended to any number of places / multiple places through the Internet. A file system, the multiple nodes formed form a file system network. When users use the ...

AI Technical Summary

Problems solved by technology

[0004] In practical applications, after a distributed file system receives a file to be stored, it usually stores its metadata information in the memory of the Name Node. However, the memory cannot store a large amount of metadata information corresponding to a large number of small files, resulting in distributed The traditional file system cannot realize the storage of massive small files

Images