File merging method and device and electronic device

Abstract

The embodiment of the invention discloses a file merging method and device and an electronic device. The file merging method includes the steps that folders which are smaller than first preset files and contained in a target system cluster are determined; files smaller than the first preset files in the folders are determined; the determined files are merged; the determined files are deleted. The file merging method and device and the electronic device in the embodiment are applied, small files are merged and deleted, and under the normal condition, storage space occupied by the merged files are smaller than storage space occupied by files before being merged, and occupying of the storage space is saved; in addition, small files do not exist in a system, file accessing is carried out without a special instruction, then script codes do not need to be modified, occupying of additional system resources is avoided, the file processing result is also not influenced, and occurrence of influencing on the file processing result is avoided.

Description

technical field [0001] The invention relates to the technical field of distributed file system clusters, in particular to a file merging method, device and electronic equipment. Background technique [0002] A distributed file system means that the physical storage resources managed by the file system are not necessarily directly connected to the local node, but are connected to the node through a computer network. The design of the distributed file system is based on the client / server model. A typical network may include multiple servers accessed by multiple users. [0003] Typically, there will be small files in a distributed file system. [0004] However, each small file occupies a block (block), and the metadata of each block is stored in the memory, causing the distributed file system to consume more memory resources in the processing of small files; in addition, due to the existence of small files, the distributed When the file system uses Har files to process small...

AI Technical Summary

Problems solved by technology

[0004] However, each small file occupies a block (block), and the metadata of each block is stored in the memory, causing the distributed file system to consume more memory resources in the processing of small files; in addition, due to the existence of small files, the distributed When the file system uses Har files to process small files, it needs to use a special har command. Using the har command will change the code of the script, and the user needs to modify the changed script code, resulting in the occupation of additional system resources; when using Sequence Files When processing small files, the file stamps of Sequence Files will be irregularly added at the beginning and end of small files, which will affect the results of file processing

Images