Method and system for removing duplicated files

Abstract

The invention discloses a method and system for removing duplicated files. The method comprises the steps of: S1, generating exclusive file identification information by a client side according to file contents; S2, sending the generated file identification information to an application server; S3, detecting whether the file identification information exists in a database server by the application server; and S4, feeding back detection result information by the application server. In the invention, corresponding file identification information is generated by an identification information generating module I according to the file contents, whether same file identification information exists in the database server is detected by using a detection module arranged on the application server, and short file identification information is only detected but redundant file contents are not detected, therefore, duplicated file removing efficiency is greatly improved, and server resources are greatly saved. Mass data statistics indicate that the duplicated file removing efficiency is improved by 30 percent after the method and system are adopted.

Description

technical field [0001] The invention relates to a method and system for sorting duplicate files, belonging to the field of network communication. Background technique [0002] Storing files is very important for computer networks. By storing useful files in the storage space, users can access and browse them. With the development of society, more and more useful files need to be stored, and the demand for storage space is also increasing. Therefore, an effective method is to sort the files to be stored, that is, to detect whether the storage space contains The file to be stored, if there is, the file will not be stored, if not, it will be stored. [0003] However, in the prior art, the deduplication is carried out by comparing the content of the file to be stored by the user with the content of the file in the storage space, resulting in very low deduplication efficiency, and comparing the content of the file requires a large amount of server resources , especially files w...

AI Technical Summary

Problems solved by technology

[0003] However, in the prior art, the deduplication is carried out by comparing the content of the file to be stored by the user with the content of the file in the storage space, resulting in very low deduplication efficiency, and comparing the content of the file requires a large amount of server resources , especially files with a lot of content and a relatively large space, resulting in a serious waste of server resources

Images