Quick duplication removal method, and terminal

Abstract

The embodiment of the invention discloses a quick duplication removal method, and a terminal. The method comprises the following steps that: if an uploading instruction of a file to be uploaded is received, according to a preset rule, obtaining a file block from the file to be uploaded; obtaining file block information, wherein the file block information comprises the size, the offset and the MD5(Message Digest 5) value of the file block; sending query information to a server, wherein the query information comprises the size and the file block information of the file to be uploaded; if a first query result returned from the server is received, judging whether the first query result is that the file block which is the same with the file block contents is in the server or not, and retrieving whether the file block which is the same with the contents of the file block is in the presence or not by the server according to the query information so as to obtain a first query result; and if the first query result shows that the file block which is the same with the contents of the file block is not in the presence, uploading the file to be uploaded. The embodiment of the invention also discloses a terminal. By use of the embodiment of the invention, time can be shortened, user experience is improved, and the method and the terminal are suitable for uploading the application scene of abig file on the mobile terminal with weak computation ability.

Description

technical field [0001] The invention relates to the field of network technology, in particular to a fast deduplication method and a terminal. Background technique [0002] In order to prevent different users from storing files with the same content on the cloud, or the same user from storing files with different names but the same content on the cloud, the cloud must identify files with duplicate content by means of deduplication. At present, the commonly used deduplication method is that the client first calculates the MD5 value of the entire file to be uploaded, and then uploads the file length and file fingerprint to the cloud, and the cloud searches according to the file length and MD5 value. For files with the same file length and MD5 value, the user does not need to re-upload, and the cloud does not need to re-store; storage. [0003] However, the current method of weight removal has the following disadvantages: [0004] The logic of the terminal is not strong, that...

AI Technical Summary

Problems solved by technology

[0004] The logic of the terminal is not strong, that is, the MD5 value must be calculated for the entire content of the uploaded file every time, and then the MD5 value and file length are uploaded to the cloud for the cloud to perform weight sorting. Therefore, the current weight sorting method is not suitable For mobile devices with weak computing power, such as mobile phones, if a file of hundreds of megabytes is uploaded, it will take a long time to calculate the MD5 value on the mobile device and affect the user experience

Images