File deduplication method for terminal device, terminal device and storage medium

Abstract

The embodiment of the application discloses a file deduplication method for terminal equipment, a terminal equipment and a storage medium, which are used for eliminating redundant files generated by a user when using the terminal equipment and reducing the time required by the deduplication process. The method comprises the following steps: obtaining a write request; obtaining a foreground hash value and metadata of a write page according to the write request; obtaining idle information of the terminal equipment and determining a trigger timing; when the trigger timing is met, calculating a file identical to the foreground hash value, and outputting inode information of the file identical to the foreground hash value; calculating a background hash value according to the foreground hash value, the metadata and the inode information; and using a hard link method to deduplicate the same file in the background hash value.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of terminal equipment, in particular to a file deduplication method for terminal equipment, terminal equipment and storage medium. BACKGROUND

[0002] According to research, a deep user of a smart phone will generate a considerable amount of redundant files. At present, most of the smart phone deduplication methods are performed in the background. These deduplication programs will detect the existing redundant files through full-disk scanning of the smart phone, and then provide the user for elimination.

[0003] Nowadays, smart phones will encrypt files for security considerations. Since Android 7.0, smart phones mostly support the method of FBE (File-based Encryption). The advantage of this encryption method is that one file has one key. If the same file deduplication is performed using the full-disk scanning method, each file in the smart phone needs to be scanned, and a decryption process needs to be performed for each file, which causes a large time and system resource overhead. SUMMARY

[0004] The embodiments of the present application provide a file deduplication method for terminal equipment, terminal equipment and storage medium, which is used to eliminate the redundant files generated by the user when using the terminal equipment, and also reduces the time required by the deduplication process.

[0005] The first aspect of the present application provides a file deduplication method for terminal equipment, which is applied to terminal equipment, and can include:

[0006] Obtaining a write request;

[0007] According to the write request, calculating the foreground hash value and metadata of the write page through the foreground;

[0008] Obtaining the idle information of the terminal equipment, and determining the triggering time;

[0009] When the triggering time is met, calculating the files with the same foreground hash value, and outputting the inode information of the files with the same foreground hash value;

[0010] According to the foreground hash value, the metadata and the inode information, calculating the background hash value;

[0011] Using the hard link method to deduplicate the same files in the background hash value.

[0012] The second aspect of the present application provides terminal equipment, which can include:

[0013] acquire a write request;

[0014] The processing module is configured to calculate a foreground hash value and metadata of a write page according to the write request, acquire idle information of the terminal device, determine a triggering time, calculate files identical to the foreground hash value when the triggering time is met, output inode information of the files identical to the foreground hash value, calculate a background hash value according to the foreground hash value, the metadata, and the inode information, and use a hard link method to remove redundancy for the files identical in the background hash value.

[0015] The third aspect of the present application provides a terminal device, comprising:

[0016] a memory storing executable program codes;

[0017] a processor coupled to the memory;

[0018] The processor is configured to perform the method according to the first aspect of the present application.

[0019] Another aspect of the embodiments of the present application provides a computer readable storage medium comprising instructions which, when executed on a processor, cause the processor to perform the method according to the first aspect of the present application.

[0020] Another aspect of the embodiments of the present application discloses a computer program product which, when executed on a computer, causes the computer to perform the method according to the first aspect of the present application.

[0021] Another aspect of the embodiments of the present application discloses an application publishing platform configured to publish a computer program product which, when executed on a computer, causes the computer to perform the method according to the first aspect of the present application.

[0022] From the above technical solutions, the embodiments of the present application have the following advantages:

[0023] In the embodiments of the present application, a write request is acquired, a foreground hash value and metadata of a write page are calculated according to the write request, idle information of a terminal device is acquired, a triggering time is determined, files identical to the foreground hash value are calculated when the triggering time is met, inode information of the files identical to the foreground hash value is output, a background hash value is calculated according to the foreground hash value, the metadata, and the inode information, and a hard link method is used to remove redundancy for the files identical in the background hash value. The method is used to eliminate redundant files generated by a user when using a terminal device, and also reduces the time required by a redundancy removal process. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the present application, the following will briefly introduce the drawings needed to be used in the embodiments and prior art description. Obviously, the drawings described below are only some of the embodiments of the present application, and other drawings can also be obtained based on these drawings.

[0025] Figure 1 An architecture diagram about file reading and writing in the prior art;

[0026] Figure 2 An embodiment diagram of a file deduplication method about a terminal device in the embodiments of the present application;

[0027] Figure 3A An architecture diagram about file reading and writing applied in the embodiments of the present application;

[0028] Figure 3B An diagram of calculating a foreground hash value and a background hash value in the embodiments of the present application;

[0029] Figure 4 Another embodiment diagram of a file deduplication method about a terminal device in the embodiments of the present application;

[0030] Figure 5 A diagram of judging whether a file is a hard link file for reading and writing in the embodiments of the present application;

[0031] Figure 6 An embodiment diagram of a terminal device in the embodiments of the present application;

[0032] Figure 7 Another embodiment diagram of a terminal device in the embodiments of the present application. DETAILED DESCRIPTION

[0033] The embodiments of the present application provide a file deduplication method about a terminal device, a terminal device and a storage medium, which are used to eliminate redundant files generated by a user when using the terminal device, and also reduce the time required by the deduplication process.

[0034] In order to make the personnel in the technical field better understand the present application, the technical solutions in the embodiments of the present application will be described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all should belong to the scope of protection of the present application.

[0035] Since the deduplication procedure involves the file read-write process, the scenario involves the background application (Application, App) application and the background virtual File System (VFS), disk cache, Block Input / Output (Block I / O) layer, and hardware storage. After the App sends a read request, the VFS layer responds to the application and first determines whether it is a normal read-write or a Direct I / O mode. If it is a normal read-write mode, the VFS layer first searches for the file page in the disk cache, and if it is found, it is directly returned. If the required file page is not found, it needs to be read from the underlying disk. Using the Direct I / O mode will bypass the disk cache and directly search for the file in the disk. The file writing process is similar to file reading. After the VFS layer obtains the system response, it first writes the file to the disk cache (using the Direct I / O mode directly writes to the disk), and then writes to the disk. As shown in Figure 1 , it is an architecture diagram of the prior art about file reading and writing.

[0036] Since Android 7.0, the smart phone supports FBE encryption, and the encryption occurs between the VFS layer and the Block I / O layer, as shown in Figure 1 ① and ②. When reading the file in ①, the data read from the disk is first decrypted and then saved to the disk cache. When writing the file in ②, the data in the disk cache is first encrypted and then saved to the disk.

[0037] As shown in Figure 1 , the plaintext data in the disk cache is mainly read to calculate the foreground hash value, and the foreground hash value and some metadata are saved. The metadata includes the file name and inode number (as shown in Figure 1 ③); the same file is matched by reading the saved file in the foreground, and a hash calculation is performed again to confirm whether it is matched (as shown in Figure 1 ④); and the redundant file is hard linked to the target file.

[0038] The following is an embodiment of the present application. As shown in Figure 2 , it is an embodiment diagram of the file deduplication method of the terminal device in the embodiment of the present application, which can include:

[0039] 201, a write request is obtained.

[0040] As shown in Figure 3AAs shown, an architecture diagram about file reading and writing applied by the embodiment of the present application is shown. In Figure 3A As shown, the disk cache includes module 1 for foreground hash value calculation; the App application includes module 2, module 3 and module 4, module 2 is used for deduplication triggering program; module 3 is used for background hash program; and module 4 is used for link deduplication.

[0041] For example, a series of write requests are generated when the user uses the App, and the write requests are sent to the virtual file system VFS.

[0042] 202. According to the write request, the foreground hash value and the metadata of the write page are calculated by foreground calculation.

[0043] For example, the VFS transmits the write request into the disk cache, the disk cache inputs the write request transmitted by the VFS, and module 1 calculates the hash value of the write page according to the write request, and outputs the foreground calculated foreground hash value and part of the metadata.

[0044] Optionally, the terminal device calculates the foreground hash value and the metadata of the write page by foreground calculation according to the write request, which can include that the terminal device calculates the foreground hash value and the metadata of the write page by foreground using Alder32 method according to the write request.

[0045] Optionally, the terminal device calculates the foreground hash value and the metadata of the write page by foreground calculation according to the write request, which can include that the terminal device calculates the foreground hash value and the metadata of the last page of the target file by foreground calculation according to the write request.

[0046] For example, in module 1, the write request transmitted by the VFS is input, and since module 2 is in the reading and writing process, the influence on the reading and writing process needs to be considered, too long time consumption will cause the user experience to be poor, so the accuracy needs to be sacrificed and the hash method with weak strength is used, and the method of Alder32 is used in the present application. In addition, since the role of the foreground is only to quickly exclude different files, the hash calculation of the entire file is not required, and only the last page of the file needs to be calculated to reduce the calculation amount of the foreground hash.

[0047] It can be understood that in the embodiment of the present application, the contents of multiple files need to be compared in the deduplication process, and the hash (Hash) method is used for calculation. But according to different functions to be implemented, it can be divided into foreground hash calculation and background hash calculation, which can also be called foreground hash value calculation and background hash value calculation.

[0048] The foreground hash calculation is briefly described as follows:

[0049] The main function of foreground hashing is to quickly calculate and save the hash value before a file is written to disk, so that different files can be quickly eliminated when further compared in the background. Since foreground hashing occurs in the kernel, the following methods are used to reduce the computational burden on the kernel:

[0050] (1) Granularity of hash calculation

[0051] When performing hash calculations in the foreground, the calculation granularity is designed to be page-level. This is because file reading and writing in the kernel are performed at the page level, allowing for real-time calculations during file writing. Furthermore, to further reduce computational overhead, the design does not perform hash calculations on every single page of the file, as the purpose of foreground hashing is to quickly eliminate different files; therefore, only the last page needs to be calculated.

[0052] (2) Hash Algorithm Selection

[0053] As mentioned above, since the ultimate goal of the front end is to be fast and not to incur too much computational overhead on the kernel, traditional cryptographic hash methods (such as MD5, SHA256, etc.) will not be chosen for the hash algorithm. Although such hash methods have a low collision rate and high accuracy, their computational overhead is too high. Therefore, a checksum method such as Alder32 will be considered.

[0054] 203. Obtain the idle information of the terminal device and determine the triggering time.

[0055] Optionally, the process of the terminal device acquiring idle information and determining the triggering timing may include: the terminal device acquiring idle information and determining the triggering timing based on a decision tree method.

[0056] Optionally, the process of the terminal device acquiring idle information and determining the triggering timing may include: the terminal device acquiring idle information and determining the triggering timing based on the CART classification tree algorithm.

[0057] Optionally, the idle information of the terminal device includes: the current number of processes, whether the screen is off, the screen-off time, the battery level, and whether it is charging.

[0058] For example, module 2 in the app takes into account the system's idle information (including, for example, the number of current processes, whether the screen is off and the screen-off time) and battery level (whether it is charging), determines whether to start deduplication using a decision tree method, and outputs the information after the decision, i.e. the trigger timing.

[0059] It can be understood that the module 2 inputs the idle information of the system, and the module 2 triggers the background deduplication program to trigger the deduplication program at a suitable time. In the present application, the method of decision tree is used to train the idle information of the terminal device system (including, for example, the number of current processes, whether the screen is off, the screen-off time, etc.) and the battery power (whether charging) to generate a decision tree and output a suitable trigger time.

[0060] 204、According to the trigger time, the file with the same foreground hash value is calculated, and the inode information of the file with the same foreground hash value is output.

[0061] Optionally, the terminal device calculates the file with the same foreground hash value according to the trigger time, and outputs the inode information of the file with the same foreground hash value, which can include that the terminal device calculates the file with the same foreground hash value and outputs the inode information of the file with the same foreground hash value when the trigger time is met.

[0062] It can be understood that the module 3 has two input information, (1) input the trigger time information of the module 2, and start deduplication if the trigger condition is met; (2) input the foreground hash value calculated by the module 1.

[0063] Exemplarily, the module 3 is a background deduplication program, which calculates the file with the same foreground hash value according to the trigger time, and outputs the inode information of the file with the same foreground hash value.

[0064] 205、According to the foreground hash value, the metadata and the inode information, the background hash value is calculated.

[0065] The terminal device calculates the background hash value according to the foreground hash value, the metadata and the inode information, which can include that the terminal device calculates the background hash value using MD5 or SHA256 method according to the foreground hash value, the metadata and the inode information.

[0066] Optionally, the method can further include that the terminal device records the background hash value of the file which is not the same in the background hash value, and directly compares the file when the foreground hash value is calculated next time.

[0067] Exemplarily, the module 3 further calculates the background hash value of the file with the same foreground hash value, judges whether they are the same, and outputs the inode information of the file with the same background hash value.

[0068] It can be understood that the module ③ obtains the files with the same foreground hash value by matching the result of the foreground hash value, and then performs a new round of background hash value calculation with higher strength and lower collision rate on the files with the same foreground hash value to confirm whether the files are the same. If it is found through the background hash value calculation that the files are the same, it is confirmed that the file is a redundant file for deduplication, otherwise the background hash value of the file needs to be recorded, and when there is a file with the same foreground hash value next time, the background hash value of the target file does not need to be calculated and can be directly compared. As shown in Figure 3B , it is a schematic diagram of calculating the foreground hash value and the background hash value in the embodiment of the application.

[0069] The background hash value calculation is briefly described below:

[0070] For example, after obtaining the data through the foreground and determining that the foreground hash values of two files are the same, it is further needed to determine whether the contents of the two files are the same.

[0071] (1) Hash calculation granularity and algorithm selection

[0072] Unlike the foreground hash, the background hash pursues higher accuracy. Therefore, the hash granularity of the background calculation is file granularity, that is, the hash calculation is performed on the entire file. At the same time, the hash algorithm no longer uses the checksum method but uses the encryption type hash method with lower collision rate, that is, the MD5 method and the like, because the role of the background is to perform secondary confirmation on the foreground, and a more accurate method needs to be used.

[0073] (2) Background deduplication triggering time

[0074] Since the background hash needs to consume a certain amount of time, the triggering time of the background hash can affect the user experience, and therefore the triggering time of the background deduplication needs to be managed. Here, the decision tree method is used. The decision tree method is used to train the idle information of the mobile phone system (including the number of current processes, whether the screen is off and the screen-off time) and the battery power (whether charging) to generate a decision tree to determine whether to perform deduplication. In this scheme, the CART classification tree algorithm is selected. The CART classification tree method uses the Gini coefficient instead of the information gain ratio. The Gini coefficient represents the impurity of the model, and the smaller the Gini coefficient, the lower the impurity and the better the feature.

[0075] Suppose there are K categories, and the probability of the Kth category is p k , and the Gini coefficient of the probability distribution is expressed as:

[0076]

[0077] Since this scheme only has two categories—those that trigger redundancy removal and those that do not—it is equivalent to a binary classification problem. The expression for the Gini coefficient of the probability distribution can be simplified to:

[0078] Gini(p) = 2p(1-p)

[0079] Where p is the probability of triggering redundancy removal.

[0080] CART selects the attribute that minimizes the GINI value of its child nodes as the splitting scheme. That is, minimizing (the classification tree):

[0081]

[0082] Assume that the following five attributes are defined in this scheme: current number of processes (greater than / less than 10), screen off (yes / no), screen off time (greater than / less than 10 seconds), battery level (greater than / less than 50%), and charging (yes / no). For each feature, a Gain value is calculated. The feature with the smallest Gain value is selected as the optimal feature, and this process is repeated to train a decision tree.

[0083] 206. For files with the same background hash value, use hard linking to remove redundancy.

[0084] For example, in module 4, file information with the same background hash value issued by input module 3 is deduplicated using a hard link method. That is, files with the same content are linked together using hard links to achieve deduplication.

[0085] Optionally, the terminal device may calculate the front-end hash value and metadata of the page to be written based on the write request, which may include: the terminal device calculating the front-end hash value, metadata and user tag of the page to be written based on the write request.

[0086] The terminal device calculates the background hash value based on the front-end hash value, the metadata, and the inode information. This may include: when a file with the same front-end hash value belongs to the same user, the terminal device calculates the background hash value based on the front-end hash value, the metadata, and the inode information.

[0087] It can be understood that the application also solves the problem of privacy protection after deduplication using hard links. Although deduplication using hard links greatly reduces the memory occupation, because the hard link file itself does not occupy memory, multiple hard link files share the same inode. This will cause security problems, for example, there may be multiple users using the same App, if there are redundant files between multiple users, according to the above method, the redundant files will be eliminated, so the files of multiple users will use the same inode, causing the problem of data leakage. In view of this problem, a user tag is introduced during deduplication, and the role of the tag is to identify the current user with the help of the operating system.

[0088] For example, the user issues a write request; the operating system identifies the current user after receiving the write request, and adds a user tag to the write file; the foreground performs hash calculation on the write file, and saves the hash value and the user tag; the idle information of the system (including the number of current processes, whether the screen is off, and the screen-off time) and the battery power (whether charging) are input, and the decision tree method is used to determine whether to start deduplication, and the trigger timing information after decision is output; after the deduplication program is successfully triggered through the trigger timing, before further calculating the background hash value of the same foreground hash value file, it is first determined whether they belong to the same user, if the users are different, the background hash value is not calculated. If the users are the same, the inode information of the same file is output; several files with the same file content are linked using hard links to achieve deduplication.

[0089] In the embodiment of the present application, a write request is acquired; according to the write request, a foreground hash value and metadata of a write page are calculated through the foreground; idle information of a terminal device is acquired to determine a triggering occasion; in the case of meeting the triggering occasion, a file identical to the foreground hash value is calculated, and inode information of the file identical to the foreground hash value is output; a background hash value is calculated according to the foreground hash value, the metadata and the inode information; and a hard link method is used to deduplicate the file identical in the background hash value. The present technical solution is used to eliminate the redundant files generated by a user when using a terminal device, and also reduces the time required by the deduplication process. The present technical solution is characterized in that the deduplication method of an Android terminal device in an encrypted case is optimized, and a foreground and a background are combined. In this process, the following beneficial effects are achieved: (1) the deduplication program is used, and the most significant benefit is to reduce the redundant files occupying the memory of the mobile phone; (2) through the cooperation of the foreground and background hash values, the foreground hash value can quickly remove different files and locate the same file, greatly reducing the number of files that need to be scanned in the background. The time required by the deduplication process is greatly reduced. Moreover, only a lightweight hash function is used in the foreground, and the calculation overhead of the foreground is not too large. The embodiment of the present application can eliminate the redundant files generated by a user when using a smart terminal device (such as forwarding the same picture, file, etc.).

[0090] As shown in Figure 4 , it is another embodiment schematic diagram of the file deduplication method of a terminal device in the embodiment of the present application, which can include:

[0091] 401, a write request is acquired.

[0092] 402, according to the write request, a foreground hash value and metadata of a write page are calculated through the foreground.

[0093] 403, idle information of a terminal device is acquired to determine a triggering occasion.

[0094] 404, according to the triggering occasion, a file identical to the foreground hash value is calculated, and inode information of the file identical to the foreground hash value is output.

[0095] 405, a background hash value is calculated according to the foreground hash value, the metadata and the inode information.

[0096] 406, a hard link method is used to deduplicate the file identical in the background hash value.

[0097] It should be noted that steps 401-406 in the embodiment of the present application are similar to steps 201-206 shown in Figure 2 , and will not be described here.

[0098] It can be understood that after the matching in the foreground and the background is completed and it is confirmed that the two files have the same content, the two files need to be de-duplicated. When de-duplication is performed, a hard link method (using the command ln-f) is used to connect the two files. The command is used to delete the current file and then generate a hard link file pointing to the target file, so that the deletion of the redundant file is realized.

[0099] For the de-duplicated file, only one backup of the file is left, and when there are multiple users reading and writing the same file, the problem of asynchronous update may be caused. For example, there is a de-duplicated file file, and user process A reads and writes the file, and user process B simultaneously performs a write operation on the file, which may cause A to read the file file before B modifies it or read the file file' after B modifies it. To solve this problem, the present application makes certain modifications to the reading and writing operations of the file, as shown below.

[0100] 407、Detecting a read / write request of a user to a first file.

[0101] For example, user A issues a read / write request to file A, and the terminal device detects the read / write request of user A to file A.

[0102] 408、According to the read / write request, judging the attribute i_nlink of the first file inode.

[0103] For example, after receiving the read / write request, file A judges the i_nlink attribute of the file inode and outputs the judgment result (Y / N).

[0104] 409、If the attribute i_nlink=1, it is determined that the first file is a non-hard link file, and the first file is read / written.

[0105] For example, as shown in Figure 5 , it is a schematic diagram for judging whether the file is a hard link file for reading and writing in the embodiment of the present application. If the attribute i_nlink=1, it means that the file A is a non-hard link file, and the file A is directly transmitted to user A. As shown in Figure 5 , the reading / writing process is completed.

[0106] 410、If the attribute i_nlink>2, it is determined that the first file is a hard link file, a temporary file of the first file is generated, and the temporary file is read / written.

[0107] ​Optionally, if the attribute i_nlink>2, the first file is determined as a hard link file, a temporary file of the first file is generated, and the temporary file is read / written, which can comprise: if the attribute i_nlink>2, the first file is determined as a hard link file, a temporary file of the first file is generated, and the temporary file is read / written, if the operation is reading, the temporary file is deleted, and if the operation is writing, the temporary file is saved.

[0108] As shown in FIG. 2, if the attribute i_nlink>2, it indicates that the file is a hard link file, and then the process proceeds to the step shown in FIG. 2②. Figure 5 As shown in FIG. 2, if the attribute i_nlink>2, it indicates that the file is a hard link file, and then the process proceeds to the step shown in FIG. 2②.

[0109] In the embodiment of the present application, a write request is acquired, and a foreground hash value and metadata of a write page are calculated according to the write request. Idle information of a terminal device is acquired, and a trigger occasion is determined. In the case of meeting the trigger occasion, a file with the same foreground hash value is calculated, and index node information of the file with the same foreground hash value is output. A background hash value is calculated according to the foreground hash value, the metadata, and the index node information. The same file in the background hash value is removed by using a hard link method. The technical scheme is used for eliminating redundant files generated by a user when using a terminal device, and also shortens the time required by a deduplication process. The technical scheme is characterized in that a deduplication method for an Android terminal device in an encrypted case is optimized, and a foreground and a background are combined. In the process, the following beneficial effects are achieved: (1) the deduplication program is used, and the most significant benefit is to reduce the redundant files occupying the memory of the mobile phone. (2) by calculating the foreground and background hash values, the foreground hash value can quickly remove different files and locate the same file, greatly reducing the number of files that need to be scanned in the background. The time required by the deduplication process is greatly shortened. Moreover, only a light hash function is used in the foreground, and the calculation cost of the foreground is not high. The embodiment of the present application can eliminate the redundant files generated by the user when using the smart terminal device (such as forwarding the same picture, file, etc.).

[0110] Optionally, a large number of same files can be copied in a competitor device, and then the storage space usage is observed. If the storage space does not decrease accordingly, it can be judged that there is infringement.

[0111] As shown in FIG. 2, if the attribute i_nlink>2, it indicates that the file is a hard link file, and then the process proceeds to the step shown in FIG. 2②. Figure 6 As shown in FIG. 2, if the attribute i_nlink>2, it indicates that the file is a hard link file, and then the process proceeds to the step shown in FIG. 2②.

[0112] The acquisition module 601 is configured to acquire a write request.

[0113] The processing module 602 is configured to calculate a foreground hash value and metadata of a write page according to the write request, acquire idle information of a terminal device, determine a trigger occasion, calculate a file identical to the foreground hash value in a case where the trigger occasion is met, and output inode information of the file identical to the foreground hash value, calculate a background hash value according to the foreground hash value, the metadata, and the inode information, and use a hard link method to remove redundancy for the file identical in the background hash value.

[0114] Optionally, the processing module 602 is specifically configured to calculate a foreground hash value and metadata of a write page by foreground using an Alder32 method according to the write request.

[0115] Optionally, the processing module 602 is specifically configured to calculate a foreground hash value and metadata of a last page of a target file by foreground according to the write request.

[0116] Optionally, the processing module 602 is specifically configured to calculate a background hash value by using an MD5 or SHA256 method according to the foreground hash value, the metadata, and the inode information.

[0117] Optionally, the processing module 602 is specifically configured to acquire idle information of a terminal device and determine a trigger occasion according to a CART classification tree algorithm.

[0118] Optionally, the processing module 602 is further configured to detect a read / write request of a first file by a user, judge an attribute i_nlink of an inode of the first file according to the read / write request, determine that the first file is a non-hard link file if the attribute i_nlink=1, and perform read / write on the first file, and determine that the first file is a hard link file if the attribute i_nlink>2, generate a temporary file of the first file, and perform read / write on the temporary file.

[0119] Optionally, the processing module 602 is specifically configured to calculate a foreground hash value, metadata, and a user tag of a write page by foreground according to the write request.

[0120] The processing module 602 is specifically configured to calculate a background hash value according to the foreground hash value, the metadata, and the inode information in a case where a file identical to the foreground hash value belongs to a same user.

[0121] Optionally, the processing module 602 is further configured to record a background hash value of a file not identical in the background hash value, and directly compare the file not identical when a foreground hash value of a file is calculated next time.

[0122] As Figure 7 shown, it is another embodiment of the terminal device in the embodiments of the present application, which can include:

[0123] Figure 7 It is a block diagram showing part of the structure of a mobile phone related to the terminal device provided by the embodiments of the present application. Referring to Figure 7 , the mobile phone includes: radio frequency (RF) circuit 710, memory 720, input unit 730, display unit 740, sensor 750, audio circuit 760, wireless fidelity (Wi-Fi) module 770, processor 780, and power supply 790, etc. Those skilled in the art can understand that Figure 7 the structure of the mobile phone shown in the embodiments of the present application does not constitute a limitation to the mobile phone, and can include more or less components than the diagram, or combine some components, or different arrangement of components.

[0124] The various components of the mobile phone will be specifically introduced as follows: Figure 7

[0125] The RF circuit 710 can be used for receiving and sending signals in the process of information or call, in particular, receiving the downlink information of the base station and processing it by the processor 780; in addition, sending the uplink data to the base station. Usually, the RF circuit 710 includes but is not limited to antenna, at least one amplifier, transceiver, coupler, low noise amplifier (LNA), duplexer, etc. In addition, the RF circuit 710 can also communicate with the network and other devices through wireless communication. The above wireless communication can use any communication standard or protocol, including but not limited to global system for mobile communication (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), long term evolution (LTE), email, short message service (SMS), etc.

[0126] ​The memory 720 can be used to store software programs and modules, and the processor 780 can execute various function applications and data processing of the mobile phone by running the software programs and modules stored in the memory 720. The memory 720 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), and the like; and the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), and the like. In addition, the memory 720 can include a high-speed random access memory, and can also include a non-volatile memory, for example, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device.

[0127] The input unit 730 can be used to receive inputted digital or character information, and to generate key signal input related to the user settings and function control of the mobile phone. Specifically, the input unit 730 can include a touch panel 731 and other input devices 732. The touch panel 731, also known as a touch screen, can collect the touch operation of a user thereon or nearby (such as the operation of the user using a finger, a stylus, or any suitable object or accessory on or near the touch panel 731), and drive the corresponding connection device according to the pre-set program. Optionally, the touch panel 731 can include two parts of a touch detection device and a touch controller. The touch detection device detects the touch position of the user and detects the signal caused by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, and converts it into touch coordinates, and then sends it to the processor 780, and can also receive the command from the processor 780 and execute it. In addition, the touch panel 731 can be implemented in various types such as a resistive type, a capacitive type, an infrared type, and a surface acoustic wave type. In addition to the touch panel 731, the input unit 730 can also include other input devices 732. Specifically, the other input devices 732 can include one or more of a physical keyboard, a function key (such as a volume control key, an on-off key, etc.), a trackball, a mouse, a joystick, and the like.

[0128] The display unit 740 can be used to display information input by a user or information provided to the user as well as various menus of the phone. The display unit 740 can include a display panel 741, which can be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like. Further, a touch panel 731 can cover the display panel 741, and when the touch panel 731 detects a touch operation thereon or nearby, it transmits to the processor 780 to determine the type of touch event, and then the processor 780 provides corresponding visual output on the display panel 741 according to the type of touch event. Although in the above description, the touch panel 731 and the display panel 741 are implemented as two independent components to realize the input and output functions of the phone, in some embodiments, the touch panel 731 and the display panel 741 can be integrated to realize the input and output functions of the phone. Figure 7

[0129] The phone can also include at least one sensor 750, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor can include an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 741 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 741 and / or the backlight when the phone is moved to the ear. As one of the motion sensors, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), and when at rest, it can detect the magnitude and direction of gravity, which can be used for applications that identify the posture of the phone (such as landscape / portrait screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), and the like. As for other sensors that the phone can also be configured, such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, and the like, they will not be described here.

[0130] The audio circuit 760, the speaker 761, and the microphone 762 can provide an audio interface between the user and the phone. The audio circuit 760 can convert received audio data into an electrical signal, transmit it to the speaker 761, and convert it into a sound signal output by the speaker 761; on the other hand, the microphone 762 converts the sound signal collected into an electrical signal, which is received by the audio circuit 760 and converted into audio data, which is output to the processor 780 for processing, and then transmitted to another phone via the RF circuit 710, or output to the memory 720 for further processing.

[0131] Wi-Fi belongs to a short-range wireless transmission technology, and the phone can help users send and receive emails, browse web pages, and access streaming media through the Wi-Fi module 770, which provides users with wireless broadband Internet access. Although Figure 7 ​A Wi-Fi module 770 is shown, but it is understood that it is not an essential component of the mobile phone and can be omitted as needed without changing the nature of the application.

[0132] The processor 780 is the control center of the mobile phone, which connects all parts of the mobile phone through various interfaces and lines, performs various functions of the mobile phone and processes data by running or executing software programs and / or modules stored in the memory 720 and calling data stored in the memory 720, thereby overall monitoring the mobile phone. Optionally, the processor 780 can include one or more processing units; preferably, the processor 780 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application programs, etc., and the modem processor mainly processes wireless communication. It is understood that the above-mentioned modem processor can also not be integrated into the processor 780.

[0133] The mobile phone also includes a power supply 790 (such as a battery) for supplying power to various components, and preferably the power supply can be logically connected to the processor 780 through a power management system, so as to realize functions such as management of charging, discharging and power consumption management through the power management system.

[0134] Although not shown, the mobile phone can also include a camera, a Bluetooth module, etc., which will not be described here.

[0135] In the embodiments of the present application, the processor 780 is configured to acquire a write request; according to the write request, calculate a foreground hash value and metadata of a write page through foreground calculation; acquire idle information of a terminal device, and determine a trigger occasion; in the case of meeting the trigger occasion, calculate a file with the same foreground hash value, and output inode information of the file with the same foreground hash value; according to the foreground hash value, the metadata and the inode information, calculate a background hash value; and use a hard link method to remove redundancy for the same file in the background hash value.

[0136] Optionally, the processor 780 is specifically configured to calculate the foreground hash value and the metadata of the write page through foreground using an Alder32 method according to the write request.

[0137] Optionally, the processor 780 is specifically configured to calculate the foreground hash value and the metadata of the last page of the target file through foreground calculation according to the write request.

[0138] Optionally, the processor 780 is specifically configured to calculate the background hash value using an MD5 or SHA256 method according to the foreground hash value, the metadata and the inode information.

[0139] Optionally, the processor 780 is specifically configured to acquire the idle information of the terminal device, and determine the triggering occasion according to the CART classification tree algorithm.

[0140] Optionally, the processor 780 is further configured to detect a read / write request of a user on a first file, judge an attribute i_nlink of the first file inode according to the read / write request, determine that the first file is a non-hard link file if the attribute i_nlink = 1, and perform read / write on the first file, and determine that the first file is a hard link file if the attribute i_nlink > 2, and generate a temporary file of the first file and perform read / write on the temporary file.

[0141] Optionally, the processor 780 is specifically configured to calculate a foreground hash value, metadata and a user tag of a write page through foreground calculation according to the write request.

[0142] The processor 780 is specifically configured to calculate a background hash value according to the foreground hash value, the metadata and the inode information in a case that files with the same foreground hash value belong to the same user.

[0143] Optionally, the processor 780 is further configured to record the background hash value of the different files in the background hash value, and directly compare the files when calculating the foreground hash value next time.

[0144] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium that can be stored by the computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (for example, floppy disk, hard disk, magnetic tape), optical media (for example, DVD), or semiconductor media (for example, solid state disk (SSD)) and the like.

[0145] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0146] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the described device embodiments are merely schematic, and the division of the units is merely a logical function division, and there can be another division manner in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0147] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, that is, can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0148] In addition, each functional unit in the various embodiments of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0149] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that makes a contribution to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0150] The above-described embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method of file deduplication with respect to a terminal device, characterized by, The method is applied to a terminal device, and the method comprises: acquiring a write request; calculating foreground hash values and metadata of a write page according to the write request; acquiring idle information of the terminal device to determine a triggering occasion; when the triggering occasion is met, calculating files identical to the foreground hash values and outputting inode information of the files identical to the foreground hash values; when the files identical to the foreground hash values belong to the same user, calculating background hash values according to the foreground hash values, the metadata and the inode information; using a hard link method to remove redundancy of the files identical in the background hash values.

2. The method of claim 1, wherein, The calculation of the foreground hash values and the metadata of the write page according to the write request comprises: calculating the foreground hash values and the metadata of the write page according to the write request by using an Alder32 method in the foreground.

3. The method according to claim 1 or 2, characterized in that, The calculation of the foreground hash values and the metadata of the write page according to the write request comprises: calculating the foreground hash values and the metadata of a last page of a target file according to the write request.

4. The method according to claim 1 or 2, characterized in that, The calculation of the background hash values according to the foreground hash values, the metadata and the inode information comprises: calculating the background hash values according to the foreground hash values, the metadata and the inode information by using an MD5 or SHA256 method.

5. The method according to claim 1 or 2, characterized in that, The acquisition of the idle information of the terminal device to determine the triggering occasion comprises: acquiring the idle information of the terminal device and determining the triggering occasion according to a CART classification tree algorithm.

6. The method according to claim 1 or 2, characterized in that, The method further comprises: detecting a read / write request of a user to a first file; judging an attribute i_nlink of an inode of the first file according to the read / write request; if the attribute i_nlink = 1, determining that the first file is a non-hard link file and performing read / write on the first file; if the attribute i_nlink > 2, determining that the first file is a hard link file and generating a temporary file of the first file, and performing read / write on the temporary file.

7. The method according to claim 1 or 2, characterized in that, The method further comprises: for the files different in the background hash values, recording the background hash values of the different files, and directly comparing when the foreground hash values of the files are calculated next time.

8. A terminal device, comprising: comprise: a storage memory storing executable program codes; a processor coupled with the storage memory; ​ ​ 9. A terminal device, comprising: ​ ​ ​ The processor is configured to perform the method of any one of claims 1-7.

10. A computer-readable storage medium comprising instructions that, when executed on a processor, cause the processor to perform the method of any one of claims 1-7.

Images