Processing method and device
By deduplication of the data and signature parts of the signed files in the file set and generating configuration files, the problem of not being able to effectively remove duplicate parts of the file set in the prior art is solved, and more efficient file compression and recoverability are achieved.
Patent Information
- Application Number
- CN202110346716.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-03-31
- Publication Date
- 2025-03-21
- Estimated Expiration
- 2041-03-31
AI Technical Summary
When compressing a file set, the duplicate data part and signature part in the signed file cannot be effectively removed, resulting in a large amount of compressed file data and failing to achieve maximum file compression.
By deduplication of the data part and/or the signature part of the file set to be compressed, the configuration file record deduplication information is generated, and the configuration file and the deduplication file set are compressed to form a compressed file.
It effectively reduces the amount of data in the file set, achieves more efficient file compression, and ensures the recoverability of compressed files.
Smart Images

Figure CN113051238B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of information processing technology, and more specifically, to a processing method and device. Background Art
[0002] A file set includes multiple files and has a large amount of data, so it is often necessary to compress the file set.
[0003] There may be many files with the same data content in the file set. When there are many duplicate contents in the application files, even if the files are compressed, the data volume of the compressed files will be relatively large, and the maximum file compression is not achieved. Summary of the invention
[0004] The present application provides a processing method and device.
[0005] A processing method includes:
[0006] Obtaining a first file set to be compressed, the first file set comprising a plurality of first-type files with signatures;
[0007] Deduplication of data portion and / or signature portion of the first type of files with signatures is performed to obtain a second file set after deduplication;
[0008] Generate a configuration file, wherein the configuration file records at least configuration information of the data part and the signature part in the first file set, so as to enable the second file set to be restored to the first file set;
[0009] The configuration file and the second file set are compressed to obtain a compressed file of the first file set.
[0010] In another possible implementation, before deduplicating the data portion and / or the signature portion of the first type of files with signatures to obtain the deduplicated second file set, the method further includes:
[0011] The data part and the signature part of the first type of file are disassembled.
[0012] In a possible implementation, the method further includes:
[0013] If the data portion of the first type of file is a compressed data portion, decompress the compressed data portion to obtain a decompressed first type of file with a signature, and return to perform the operation of decompressing the data portion and the signature portion of the first type of file for the decompressed first type of file;
[0014] If the data portion of the first type of file does not belong to the compressed data portion, the decomposition of the first type of file is terminated.
[0015] In yet another possible implementation, the method further includes:
[0016] For a first type of file whose decompressed data portion is a compressed data portion, determining a file association relationship between the first type of file and the first type of file decompressed from the first type of file;
[0017] Determining a decompression parameter used to decompress the first type of file;
[0018] The configuration file also includes: the file association relationship and the decompression parameters corresponding to the decompressed first type file, so as to restore the first type file containing the compressed data portion corresponding to the decompressed first type file.
[0019] In another possible implementation, the deduplication of the data portion and / or the signature portion of the first type of files with signatures to obtain a deduplicated second file set includes:
[0020] Determine at least one first file group among the plurality of first-type files, and perform file deduplication on the first file group, wherein the first file group includes first-type files having the same data portion and signature portion;
[0021] and / or,
[0022] Determine at least one second file group among the plurality of first type files, and deduplicate data parts of the first type files in the second file group, wherein the second file group includes first type files with the same data parts but different signature parts;
[0023] and / or,
[0024] At least one third file group among the multiple first type files is determined, and signature parts of the first type files in the third file group are deduplicated, wherein the third file group includes first type files with different data parts but the same signature parts.
[0025] In a possible implementation, the first file set further includes: a plurality of second type files without signatures;
[0026] The method further comprises:
[0027] Deduplicating the plurality of files of the second type to remove duplicate files of the second type;
[0028] The configuration file also includes: configuration information of the second type of file.
[0029] In yet another possible implementation, the method further includes:
[0030] Decompressing the compressed file to obtain the configuration file and the second file set;
[0031] The second file set is restored to the first file set according to the configuration information of the data part and the signature part of the first type file in the configuration file.
[0032] In another possible implementation, the configuration file further records: the file association relationship and the decompression parameters corresponding to the decompressed first type of files;
[0033] The step of restoring the second file set to the first file set according to the configuration information of the data part and the signature part of the first type file in the configuration file includes:
[0034] If it is determined according to the configuration file that the data portion of the first type of file to be restored in the first file set is a compressed data portion, according to the configuration information of the data portion and the signature portion of the first type of file in the configuration file, the data portion and the signature portion of the first type of files at all levels associated with the first type of file are obtained from the second file set, and according to the file association relationship and the decompression parameters, the data portion and the signature portion of the first type of file and the first type of files at all levels associated with the first type of file are respectively compressed and assembled to obtain the restored first type of file;
[0035] If it is determined according to the configuration file that the data portion of the first type of file to be restored in the first file set does not belong to the compressed data portion, according to the configuration information of the data portion and the signature portion of the first type of file in the configuration file, the data portion and the signature portion constituting the first type of file are obtained from the second file set and assembled to obtain the restored file of the first type;
[0036] The first file set is formed by using the restored multiple files of the first type.
[0037] Among them, another processing method includes:
[0038] Get the compressed file;
[0039] Decompressing the compressed file to obtain a configuration file and an initial file set to be restored, wherein the configuration file records configuration information of a data portion and a signature portion in a first type of file with a signature in a target file set, and the initial file set is obtained after deduplicating the data portion and / or the signature portion of at least the first type of file in the target file set;
[0040] The initial file set is restored to the target file set according to the configuration information of the data part and the signature part of the first type of file in the configuration file.
[0041] In another possible implementation, the configuration file further records: a file association relationship between a first type file whose data portion is a compressed data portion and a first type file decompressed from the first type file;
[0042] The step of restoring the initial file set to the target file set according to the configuration information of the data part and the signature part of the first type of file in the configuration file includes:
[0043] For a first type of file whose data portion is a compressed data portion, according to the configuration information of the data portion and the signature portion of the first type of file in the configuration file, the data portion and the signature portion of the first type of files at various levels associated with the first type of file are obtained from the initial file set, and according to the file association relationship, the data portion and the signature portion of the first type of file and the first type of files at various levels associated with the first type of file are respectively compressed and assembled to obtain the first type of file;
[0044] For a first type of file whose data part does not belong to the compressed data part, according to the configuration information of the data part and the signature part of the first type of file in the configuration file, the data part and the signature part constituting the first type of file are obtained from the initial file set and assembled to obtain the first type of file;
[0045] A target file set is formed using a plurality of first-type files.
[0046] A processing device includes:
[0047] A file obtaining unit, used for obtaining a first file set to be compressed, wherein the first file set includes a plurality of first-type files with signatures;
[0048] A data deduplication unit, configured to perform data deduplication and / or signature deduplication on the first type of files with signatures to obtain a second file set after deduplication;
[0049] a configuration generating unit, configured to generate a configuration file, wherein the configuration file records at least configuration information of the data part and the signature part in the first file set, so as to enable the second file set to be restored to the first file set;
[0050] The file compression unit is used to compress the configuration file and the second file set to obtain a compressed file of the first file set.
[0051] Among them, another processing device includes:
[0052] A file obtaining unit, used for obtaining a compressed file;
[0053] A file decompression unit, used to decompress the compressed file to obtain a configuration file and an initial file set to be restored, wherein the configuration file records configuration information of a data part and a signature part in a first type of file with a signature in a target file set, and the initial file set is obtained after deduplication of a data part and / or a signature part of at least the first type of file in the target file set is performed;
[0054] The file restoration unit is used to restore the initial file set to the target file set according to the configuration information of the data part and the signature part of the first type of file in the configuration file.
[0055] In another aspect, the present application also provides an electronic device, comprising:
[0056] Processor and memory;
[0057] Wherein, the processor is used to execute any one of the processing methods described above;
[0058] The memory is used to store programs required for the processor to perform operations.
[0059] On the other hand, the present application also provides a computer-readable storage medium, which stores at least one instruction, at least one program, a code set or an instruction set. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the processing method described in any one of the above embodiments.
[0060] It can be seen from the above scheme that after obtaining the first file set to be compressed, the present application will deduplicate the data part and / or the signature part of the first type of files with signatures in the first file set, so as to remove the repeated data part and / or the signature part in the first file set, so that the data volume of the second file set obtained after deduplication is reduced relative to the first file set. Moreover, after deduplication of the first file set, a configuration file for restoring the second file set to the first file set will be generated. On this basis, after compressing the second file set and the configuration file, it will not affect the recovery of the first file set, and the data volume of the compressed file can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS
[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0062] Figure 1 A schematic diagram of a process flow of a processing method provided in an embodiment of the present application;
[0063] Figure 2 A schematic diagram of another process flow of the processing method provided in the embodiment of the present application;
[0064] Figure 3 A schematic diagram of disassembling a signed file provided in an embodiment of the present application;
[0065] Figure 4 A schematic diagram of another process flow of the processing method provided in the embodiment of the present application;
[0066] Figure 5 Another schematic diagram of a file with a signature in a disassembled file set provided by an embodiment of the present application;
[0067] Figure 6 For this application Figure 5 A schematic diagram of deduplication of data parts of each file obtained after disassembly;
[0068] Figure 7 for Figure 5 The remaining data part and signature part of the file set after the data part is deduplicated;
[0069] Figure 8 A schematic flow chart of another processing method provided in an embodiment of the present application;
[0070] Fig. 9 A schematic diagram of the structure of a processing device provided in an embodiment of the present application;
[0071] Fig.10 A schematic diagram of the composition structure of another processing device provided in an embodiment of the present application;
[0072] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.
[0073] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims and the above drawings are used to distinguish similar parts, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated herein. DETAILED DESCRIPTION
[0074] The solution of the present application is applicable to the compression processing and recovery of a file set including at least program files with signatures, so as to achieve more effective compression of the file set and reduce the data volume of the compressed file set while ensuring that the compressed file can be effectively restored to the file state before compression.
[0075] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0076] See also Figure 1 , Figure 1 A flow chart of a processing method provided in an embodiment of the present application. The method of this embodiment is applied to an electronic device, which may be a mobile phone, a laptop computer, a desktop computer, a workstation or a server, etc., without limitation.
[0077] The processing method of this embodiment may include:
[0078] S101, obtaining a first file set to be compressed.
[0079] The first file set includes a plurality of first-type files with signatures.
[0080] It is understood that a signed file refers to a file with a digital signature, and a signed file generally consists of a signature part and a data part. The digital signature on a file can represent the validity of the data part of the file, such as proving that the file or the data in the file has not been modified. For example, the file to be signed can be an executable file with a signature.
[0081] There are many possibilities for the first file set including multiple signed first-type files, which are not limited in this application. For example, the first file set may be a program file of an application, which may include multiple signed files, and the multiple files to be signed may be plug-in files or library files, etc., which are not limited.
[0082] In this application, in order to distinguish from subsequent files without signatures, files with signatures are referred to as first-type files, and subsequent files without signatures are referred to as second-type files.
[0083] S102, deduplication of data part and / or signature part of the first type of files with signatures is performed to obtain a second file set after deduplication.
[0084] In the present application, S102 can be to use the data part or the signature part in the first type of file as the deduplication object that needs to be detected for duplication, and deduplicate the data part or the signature part in the first type of file to deduplicate the repeated data part or the repeated signature part in the first file set.
[0085] In the present application, step S102 can also be performed to: deduplicate data parts of different first type files in the first file set, and deduplicate signature parts of different first type files in the first file set, thereby deduplicating repeated data parts and repeated signature parts in the first file set.
[0086] For example, taking data portion deduplication of first type files as an example, it is necessary to determine the data portion included in each of multiple first type files, and remove the repeated data portions in multiple first type files, so that there are no repeated data portions belonging to different first type files in the second file set after data portion deduplication.
[0087] Similarly, the signature parts of the first type files are deduplicated to remove duplicate signature parts in multiple first type files, so that the second file set obtained after deduplication does not contain duplicate signature parts belonging to different first type files.
[0088] It can be understood that by using one or both of the data part and the signature part in the first file set as deduplication objects, it is possible to reduce the situation where the files cannot be deduplicated due to being different but the data part or the signature part being the same, thereby more effectively reducing the presence of duplicate data in the first file set.
[0089] It is understandable that, in practical applications, the data part and the signature part can also be used as a whole to deduplicate the data part and the signature part of the first file set, so as to remove the first type of files with the same data part and the signature part. Of course, when deduplicating the data part and the signature part of the first file set respectively, it is actually possible to deduplicate the file set with the same signature part and data part.
[0090] S103, generating a configuration file.
[0091] The configuration file at least records configuration information of the data part and the signature part in the first file set, so that the second file set can be restored to the first file set.
[0092] The configuration file can be used to determine each first type file included in the first file set and the data part and signature part included in each first type file, so that the first type files included in the first file set can be restored using the deduplicated second file set.
[0093] For example, in one possible scenario, the configuration file records identification information of each first-type file included in the first file set, identification information of the data portion and identification information of the signature portion contained in each first-type file.
[0094] The identification information of the first type of file may be the file name of the first type of file or other information that can uniquely identify the first type of file. Similarly, the identification information of the data part may be the name or unique number of the data part, and the identification information of the signature part may be the name or unique number of the signature part.
[0095] In this case, based on the information recorded in the configuration file, the first type files included in the first file set can be determined. At the same time, for each first type file in the first file set, based on the identification information of the data part and the identification information of the signature part of the first type file, the data part and the signature part belonging to the first type file can be determined from the second file set, thereby reconstructing each first type file in the first file set and restoring the first file set.
[0096] It can be understood that each of the first type files in the first file set has its own file address range. In order to restore the first file set more accurately based on the second file set, the configuration file can also include: the address range of the data part and the address range of the signature part of each first type file. On this basis, the address range of each first type file in the first file set can be reflected through the configuration file. Therefore, based on the address range of the data part and the address range of the signature part of the first type file recorded in the configuration file, the address range of the restored first type file in the first file set is determined to restore the position of the first type file in the first file set.
[0097] In another possible implementation, the configuration file may also be identification information recording at least one first type file whose data part and / or signature part are deduplicated in the first file set and at least one first type file whose signature part and / or data part are deduplicated. On this basis, based on the configuration file, it can be determined which first type files have one or both of the data part and the signature part removed, so that the removed data part and / or signature part of these first type files can be restored.
[0098] S104, compress the configuration file and the second file set to obtain a compressed file of the first file set.
[0099] Compressing the configuration file together with the second file set can make the compressed file contain information capable of restoring the first file set, so that the first file set can be restored based on the second file set and the configuration file.
[0100] The inventor of the present application has found through research that: in the process of compressing a file set, each file in the file set can be used as a deduplication object for file deduplication, so that duplicate files can be removed. However, in the file deduplication method, only different files that are completely identical as a whole will be considered duplicate files. For files with signatures, it is easy to have a situation where the data parts are exactly the same but the files are different only due to the different signature parts, so that the duplicate data parts between different files cannot be deduplicated. Similarly, the duplicate signature parts between different files cannot be deduplicated. Based on this, the method of deduplicating the entire file cannot maximally dedupe the duplicate data in the file set.
[0101] In the solution of the present application, the entire file is not used as the deduplication object, but any one of the data part and the signature part in the first type of file set is used as the deduplication object, so that the repeated data part and signature part in the first file set can be deduplicated.
[0102] In the embodiment of the present application, there may be many possible ways to compress the second file set and the configuration file, and the present application does not impose any limitation on this.
[0103] From the above, it can be seen that after obtaining the first file set to be compressed, the present application will deduplicate the data part and / or the signature part of the first type of files with signatures in the first file set, so as to remove the repeated data part and / or the signature part in the first file set, so that the data volume of the second file set obtained after deduplication is reduced relative to the first file set. Moreover, after deduplication of the first file set, a configuration file for restoring the second file set to the first file set will be generated. On this basis, after compressing the second file set and the configuration file, it will not affect the recovery of the first file set, and the data volume of the compressed file can be reduced.
[0104] It can be understood that in the present application, when deduplicating the data part and / or signature part of the first type of files in the first file set, it is possible to first determine which first type of files have the same data part or signature part, or both the signature part and the data part are the same.
[0105] Specifically, the present application may include: deduplicating the data part and / or the signature part of the first type of files in the first file set:
[0106] Determine at least one first file group among the plurality of first-type files, and perform file deduplication on the first file group, wherein the first file group includes first-type files having the same data portion and signature portion;
[0107] and / or,
[0108] Determine at least one second file group among the plurality of first type files, and deduplicate the data portion of the first type files in the second file group, wherein the second file group includes the first type files having the same data portion but different signature portions;
[0109] and / or,
[0110] At least one third file group among the multiple first type files is determined, and signature parts of the first type files in the third file group are deduplicated, wherein the third file group includes first type files having different data parts but the same signature parts.
[0111] Different types of file groups and different file groups of the same type contain different first type files. For example, different first file groups contain different first type files, and the first file group is different from any of the second file group and the third file group.
[0112] In one possible implementation, in order to facilitate comparison of repeated data parts or signature parts in each first type file, the present application may also decompose the data part and signature part of each first type file before deduplicating the first file set.
[0113] Disassembling a file refers to analyzing and determining the various components of a file. For example, based on the respective component features of the data part and the signature part of the first type of file, the data part and the signature part can be disassembled from the first type of file.
[0114] It is understandable that some of the first type of files in the first file set may be compressed files. In this case, the first type of files need to be decompressed first, and then the decompressed files need to be disassembled. Among them, when the first type of files are compressed files, the data part of the first type of files is a compressed data part, so the compressed data part needs to be decompressed. At the same time, for the decompressed signed files, it is still necessary to continue to disassemble so that duplicate data can be removed to the greatest extent during deduplication.
[0115] The following is an explanation of a possible implementation. Figure 2 As shown, it shows another flow chart of a processing method provided by an embodiment of the present application. This embodiment may include:
[0116] S201, obtaining a first file set to be compressed.
[0117] The first file set includes a plurality of first-type files with signatures.
[0118] S202: For each first type file, separate the data portion and the signature portion of the first type file.
[0119] S203, detecting whether the data portion of the first type of file is a compressed data portion, if yes, executing step S204; if no, ending the disassembly of the first type of file and executing step S205.
[0120] The compressed data portion refers to the compressed data portion.
[0121] S204, decompressing the compressed data portion of the first type of file to obtain a first type of file with a signature decompressed from the compressed data portion, and returning to step S202 for the decompressed first type of file.
[0122] It is understandable that if the data portion of the first type file is a compressed data portion, then after decompressing the first compressed data portion, at least one first type data with a signature may still be decompressed. For the first type file with a signature decompressed, further decomposition is still required to subsequently analyze whether the data portion of the decompressed first type file is repeated with other first type files.
[0123] It can be seen that if the data portion of the first type file is a compressed data portion, it may be decompressed and disassembled through one or more iterations until the data portions of the first type files at all levels contained in the first type file are disassembled.
[0124] For ease of understanding, the following takes a signed file: signature file M as an example, assuming that the data part of the signature file is a compressed data part. Figure 3 Provide explanation.
[0125] exist Figure 3 FIG. 4 shows the process of disassembling and decompressing the signature file M layer by layer from left to right.
[0126] like Figure 3 As shown, the signature file M is decomposed into the signature part of the signature file M and the compressed data part of the signature file M.
[0127] The compressed data part of the signature file M can be decompressed to obtain two files with signatures, namely: signature file M1 and signature file M2.
[0128] On this basis, it is also necessary to disassemble the signature file M1 and the signature file M2 separately.
[0129] like Figure 3 As shown, the signature file M1 is decomposed into the data part of the signature file M1 and the signature part of the signature file M1. Assuming that the data part of the signature file M1 does not belong to the compressed data part, the decomposition of the signature file M1 is completed.
[0130] The signature file M2 is decomposed into the data part of the signature file M2 and the signature part of the signature file M2. Assuming that the data part of the signature file M2 is still a compressed data part, the data part of the signature file M2 needs to be decompressed. Figure 3 , the data part of the signature file M2 is decompressed to obtain the signature file M21.
[0131] The signature file M21 still needs to be disassembled, such as Figure 3 , the signature file M21 is decomposed into the data part of the signature file M21 and the unsigned part of the signature file M21. Assuming that the data part of the signature file M21 does not belong to the compressed data part, the decomposition of the signature file M21 can be ended. At the same time, since the signature files at all levels under the signature file M have been decomposed, all decompositions for the signature file M are completed.
[0132] S205: If all the first type files in the first file set have been decomposed, deduplication of data parts and / or deduplication of signature parts is performed on the multiple first type files determined from the first file set to obtain a second file set after deduplication.
[0133] It can be understood that after the previous steps, the disassembled first type files and the first type files that originally existed in the first file set can be obtained. Therefore, it is necessary to combine the data part and the signature part contained in each first type file and deduplicate one or both of the data part and the signature part respectively.
[0134] The specific deduplication process is similar to that of the previous embodiment and will not be described in detail.
[0135] For ease of understanding, assume that the first file set includes signature file A and the previous Figure 3 Taking the signature file M mentioned in the example as an example, it is assumed that the signature file A is decomposed into the data part of the signature file A (not belonging to the compressed data part) and the signature part of the signature file A.
[0136] After multiple layers of decompression and disassembly, the signature file M finally obtains: the signature part of signature file M1, the data part of signature file M1, the signature part of signature file M2, the data part of signature file M21 under signature file M2, and the signature part of signature file M21.
[0137] On this basis, when deduplicating the data part, it is necessary to compare the data part of signature file M1, the data part of signature file M21 and the data part of signature file A to see if there are any duplicate data parts. Assuming that the data parts of signature file A and signature file M21 are duplicated, the data part of signature file A or signature file M21 can be removed.
[0138] Similarly, to deduplicate the signature part, the signature part of signature file A, the signature part of signature file M, the signature part of signature file M1, the signature part of signature file M2, and the signature part of signature file M1 can be compared to see if there are duplicate signature parts. If there are duplicate signature parts, the duplicate signature parts can be deduplicated.
[0139] In a possible implementation, the signature part of any first type file may include a signature certificate public key and a signature result part.
[0140] The signature result part is obtained by signing the data part in the first type file using the signature certificate public key. Therefore, only when the data part and the signature certificate public key are exactly the same, the signature result part will be exactly the same. Two first type files with the same signature result part are actually the same first type files. For the same first type files, file deduplication can also be used to remove duplicate files. Of course, when the data part and the signature part are the same, the duplicate data part and the signature part are removed to remove the duplicate first type files.
[0141] In order to repeat the file deduplication operation when deduplicating the signature part, in the present application, when deduplicating the signature part of the first type of file alone, the signature certificate public key of the first type of file can be deduplicated.
[0142] S206, for a first type file whose disassembled data portion is a compressed data portion, determine a file association relationship between the first type file and a first type file decompressed from the first type file, and a decompression parameter used to decompress the decompressed first type file.
[0143] The file association relationship can ensure the relationship between the first type files of each layer decomposed from the first type file.
[0144] For example, still with the previous Figure 3 The example in Figure 3 From the disassembly process of the signature file M, it can be seen that the file association relationship between the signature files at least includes:
[0145] The compressed data part of the signature file M includes: signature file M1 and signature file M2;
[0146] The compressed data portion of the signature file M2 includes: a signature file M21.
[0147] The decompression parameters used for the decompressed first type file are the decompression parameters used for decompressing the compressed data portion containing the first type file.
[0148] If still Figure 3 As an example, it is assumed that the decompression parameter used for the compressed data part of the signature file M can be decompression parameter 1, that is, the corresponding decompression parameters of the signature file M1 and the signature file M2 are decompression parameter 1. The decompression parameter of the compressed data part of the decompressed signature file M2 is decompression parameter 2.
[0149] It can be understood that step S206 is an optional step. If it is only necessary to determine each data part or signature part in the file set, or if the relationship between files does not need to be concerned, step S206 may not be performed.
[0150] S207, generating a configuration file.
[0151] At least configuration information of the data part and the signature part in the first file set is recorded, so that the second file set can be restored to the first file set.
[0152] Optionally, on the premise that the above file association relationship and decompression parameters are determined in step S206, the configuration file also includes: the file association relationship determined above and the decompression parameters corresponding to the decompressed first type file, so as to be able to restore the first type file containing the compressed data part corresponding to the decompressed first type file.
[0153] S208, compress the configuration file and the second file set to obtain a compressed file of the first file set.
[0154] In one possible implementation, considering that each first type file in the first file set has been broken down into a signature part and a data part, after deduplication of the signature part and / or data part of each first type file in the first file set, the deduplicated signature part and data part can be spliced according to the remaining signature part and / or data part of the first type file remaining after deduplication in the first file set to obtain a second file set.
[0155] It can be understood that, before deduplicating the data part or the signature part of the first type of file with a signature in the first file set, this embodiment will disassemble the data part and the signature part of the first type of file, and when the disassembled data part is a compressed data part, the compressed data part will be decompressed and then disassembled, so as to more comprehensively determine the various data parts and signature parts contained in the first file set, thereby reducing the part of the data part or the signature part that cannot be deduplicated due to being in a compressed form, further improving the comprehensiveness of deduplication of the first file set, so that the second file set obtained by deduplication can be compressed into a compressed file with a smaller data volume.
[0156] It is understandable that, for files with signatures, some of the data parts in such files can be divided into multiple parts. For example, in one possible case, the data part of the first type of file may include: data subject, signature description information, and initial data verification information for verifying the data subject and signature description information. In this case, since the signature description information is related to the signature part, in the case where the signature parts of different first type files have the same data subject, the data part of the first type of file as a whole may be considered as different content due to the different signature parts.
[0157] In order to further remove duplicate data, the present application may also calculate data verification information of the data subject based on the data subject included in the data part in the first type of file, and determine the data subject and the calculated data verification information as the corrected data part in the first type of file.
[0158] On this basis, when deduplicating the data parts in the file set, the modified data parts in each first type of file may be deduplicated to remove repeated modified data parts in the file set.
[0159] To facilitate understanding of the solution of the present application, the following takes deduplication of data of a first type of file in a first file set as an example, and illustrates the example of a file set of an application.
[0160] like Figure 4, which shows another flow chart of a file processing method of the present application. The method of this embodiment may include:
[0161] S401, obtaining a first file set of an application program to be compressed.
[0162] The first file set includes a plurality of first-type files with signatures in the application.
[0163] like Figure 5 As shown, the first file set of the application may include file A, file B, and file D with signatures.
[0164] S402: For each first type file, decompose the first type file into a data portion, a signature public key portion, and a signature result portion.
[0165] In this embodiment, it is explained by taking the example that the signature part includes the signature public key part and the signature result part.
[0166] S403, detecting whether the data portion of the first type of file is a compressed data portion, if yes, executing step S404; if no, ending the disassembly of the first type of file and executing step S405.
[0167] The compressed data portion refers to the compressed data portion.
[0168] S404, decompressing the compressed data portion of the first type of file to obtain a first type of file with a signature decompressed from the compressed data portion, and returning to step S402 for the decompressed first type of file.
[0169] like Figure 5 As shown, the signature file A with a signature in each first file set can be disassembled into the data part Aa of the signature file A, the signature certificate public key part Aa of the signature file A, and the signature result part Aa. Among them, if the data part Aa of the signature file A does not belong to the compressed data part, the disassembly process of the signature file A ends.
[0170] Similar to the decomposition of signature file A, signature file B is decomposed into data part Bb, signature certificate public key part Bb and signature result part Bb.
[0171] The signature file D is decomposed into the data part Dd, the signature certificate public key part Dd and the signature result part Dd. Among them, the data part Dd is compressed data, so the data part Dd is decompressed to obtain the signature file DA and the signature file E. Among them, the signature file DA is decomposed into the data part Aa, the signature certificate part DA and the signature result part DA.
[0172] The signature file E is decomposed into the data part Ee, the signature certificate public key part Aa and the signature result part Ee.
[0173] Among them, the data part Ee still belongs to the compressed data part, so the data part Ee is further decompressed to obtain the signature file B. The signature file B is disassembled to obtain the data part Bb, the signature certificate public key part Bb and the signature result part Bb.
[0174] S405: If all the first type files in the first file set have been disassembled, deduplicate the data portion of the determined multiple first type files, and deduplicate the signature certificate public key portion of the multiple first type files.
[0175] S406: concatenate the remaining data part and the signature part in the first file set after deduplication into one file to obtain a second file set.
[0176] like Figure 5 As shown, if the data part Aa in the signature file DB obtained by disassembling the signature file D by one or more layers is repeated with the data part Aa in the signature file A directly obtained from the first file set above, then one data part Aa can be deleted, such as Figure 6 As shown, the data portion Aa in the signature file DB is removed.
[0177] Similarly, if the signature file B obtained by disassembling and decompressing the signature file E is the same as the data signature part and the signature certificate public key part in the above signature file B, then one signature file B can be removed. Figure 6 The data part and signature part of signature file B decomposed from signature file E are removed.
[0178] At the same time, the signature certificate public key part Aa disassembled from the signature file E is also removed from the signature certificate public key part Aa in the signature file A above, so a signature certificate public key Aa needs to be removed. Figure 6 As shown, the signature certificate public key part Aa in the signature file E is removed.
[0179] After deduplication, the Figure 6 After the remaining parts of each signature file are spliced together, the second file set contains the following contents: Figure 7 shown.
[0180] S407, for a first type file whose disassembled data portion is a compressed data portion, determine a file association relationship between the first type file and a first type file decompressed from the first type file, and a decompression parameter used to decompress the decompressed first type file.
[0181] It can be understood that the file association relationship and the decompression parameters can be obtained in the process of disassembling the first type files in the first file set and decompressing the compressed data part.
[0182] like Figure 5 As shown, the file association relationship may include: the data compression part decomposed from the signature file D includes the signature file DA and the signature file E; and the data compression part of the signature file E may include the signature file B.
[0183] The decompression parameters may be parameters used to decompress the data compression part of each signature file, wherein the decompression parameters of the data compression parts of different signature files may be the same or different, and there is no limitation on this.
[0184] S408: Generate a configuration file.
[0185] The configuration file includes the identification information of each first type file included in the first file set; the identification information and address range corresponding to the data part, the signature certificate public key part and the signature result part contained in each first type file; the file association relationship; and the compression parameters.
[0186] For example, the above Figure 5 and 6 For example, the configuration file needs to record the file names of signature file A, signature file B and signature file D, the data parts or the composition of the data parts, and the identifiers of the signature certificate public key part and the signature result part, among which the composition of the data part can be reflected through the file association relationship.
[0187] At the same time, for Figure 5 and 6 Each signature file can also record the address space of each component in the signature file. For example, taking signature file A as an example, signature file A includes data part Aa, signature certificate public key part Aa and signature result part Aa. Based on this, the configuration file needs to record the address space information of each component of signature file A as follows:
[0188] {
[0189] "Signature File A": {
[0190] Signature file A data part "0-10000",
[0191] Signature file A signature certificate public key part "10000-10500",
[0192] Signature file A signature result part "10500-10700",
[0193] }
[0194] ·······
[0195] }.
[0196] Of course, the configuration information corresponding to other signature files A can also be recorded in the above manner, which will not be repeated here.
[0197] S409, compress the configuration file and the second file set to obtain a compressed file of the application.
[0198] It can be understood that in the above embodiments of the present application, the first file set may also include a plurality of second type files without signatures.
[0199] If the first file set includes multiple second-type files without signatures, the present application can also deduplicate the multiple second-type files to remove duplicate second-type files in the first file set, so that the obtained second file set includes at least one second-type file that has been deduplicated.
[0200] On this basis, the configuration file may also record configuration information of the second type of files. Based on the configuration information of the second type of files and the second file set, each second type of file included in the first file set may be restored.
[0201] It can be understood that the above processing method is described from the perspective of deduplication and compression of the first file set in the embodiment.
[0202] Based on the above embodiment, after obtaining the compressed file of the first file set, if it is necessary to restore the first file set based on the compressed file, the configuration file and the second file set can be first decompressed from the compressed file, and then the second file set is restored in combination with the configuration file to obtain the first file set. The process of restoring the first file set from the second file set is the reverse process of deduplicating the first file set to obtain the second file set.
[0203] For example, in a possible implementation, the compressed file can be decompressed to obtain a configuration file and a second file set. Accordingly, the second file set can be restored to the first file set according to the configuration information of the data part and the signature part of the first type file in the configuration file.
[0204] In a possible implementation manner, the configuration file further records: a file association relationship and decompression parameters between a first type file whose data portion is a compressed data portion and a first type file decompressed from the first type file.
[0205] If it is determined according to the configuration file that the data portion of the first type of file to be restored in the first file set is a compressed data portion, according to the configuration information of the data portion and the signature portion of the first type of file in the configuration file, the data portion and the signature portion constituting the first type of file are obtained from the second file set and assembled to obtain the restored first type of file. For details, please refer to the related introduction above, which will not be repeated here.
[0206] If it is determined according to the configuration file that the data portion of the first type of file to be restored in the first file set does not belong to the compressed data portion, the data portion and the signature portion of the first type of files at all levels associated with the first type of file are obtained from the second file set according to the configuration information of the data portion and the signature portion of the first type of file in the configuration file, and according to the file association relationship and decompression parameters, the data portion and the signature portion of the first type of file and the first type of files at all levels associated with the first type of file are compressed and assembled respectively to obtain the restored first type of file.
[0207] Based on the configuration file, a first file set is formed by using the restored plurality of first-type files.
[0208] The following describes the process of restoring the first file set based on the compressed file obtained in the previous embodiment in conjunction with the flow chart. Figure 8 As shown, it shows a flow chart of another processing method of the present application. The method of this embodiment may include:
[0209] S801, obtain a compressed file.
[0210] The compressed file may be a compressed file of the first file set generated by the method mentioned in any of the above embodiments.
[0211] S802, decompress the compressed file to obtain a configuration file and an initial file set to be restored.
[0212] The configuration file records configuration information of the data part and the signature part in the first type of files with signatures in the target file set.
[0213] The initial file set is obtained by deduplicating the data part and / or the signature part of at least the first type of files in the target file set. The initial file set is equivalent to the second file set mentioned in the previous embodiment. For details, please refer to the relevant introduction of the previous embodiment, which will not be repeated here.
[0214] S803: restore the initial file set to a target file set according to the configuration information of the data part and the signature part of the first type of file in the configuration file.
[0215] It can be understood that since the configuration file records the configuration information of the data part and the signature part in the first type of files with signatures in the target file set, the data part and the signature part contained in each first type of file in the target file set can be determined according to the configuration file, so as to reconstruct each first type of file in the target file set and restore the target file set.
[0216] The target file set is equivalent to the first file set mentioned in the previous embodiment.
[0217] For example, the configuration information includes at least identification information of the data part and the signature part in the first type of file. Since the initial file set includes the first type of files after deduplication, the initial file set includes the data part and the signature part of each first type of file remaining after the target file set has been deduplicated after the data part and / or the signature part are deduplicated. The data part and the signature part removed from the target file set can be found from the data part and the signature part remaining in the initial file set.
[0218] It can be seen that, in the case of the first type of files included in the target file set indicated in the configuration file and the identification information of the data part and the signature part in each first type of file, for each first type of file, the data part and the signature part constituting the first type of file can be obtained from the initial file set, thereby restoring the first type of file. Accordingly, the target file set can be formed based on the restored first type of files.
[0219] It is understandable that if the configuration file records the address ranges of each first-type file and the data part and the signature part in the first-type file, then when restoring each first-type file, the data part and the signature part can be assembled according to the respective address areas of the data part and the signature part in the first-type file to restore the first-type file. Accordingly, after restoring each first-type file, multiple first-type files can be combined according to the address ranges of each first-type file to restore the target file set.
[0220] Of course, the above is just an example of a restoration method. In actual applications, there may be other restoration methods, which are not limited.
[0221] In a possible implementation manner, the configuration file further records: a file association relationship between a first type file whose data portion is a compressed data portion and a first type file decompressed from the first type file.
[0222] If it is determined according to the configuration file that the data portion of the first type of file to be restored in the target file set belongs to the compressed data portion, according to the configuration information of the data portion and the signature portion of the first type of file in the configuration file, the data portion and the signature portion constituting the first type of file are obtained from the initial file set and assembled to obtain the first type of file. For details, please refer to the above related introduction, which will not be repeated here.
[0223] If it is determined according to the configuration file that the data portion of the first type of file to be restored in the target file set does not belong to the compressed data portion, the data portion and the signature portion of the first type of files at all levels associated with the first type of file are obtained from the initial file set according to the configuration information of the data portion and the signature portion of the first type of file in the configuration file, and according to the file association relationship, the data portion and the signature portion of the first type of files at all levels associated with the first type of file are compressed and assembled respectively to obtain the restored first type of file.
[0224] Based on the configuration file, a target file set is formed by using the restored plurality of first-type files.
[0225] Corresponding to a processing method of the present application, the present application also provides a processing device.
[0226] like Fig. 9 As shown, it shows a schematic diagram of a composition structure of a processing device of the present application. The device of this embodiment may include:
[0227] A file obtaining unit 901 is used to obtain a first file set to be compressed, the first file set including a plurality of first type files with signatures;
[0228] A data deduplication unit 902 is used to perform data deduplication and / or signature deduplication on the first type of files with signatures to obtain a second file set after deduplication;
[0229] A configuration generating unit 903 is used to generate a configuration file, wherein the configuration file records at least configuration information of the data part and the signature part in the first file set, so as to enable the second file set to be restored to the first file set;
[0230] The file compression unit 904 is used to compress the configuration file and the second file set to obtain a compressed file of the first file set.
[0231] In a possible implementation, the method further includes:
[0232] The file disassembling unit is used for the data deduplication unit to disassemble the data part and the signature part of the first type of file before deduplicating the data part and / or the signature part of the first type of file with signature.
[0233] In yet another possible implementation, the method further includes:
[0234] A file decompression unit is used to, after the file disassembly unit disassembles the data part, if the data part of the first type of file is a compressed data part, decompress the compressed data part to obtain a decompressed first type of file with a signature, and return to execute the operation of the disassembly unit for the decompressed first type of file; if the data part of the first type of file does not belong to the compressed data part, end the disassembly of the first type of file.
[0235] In yet another possible implementation, the method further includes:
[0236] a relationship determination unit, configured to determine, for a first type of file whose disassembled data portion is a compressed data portion, a file association relationship between the first type of file and the first type of file decompressed from the first type of file;
[0237] A parameter determination unit, used to determine a decompression parameter used to decompress the first type of file;
[0238] The configuration file also includes: the file association relationship and the decompression parameters corresponding to the decompressed first type file, so as to restore the first type file containing the compressed data portion corresponding to the decompressed first type file.
[0239] In yet another possible manner, the data deduplication unit includes:
[0240] A first data deduplication unit is used to determine at least one first file group among the plurality of first type files, and perform file deduplication on the first file group, wherein the first file group includes first type files having the same data part and signature part;
[0241] and / or,
[0242] a second data deduplication unit, configured to determine at least one second file group among the plurality of first type files, and perform data deduplication on the first type files in the second file group, wherein the second file group includes first type files with the same data part but different signature parts;
[0243] and / or,
[0244] A third data deduplication unit is used to determine at least one third file group among the multiple first type files, and deduplicate the signature part of the first type files in the third file group, wherein the third file group includes first type files with different data parts but the same signature part.
[0245] In yet another possible implementation, the first file set obtained by the file obtaining unit further includes: a plurality of second type files without signatures;
[0246] The device also includes:
[0247] A file deduplication unit, configured to perform file deduplication on the plurality of second-type files to remove duplicate second-type files;
[0248] The configuration file generated by the configuration generation unit also includes: configuration information of the second type of file.
[0249] In yet another possible implementation, the device further includes:
[0250] A file decompression unit, used for decompressing the compressed file to obtain a configuration file and a second file set to be restored;
[0251] The file restoration unit is used to restore the second file set to the first file set according to the configuration information of the data part and the signature part of the first type file in the configuration file.
[0252] In an optional manner, when the configuration file further records: a file association relationship and a decompression parameter between a first type file whose data portion is a compressed data portion and a first type file decompressed from the first type file, the file restoration unit includes:
[0253] A first file restoration unit is configured to, if it is determined according to the configuration file that the data portion of the first type of file to be restored in the first file set is a compressed data portion, obtain the data portion and the signature portion of the first type of files at all levels associated with the first type of file from the second file set according to the configuration information of the data portion and the signature portion of the first type of file in the configuration file, and compress and assemble the data portion and the signature portion of the first type of file and the first type of files at all levels associated with the first type of file according to the file association relationship and the decompression parameters, respectively, to obtain the restored first type of file;
[0254] A second file restoration unit is configured to, if it is determined according to the configuration file that the data portion of the first type of file to be restored in the first file set does not belong to the compressed data portion, obtain the data portion and the signature portion constituting the first type of file from the second file set according to the configuration information of the data portion and the signature portion of the first type of file in the configuration file, and assemble them to obtain the restored first type of file;
[0255] The file combination unit is used to use the restored multiple first-type files to form a first file set.
[0256] On the other hand, corresponding to the processing method in decompressing a compressed file in the present application, the present application also provides another processing device. Fig.10 As shown, it shows a schematic diagram of a composition structure of another processing device of the application, and the device includes:
[0257] A file obtaining unit 1001 is used to obtain a compressed file;
[0258] A file decompression unit 1002 is used to decompress the compressed file to obtain a configuration file and an initial file set to be restored, wherein the configuration file records configuration information of a data part and a signature part in a first type of file with a signature in a target file set, and the initial file set is obtained by deduplicating the data part and / or the signature part of at least the first type of file in the target file set;
[0259] The file restoration unit 1003 is used to restore the initial file set to the target file set according to the configuration information of the data part and the signature part of the first type of file in the configuration file.
[0260] In a possible implementation, when the configuration file further records: a file association relationship and a decompression parameter between a first type file whose data portion is a compressed data portion and a first type file decompressed from the first type file, the file restoration unit includes:
[0261] A first file restoration unit is configured to, if it is determined according to the configuration file that the data portion of the first type of file to be restored in the target file set is a compressed data portion, obtain the data portion and the signature portion of the first type of files at all levels associated with the first type of file from the initial file set according to the configuration information of the data portion and the signature portion of the first type of file in the configuration file, and compress and assemble the data portion and the signature portion of the first type of file and the first type of files at all levels associated with the first type of file according to the file association relationship and the decompression parameters, respectively, to obtain the restored first type of file;
[0262] A second file restoration unit is configured to, if it is determined according to the configuration file that the data portion of the first type of file to be restored in the target file set does not belong to the compressed data portion, obtain the data portion and the signature portion constituting the first type of file from the initial file set according to the configuration information of the data portion and the signature portion of the first type of file in the configuration file, and assemble them to obtain the restored first type of file;
[0263] The file combination unit is used to use the restored multiple first-type files to form a target file set.
[0264] In another aspect, the present application also provides an electronic device, such as Fig.11 As shown, it shows a schematic diagram of the composition structure of an electronic device of the present application.
[0265] The electronic device at least includes a processor 1101 and a memory 1102 .
[0266] The processor 1101 is used to execute the processing method in any one of the above embodiments.
[0267] The memory 1102 is used to store programs required by the processor to perform operations.
[0268] It is understandable that the electronic device may also include an input device 1103, an output device 1104, and a communication bus 1105. Of course, the electronic device may also have Fig.11 There may be more or fewer components, without limitation.
[0269] On the other hand, the present application also provides a computer-readable storage medium, which stores at least one instruction, at least one program, a code set or an instruction set. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the processing method described in any one of the above embodiments.
[0270] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can be referred to each other. At the same time, the features recorded in each embodiment in this specification can be replaced or combined with each other, so that professionals in the field can implement or use this application. For device embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0271] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A processing method comprising: Obtaining a first file set to be compressed, the first file set comprising a plurality of first-type files with signatures; Deduplication of the data part and the signature part of the first type of files with signatures is performed, or deduplication of the data part of the first type of files with signatures is performed to obtain a second file set after deduplication; Generate a configuration file, wherein the configuration file records at least configuration information of the data part and the signature part in the first file set, so as to enable the second file set to be restored to the first file set; Compressing the configuration file and the second file set to obtain a compressed file of the first file set; Among them, deduplicating data parts of the signed first type files includes: determining the data parts included in each of the first type files, removing repeated data parts in multiple first type files, so that there are no repeated data parts belonging to different first type files in the second file set.
2. The method according to claim 1, before performing data and signature deduplication on the first type of files with signatures, or performing data deduplication on the first type of files with signatures to obtain a second set of deduplicated files, further comprises: The data part and the signature part of the first type of file are disassembled.
3. The method according to claim 2, further comprising: If the data portion of the first type of file is a compressed data portion, decompress the compressed data portion to obtain a decompressed first type of file with a signature, and return to perform the operation of decompressing the data portion and the signature portion of the first type of file for the decompressed first type of file; If the data portion of the first type of file does not belong to the compressed data portion, the decomposition of the first type of file is terminated.
4. The method according to claim 3, further comprising: For a first type of file whose decompressed data portion is a compressed data portion, determining a file association relationship between the first type of file and the first type of file decompressed from the first type of file; Determining a decompression parameter used to decompress the first type of file; The configuration file also includes: the file association relationship and the decompression parameters corresponding to the decompressed first type file, so as to restore the first type file containing the compressed data portion corresponding to the decompressed first type file.
5. According to the method of claim 1, the step of performing data and signature deduplication on the first type of files with signatures, or performing data deduplication on the first type of files with signatures to obtain a second set of files after deduplication, comprises: Determine at least one first file group among the plurality of first-type files, and perform file deduplication on the first file group, wherein the first file group includes first-type files having the same data portion and signature portion; and / or, Determine at least one second file group among the plurality of first type files, and deduplicate data portions of the first type files in the second file group, wherein the second file group includes first type files with the same data portions but different signature portions; and / or, At least one third file group among the multiple first type files is determined, and signature parts of the first type files in the third file group are deduplicated, wherein the third file group includes first type files with different data parts but the same signature parts.
6. The method according to claim 1, wherein the first file set further comprises: Multiple second type files without signatures; The method further comprises: Deduplicating the plurality of files of the second type to remove duplicate files of the second type; The configuration file also includes: configuration information of the second type of file.
7. The method according to claim 4, further comprising: Decompressing the compressed file to obtain the configuration file and the second file set; The second file set is restored to the first file set according to the configuration information of the data part and the signature part of the first type file in the configuration file.
8. The method according to claim 7, wherein the configuration file further records: the file association relationship and the decompression parameters corresponding to the decompressed first type of files; The step of restoring the second file set to the first file set according to the configuration information of the data part and the signature part of the first type file in the configuration file includes: If it is determined according to the configuration file that the data portion of the first type of file to be restored in the first file set is a compressed data portion, according to the configuration information of the data portion and the signature portion of the first type of file in the configuration file, the data portion and the signature portion of the first type of files at all levels associated with the first type of file are obtained from the second file set, and according to the file association relationship and the decompression parameters, the data portion and the signature portion of the first type of file and the first type of files at all levels associated with the first type of file are respectively compressed and assembled to obtain the restored first type of file; If it is determined according to the configuration file that the data portion of the first type of file to be restored in the first file set does not belong to the compressed data portion, according to the configuration information of the data portion and the signature portion of the first type of file in the configuration file, the data portion and the signature portion constituting the first type of file are obtained from the second file set and assembled to obtain the restored file of the first type; The first file set is formed by using the restored multiple files of the first type.
9. A processing device comprising: A file obtaining unit, used for obtaining a first file set to be compressed, wherein the first file set includes a plurality of first-type files with signatures; A data deduplication unit, configured to perform data deduplication and signature deduplication on the first type of files with signatures, or perform data deduplication on the first type of files with signatures to obtain a second set of files after deduplication; a configuration generating unit, configured to generate a configuration file, wherein the configuration file records at least configuration information of the data part and the signature part in the first file set, so as to enable the second file set to be restored to the first file set; A file compression unit, used for compressing the configuration file and the second file set to obtain a compressed file of the first file set; Among them, deduplicating data parts of the signed first type files includes: determining the data parts included in each of the first type files, removing repeated data parts in multiple first type files, so that there are no repeated data parts belonging to different first type files in the second file set.
10. A processing method comprising: Get the compressed file; Decompressing the compressed file to obtain a configuration file and an initial file set to be restored, wherein the configuration file records configuration information of a data portion and a signature portion in a first type of file with a signature in a target file set, and the initial file set is obtained after deduplicating the data portion and the signature portion of at least the first type of file in the target file set, or the initial file set is obtained after deduplicating the data portion of at least the first type of file in the target file set; Restoring the initial file set to the target file set according to the configuration information of the data part and the signature part of the first type of file in the configuration file; Among them, deduplicating data parts of the signed first type files includes: determining the data parts included in each of the first type files, removing repeated data parts in multiple first type files, so that there are no repeated data parts belonging to different first type files in the initial file set.
Citation Information
Patent Citations
Data compression method, data decompression method and related systems
CN107506153A