File processing method and device, computer device and readable storage medium

Abstract

The application provides a file aggregation processing method and device, computer equipment and a readable storage medium. The file aggregation processing method comprises the following steps: obtaining the identification information of a file aggregation object and the identification information of a target file after determining a file aggregation operation of aggregating the file aggregation object to the target file, wherein the file aggregation object comprises a file or a snapshot; generating an operation ID of the file aggregation operation; constructing a first mapping relationship between the identification information of the file aggregation object and the operation ID, and a second mapping relationship between the operation ID and the identification information of the target file, wherein a mapping relationship group comprises the first mapping relationship and the second mapping relationship; and tracing back a file aggregation error according to the mapping relationship group. Through the application, the file aggregation error can be traced back, the technical experience requirement for the file aggregation error tracing back is reduced, and the convenience of the file aggregation error tracing back is improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a file aggregation processing method, apparatus, computer equipment, and readable storage medium. Background Technology

[0002] A snapshot refers to an image of a face or body captured by a camera at a specific time. An archive is a collection of snapshots, with each archive considered to contain all the snapshots associated with a single person. Archive aggregation is the process of determining whether a snapshot can be included in an archive or become a new archive.

[0003] To improve the effectiveness of data aggregation, the aggregation results are usually checked to identify incorrectly archived snapshots, such as aggregation of snapshots from different people into the same person's file, or aggregation of snapshots from the same person into different people's files. However, after finding such aggregation errors, due to the complexity of the aggregation algorithm itself and its many intricate steps, it is difficult to trace the error back to its source. Experienced algorithm engineers are often left to guess the possible location of the problem.

[0004] Therefore, how to better trace back archive errors has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] The purpose of this invention is to provide a file aggregation processing method, apparatus, computer device, and readable storage medium to solve the technical problems in the prior art.

[0006] On the one hand, in order to achieve the above objectives, the present invention provides a file aggregation processing method.

[0007] The file aggregation processing method includes: after determining an aggregation operation to aggregate an object to a target file, obtaining the identification information of the aggregated object and the identification information of the target file, wherein the aggregated object includes a file or a snapshot; generating an operation ID for the aggregation operation; constructing a first mapping relationship between the identification information of the aggregated object and the operation ID, and a second mapping relationship between the operation ID and the identification information of the target file, wherein the mapping relationship group includes the first mapping relationship and the second mapping relationship; and backtracking aggregation errors based on the mapping relationship group.

[0008] Further, when the aggregation error includes aggregating a first snapshot that does not belong to the first file into the first file, the step of tracing back the aggregation error according to the mapping relationship group includes: Step S11: Obtain the current query identifier information, wherein the initial query identifier information is the identifier information of the first snapshot; Step S12: Search for a first mapping relationship including the query identifier information in the mapping relationship group to obtain a first result mapping, wherein the operation ID included in the first result mapping is a query operation ID; Step S13: Search for a second mapping relationship including the query operation ID in the mapping relationship group. The mapping relationship is used to obtain the second result mapping, wherein the second result mapping includes the identifier information of the target file as the identifier information of the query file; Step S14: Record the identifier information of the query file into the aggregation path; Step S15: Determine whether the query file is the first file, wherein if the query file is not the first file, execute step S16, and if the query file is the first file, execute step S17; Step S16: Assign the identifier information of the query file to the query identifier information, and return to step S11; Step S17: Output the aggregation path.

[0009] Furthermore, when the clustering error includes clustering multiple snapshots that do not belong to the same person into the first file, the step of tracing back the clustering error according to the mapping relationship group includes: for each of the multiple snapshots, taking each snapshot as the first snapshot, using the above steps S11 to S17 to determine the clustering path of each snapshot; merging the clustering paths of the multiple snapshots to form a clustering tree.

[0010] Furthermore, the mapping relationship group also includes a third mapping relationship and a fourth mapping relationship. The method further includes: finding a second mapping relationship where the identification information is the identification information of the second file; constructing a third mapping relationship between the identification information of the second file and all operation IDs in the found second mapping relationship; finding a first mapping relationship where the operation ID is the first operation ID; constructing a fourth mapping relationship between the first operation ID and all identification information in the found first mapping relationship; the step of backtracking the file aggregation error according to the mapping relationship group includes: step S21: finding the third mapping relationship including the identification information to be analyzed in the mapping relationship group to obtain a third result mapping, wherein the initial identification information to be analyzed is the identification information of the target file. The third result mapping includes the operation ID to be analyzed; Step S22: Search for the fourth mapping relationship including the operation ID to be analyzed in the mapping relationship group to obtain the fourth result mapping, wherein the identification information included in the fourth result mapping is the component identification information; Step S23: Output the component identification information to the cluster tree to complete one recursion; Step S24: Determine whether the current recursion count meets the recursion depth threshold. If the current recursion count does not meet the recursion depth threshold, execute step S25. If the current recursion count meets the recursion depth threshold, execute step S26; Step S25: Assign the component identification information to the identification information to be analyzed and return to step S21; Step S26: Output the cluster tree.

[0011] Furthermore, the mapping relationship group also includes a fifth mapping relationship, and the method further includes: after determining the aggregation operation of the aggregation object, obtaining the operation meta-information of the aggregation operation, wherein the operation meta-information includes the parameters used when determining the aggregation operation; constructing a fifth mapping relationship between the operation ID and the operation meta-information; the step of backtracking aggregation errors according to the mapping relationship group further includes: determining the identification information on the target node, wherein the target node is a node in the aggregation path or aggregation tree; searching in the mapping relationship group for a third mapping relationship including the identification information on the target node to obtain a fifth result mapping; searching in the mapping relationship group for a fifth mapping relationship including the operation ID in the fifth result mapping to obtain a sixth result mapping; and outputting the operation meta-information included in the sixth result mapping.

[0012] Furthermore, the clustering processing method also includes: adjusting the threshold in the clustering operation based on the operation metadata.

[0013] Furthermore, the file aggregation operation includes a deletion operation. When the file aggregation operation is a deletion operation, the identification information of the target file is a predetermined identifier representing an empty file.

[0014] On the other hand, in order to achieve the above objectives, the present invention provides a file aggregation processing device.

[0015] The file aggregation processing device includes: an acquisition module, configured to acquire the identification information of the file aggregation object and the identification information of the target file after determining an aggregation operation to aggregate the file aggregation object to a target file, wherein the file aggregation object includes a file or a snapshot; a generation module, configured to generate an operation ID for the file aggregation operation; a construction module, configured to construct a first mapping relationship between the identification information of the file aggregation object and the operation ID, and a second mapping relationship between the operation ID and the identification information of the target file, wherein the mapping relationship group includes the first mapping relationship and the second mapping relationship; and a backtracking module, configured to backtrack file aggregation errors based on the mapping relationship group.

[0016] To achieve the above objectives, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above method.

[0017] To achieve the above objectives, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above method.

[0018] This invention provides a file aggregation processing method, apparatus, computer device, and readable storage medium. For an object to be aggregated, after determining the aggregation operation to aggregate it to a destination file, the method acquires the identification information of the object to be aggregated and the identification information of the destination file, generates an operation ID for the aggregation operation, and then constructs a first mapping relationship between the identification information of the object to the operation ID, and a second mapping relationship between the operation ID and the identification information of the destination file. Furthermore, aggregation errors can be traced back based on the mapping relationship group formed by these mapping relationships. Through this invention, by utilizing the construction of the mapping relationship group, aggregation errors can be traced back, reducing the technical experience requirements for aggregation error tracing and improving the convenience of aggregation error tracing. Attached Figure Description

[0019] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0020] Figure 1 This is a flowchart of the file aggregation processing method provided in Embodiment 1 of the present invention;

[0021] Figure 2This is a flowchart of a step in the file aggregation processing method provided in Embodiment 1 of the present invention, which involves tracing back the file aggregation error based on a mapping relationship group.

[0022] Figure 3 This is a flowchart of another step in the file aggregation processing method provided in Embodiment 1 of the present invention, which backtracks the file aggregation error based on the mapping relationship group;

[0023] Figure 4 This is a block diagram of the file aggregation processing device provided in Embodiment 2 of the present invention;

[0024] Figure 5 This is a hardware structure diagram of a computer device provided in Embodiment 3 of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0026] Example 1

[0027] This invention provides a file aggregation processing method that enables the backtracking of file aggregation errors. Specifically, Figure 1 The flowchart of the file aggregation processing method provided in Embodiment 1 of the present invention is as follows: Figure 1 As shown, the file aggregation processing method provided in this embodiment includes the following steps S101 to S104.

[0028] Step S101: After determining the aggregation operation to aggregate the aggregation object to the destination file, obtain the identification information of the aggregation object and the identification information of the destination file.

[0029] The aggregation objects include files or snapshots. For a file or snapshot, after determining the aggregation operation of the target file through the aggregation algorithm, the identification information of the aggregation object is obtained. For example, when the aggregation object is a file, the file number or file name is obtained; when the aggregation object is a snapshot, the snapshot code is obtained. This aggregation operation can be performed on a single aggregation object, i.e., aggregating one aggregation object to the target file; or it can be performed on multiple aggregation objects, i.e., aggregating multiple aggregation objects to the target file.

[0030] Optionally, the archiving operation includes merging and deleting operations. The merging operation includes the process of capturing images and incorporating them into the archive, as well as the process of real-name registration of the archive. The deletion operation is the discarding of captured images; some captured images are discarded during the archiving process and are not incorporated into the archive. When the archiving operation is a deletion operation, the identification information of the destination archive is a predetermined identifier representing an empty archive.

[0031] Step S102: Generate the operation ID for the file aggregation operation.

[0032] For this file aggregation operation, a unique corresponding operation ID is generated. Specifically, the operation ID can be generated using information such as operation type (merge or discard), operation time, and / or random number.

[0033] Step S103: Construct a first mapping relationship between the identification information of the archive object and the operation ID, and a second mapping relationship between the operation ID and the identification information of the destination archive.

[0034] The mapping relationship group includes the first mapping relationship and the second mapping relationship.

[0035] In step S103, a first mapping relationship is constructed, which associates the identification information of the archive object with the operation ID; a second mapping relationship is constructed, which associates the operation ID with the identification information of the destination archive. Both mapping relationships are stored in the mapping relationship group.

[0036] Step S104: Backtrack the clustering error based on the mapping relationship group.

[0037] When it is necessary to trace back to the cluster error, the cluster path or cluster tree related to the cluster error can be found by tracing back according to the mapping relationship group, so as to obtain the steps in which the cluster error occurred, which facilitates error judgment.

[0038] The file aggregation processing method provided in this embodiment, for a file to be aggregated, after determining the aggregation operation to aggregate it to a destination file, obtains the identification information of the file to be aggregated and the identification information of the destination file, and generates an operation ID for the aggregation operation. Then, it constructs a first mapping relationship between the identification information of the file to be aggregated and the operation ID, and a second mapping relationship between the operation ID and the identification information of the destination file. Furthermore, it can trace back aggregation errors based on the mapping relationship group formed by these mapping relationships. By constructing the mapping relationship group, the file aggregation processing method provided in this embodiment can realize the tracing of aggregation errors, reducing the technical experience requirements for tracing back aggregation errors and improving the convenience of tracing back aggregation errors.

[0039] Optionally, Figure 2 This is a flowchart of a step in the file aggregation processing method according to a mapping relationship group to trace back the file aggregation error in Embodiment 1 of the present invention, as shown below. Figure 2As shown, in one embodiment, when the aggregation error includes a first snapshot that does not belong to the first file being aggregated into the first file, the steps for tracing back the aggregation error according to the mapping relationship group include: Step S11: Obtain the current query identifier information, wherein the initial query identifier information is the identifier information of the first snapshot; Step S12: Search for a first mapping relationship including the query identifier information in the mapping relationship group to obtain a first result mapping, wherein the operation ID included in the first result mapping is the query operation ID; Step S13: Search for a second mapping relationship including the query operation ID in the mapping relationship group to obtain a second result mapping, wherein the identifier information of the destination file included in the second result mapping is the identifier information of the query file; Step S14: Record the identifier information of the query file in the aggregation path; Step S15: Determine whether the query file is the first file, wherein if the query file is not the first file, execute step S16, and if the query file is the first file, execute step S17; Step S16: Assign the identifier information of the query file to the query identifier information and return to step S11; Step S17: Output the aggregation path.

[0040] Specifically, when tracing how a snapshot was incorrectly archived—that is, tracing at which step in the archiving process a snapshot was ultimately archived to someone else's file—first, the snapshot's identifier is used as the initial query identifier. The first mapping relationship, including this query identifier, is searched in the mapping relationship group. The operation ID included in this first mapping relationship is the first archiving operation performed on the snapshot; in this embodiment, it is defined as the query operation ID. Then, using this query operation ID as the query condition, a second mapping relationship, including this query operation ID, is searched in the mapping relationship group. The identifier of the target file included in this second mapping relationship is the identifier of the file to which the snapshot was first archived; in this embodiment, it is defined as the query file identifier. Typically, a snapshot needs to undergo multiple archiving operations, passing through different files sequentially to complete the final archiving. Therefore, the file to which the snapshot was first archived may not be the final incorrectly archived file, but rather an intermediate node in the archiving process. Based on this, the identifier information of the queried file is first recorded in the aggregation path. Then, it is determined whether the queried file is the erroneous file that the snapshot was ultimately aggregated, i.e., the first file defined in this embodiment. If it is, the aggregation path is output. If not, the identifier information of the queried file is used as the query identifier information, and the search is performed again in the mapping relationship group. The queried file found in the second round is checked to see if it is the first file. This process is repeated until the queried file found is the first file, and the backtracking is completed to obtain the final aggregation path. Then, each node on the aggregation path can be analyzed to find the location where the aggregation error occurred.

[0041] For example, the mapping relationship group is stored in Redis. The first mapping relationship is recorded by "Identifier Information:to" to "Operation ID", and the second mapping relationship is recorded by "Operation ID:to" to "Identifier Information". If an erroneous snapshot A is merged into another person's file B, first let X = A, then query X:to = Y, then let X = Y, and continue to loop until X:to is found to be B, thus finding the merged file path from A to B, and completing the backtracking.

[0042] The archiving method provided in this embodiment can provide an archiving path when a snapshot is archived to an error file, so that researchers can directly obtain each node of the archiving process and easily find the error.

[0043] Optionally, in one embodiment, when the clustering error includes clustering multiple snapshots that do not belong to the same person into the first file, the step of tracing back the clustering error according to the mapping relationship group includes: for each of the multiple snapshots, take it as the first snapshot, and use the above steps S11 to S17 to determine the clustering path of each snapshot; merge the clustering paths of multiple snapshots to form a clustering tree.

[0044] Specifically, when dealing with multiple snapshots that do not belong to the same person but are mistakenly archived in the same file, each snapshot is first treated as the first snapshot, and then... Figure 2 The steps shown involve querying and looping to determine the aggregation path for each snapshot. Then, the aggregation paths of multiple snapshots are merged to form an aggregation tree structure. This allows us to pinpoint the location of the error within this aggregation tree structure, indicating at which step and which two or more snapshots were incorrectly merged.

[0045] The file aggregation processing method provided in this embodiment can generate multiple file aggregation trees when multiple snapshots from different people are archived into the same error file, so that researchers can directly obtain all nodes and relationships in the multiple snapshot aggregation process and easily find the error.

[0046] Optionally, in one embodiment, the mapping relationship group further includes a third mapping relationship and a fourth mapping relationship, and the file aggregation processing method further includes: finding a second mapping relationship where the identification information is the identification information of the second file; constructing a third mapping relationship between the identification information of the second file and all operation IDs in the found second mapping relationship; finding a first mapping relationship where the operation ID is the first operation ID; and constructing a fourth mapping relationship between the first operation ID and all identification information in the found first mapping relationship.

[0047] Furthermore, Figure 3This is a flowchart of another step in the file aggregation processing method provided in Embodiment 1 of the present invention, which involves backtracking the file aggregation error based on the mapping relationship group. Figure 3 As shown, the steps for backtracking the clustering error based on the mapping relationship group include: Step S21: Search for the third mapping relationship in the mapping relationship group that includes the identification information to be analyzed, and obtain the third result mapping, wherein the initial identification information to be analyzed is the identification information of the target file, and the operation ID included in the third result mapping is the operation ID to be analyzed; Step S22: Search for the fourth mapping relationship in the mapping relationship group that includes the operation ID to be analyzed, and obtain the fourth result mapping, wherein the identification information included in the fourth result mapping is the component identification information; Step S23: Output the component identification information to the clustering tree, and complete one recursion; Step S24: Determine whether the current recursion count meets the recursion depth threshold. If the current recursion count does not meet the recursion depth threshold, execute step S25. If the current recursion count meets the recursion depth threshold, execute step S26; Step S25: Assign the component identification information to the identification information to be analyzed, and return to step S21; Step S26: Output the clustering tree.

[0048] Specifically, when constructing the mapping relationship group, in addition to constructing the first and second mapping relationships, a third and fourth mapping relationship are also constructed. Among them, the third mapping relationship is the mapping relationship between a file and multiple aggregation operations, that is, the association relationship between a file and all aggregation operations that use this file as the destination file. For example, a file A includes 5 snapshots, and each snapshot is merged into file A through aggregation operations opid1, opid2, opid3, opid4 and opid5 respectively. Then the association relationship between file A and aggregation operations opid1-5 is the third mapping relationship.

[0049] The fourth mapping relationship is the mapping relationship between a file aggregation operation and multiple file aggregation objects. That is, the association relationship between a file aggregation operation and all the file aggregation objects operated on by the file aggregation operation. For example, if a file aggregation operation opid merges a file B, a snapshot C, and a snapshot D into a certain file, then the association relationship between the file aggregation operation opid and the file B, snapshot C, and snapshot D is the fourth mapping relationship.

[0050] Based on this, when the composition of a certain file needs to be analyzed, this file is taken as the target file, and the identification information of the target file is used as the initial identification information to be analyzed. A third mapping relationship including the identification information to be analyzed is searched in the mapping relationship group. The operation IDs included in the found third mapping relationship are all the clustering operations with the target file as the destination file. In this embodiment, these are defined as the operation IDs to be analyzed. Then, for each operation ID to be analyzed, a fourth mapping relationship including that operation ID is searched in the mapping relationship group. The identification information included in the fourth mapping relationship found based on an operation ID is the identification information of the clustering object operated by the clustering operation corresponding to that operation ID. This identification information is defined as the composition identification information, that is, the first-level clustering object of the target file is found. A clustering tree is generated using the composition identification information, completing one recursion. Here, a recursion depth threshold can be set, and the composition identification information is assigned to the identification information to be analyzed. Multiple recursive searches are performed to obtain a clustering tree of a certain depth. The composition of the target file can then be found within the structure of this clustering tree to analyze any errors present in the target file.

[0051] For example, mapping relationships are stored in Redis. The third mapping relationship is recorded using a list of "identifier information: from" to "operation ID," and the fourth mapping relationship is recorded using a list of "operation ID: from" to "identifier information." Examining the composition of a target file D, by querying D:from, we can obtain the snapshots / files directly merged into D. By repeatedly and recursively searching this result, we can eventually break them down into individual snapshots, resulting in a tree-like structure, i.e., a clustered file tree. In actual use, to limit resource consumption, a recursion depth threshold is used to limit the recursion depth.

[0052] The file aggregation processing method provided in this embodiment can trace back the composition of a file and output the file aggregation tree of the file, so that researchers can directly obtain all the nodes and relationships involved in the file aggregation process and easily find errors.

[0053] Optionally, in one embodiment, the mapping relationship group further includes a fifth mapping relationship, and the file aggregation processing method further includes: after determining the file aggregation operation of the file aggregation object, obtaining the operation metadata of the file aggregation operation, wherein the operation metadata includes the parameters used when determining the file aggregation operation; and constructing a fifth mapping relationship between the operation ID and the operation metadata.

[0054] Furthermore, the step of backtracking the clustering error based on the mapping relationship group also includes: determining the identification information on the target node, wherein the target node is a node in the clustering path or clustering tree; searching in the mapping relationship group for a third mapping relationship that includes the identification information on the target node to obtain a fifth result mapping; searching in the mapping relationship group for a fifth mapping relationship that includes the operation ID in the fifth result mapping to obtain a sixth result mapping; and outputting the operation metadata included in the sixth result mapping.

[0055] Specifically, when constructing the mapping relationship group, in addition to constructing the first to fourth mapping relationships mentioned above, a fifth mapping relationship is also constructed. The fifth mapping relationship is the association between a file aggregation operation and its operation metadata. The operation metadata includes the parameters used to determine the file aggregation operation, such as the file aggregation time, and the data on which the algorithm makes the decision to perform the file aggregation operation, such as the associated human body captures and their similarity when merging human body files.

[0056] Based on this, when tracing back to the archive aggregation error according to the mapping relationship group, after outputting the archive aggregation path or archive aggregation tree, for a certain node on the archive aggregation path or archive aggregation tree, defined as the target node in this embodiment, the metadata of the operation ID on that node can be further obtained. Specifically, using the identification information on the target node as the search condition, firstly, the third mapping relationship is searched in the mapping relationship group. The operation IDs included in the found third mapping relationship are all archive aggregation operations that use the file identified by the identification information on the target node as the target file. Then, for each found archive aggregation operation, the fifth mapping relationship is searched in the mapping relationship group using its ID as the search condition. The operation metadata included in the found fifth mapping relationship is also the operation metadata of the archive aggregation operations included in the formation of the target node. The operation metadata includes parameters such as the similarity threshold involved in the capture archive aggregation process. In this way, it is possible to understand whether the algorithm threshold on the key merging node is reasonable, whether the algorithm strategy is reasonable, etc. Further, optionally, the threshold in the archive aggregation operation can be adjusted according to the operation metadata, which helps to further optimize the algorithm steps.

[0057] Based on the file aggregation processing method provided in this application, the path involved in the file aggregation error can be given through the construction and backtracking of the mapping relationship group. For example, when processing the error of a capture that does not belong to the first file being aggregated into the first file, each operation can be traced back to find the file aggregation path, ensuring the accuracy of the backtracking. At the same time, by merging the file aggregation paths of multiple captures to form a file aggregation tree, the method can adapt to the backtracking of multiple capture file aggregation errors. Through the fifth mapping relationship in the mapping relationship group, the method can not only trace the file aggregation path, but also output the meta-information of each operation, which provides additional operation context and helps to understand the specific details of each file aggregation operation. Storing the mapping relationship group on Redis ensures the efficiency and scalability of the method implementation process. By using different mapping relationships, the system can quickly retrieve and update mapping information, making the processing of large-scale data more efficient. When processing the discarding, i.e. deletion operation involved in the file aggregation process, the destination file is defined as a predetermined identifier representing an empty file, so that the deletion operation and the merging operation form the same set of processing logic, are uniformly recorded, and obtain the same format of mapping relationship, making the backtracking of file aggregation errors more accurate and comprehensive.

[0058] Example 2

[0059] Corresponding to Embodiment 1 above, Embodiment 2 of the present invention provides a file aggregation processing device. The technical features and corresponding technical effects can be referred to Embodiment 1 above, and will not be repeated in this embodiment. Figure 4 As shown in Figure 4, the file aggregation processing device provided in Embodiment 2 of the present invention includes: an acquisition module 201, a generation module 202, a construction module 203, and a backtracking module 204.

[0060] The acquisition module 201 is used to acquire the identification information of the archive object and the identification information of the target file after determining the archive operation to archive the archive object to the target file, wherein the archive object includes a file or a snapshot; the generation module 202 is used to generate the operation ID of the archive operation; the construction module 203 is used to construct a first mapping relationship between the identification information of the archive object and the operation ID, and a second mapping relationship between the operation ID and the identification information of the target file, wherein the mapping relationship group includes the first mapping relationship and the second mapping relationship; and the backtracking module 204 is used to backtrack archive errors according to the mapping relationship group.

[0061] Optionally, in one embodiment, when the aggregation error includes aggregating a first snapshot that does not belong to the first file into the first file, the backtracking module performs the following steps: Step S11: Obtain the current query identifier information, wherein the initial query identifier information is the identifier information of the first snapshot; Step S12: Search for a first mapping relationship including the query identifier information in the mapping relationship group to obtain a first result mapping, wherein the operation ID included in the first result mapping is a query operation ID; Step S13: Search for a second mapping relationship including the query operation ID in the mapping relationship group. The mapping relationship is used to obtain the second result mapping, wherein the second result mapping includes the identifier information of the target file as the identifier information of the query file; Step S14: Record the identifier information of the query file into the aggregation path; Step S15: Determine whether the query file is the first file, wherein if the query file is not the first file, execute step S16, and if the query file is the first file, execute step S17; Step S16: Assign the identifier information of the query file to the query identifier information, and return to step S11; Step S17: Output the aggregation path.

[0062] Optionally, in one embodiment, when the clustering error includes clustering multiple snapshots that do not belong to the same person into the first file, the backtracking module performs the following steps: for each of the multiple snapshots, respectively regarded as the first snapshot, the clustering path of each snapshot is determined by using the above steps S11 to S17; the clustering paths of the multiple snapshots are merged to form a clustering tree.

[0063] Optionally, in one embodiment, the mapping relationship group further includes a third mapping relationship and a fourth mapping relationship, and the device further includes: a search module, configured to search for a second mapping relationship where the identification information is the identification information of the second file; a construction module, configured to construct a third mapping relationship between the identification information of the second file and all operation IDs in the found second mapping relationship; the search module is further configured to search for a first mapping relationship where the operation ID is the first operation ID; the construction module is further configured to construct a fourth mapping relationship between the first operation ID and all identification information in the found first mapping relationship; the backtracking module is further configured to perform the following step: Step S21: Search for the third mapping relationship including the identification information to be analyzed in the mapping relationship group to obtain a third result mapping, wherein the initial identification information to be analyzed... The information is the identification information of the target file, and the operation ID included in the third result mapping is the operation ID to be analyzed; Step S22: Search for the fourth mapping relationship including the operation ID to be analyzed in the mapping relationship group to obtain the fourth result mapping, wherein the identification information included in the fourth result mapping is the component identification information; Step S23: Output the component identification information to the file aggregation tree to complete one recursion; Step S24: Determine whether the current recursion count meets the recursion depth threshold. If the current recursion count does not meet the recursion depth threshold, execute step S25. If the current recursion count meets the recursion depth threshold, execute step S26; Step S25: Assign the component identification information to the identification information to be analyzed and return to step S21; Step S26: Output the file aggregation tree.

[0064] Optionally, in one embodiment, the mapping relationship group further includes a fifth mapping relationship. The acquisition module is further configured to acquire the operation metadata of the aggregation operation after determining the aggregation operation of the aggregation object, wherein the operation metadata includes the parameters used when determining the aggregation operation. The construction module is further configured to construct a fifth mapping relationship between the operation ID and the operation metadata. The backtracking module is further configured to perform the following steps: determine the identification information on the target node, wherein the target node is a node in the aggregation path or aggregation tree; search for a third mapping relationship including the identification information on the target node in the mapping relationship group to obtain a fifth result mapping; search for a fifth mapping relationship including the operation ID in the fifth result mapping in the mapping relationship group to obtain a sixth result mapping; and output the operation metadata included in the sixth result mapping.

[0065] Optionally, in one embodiment, the aggregation processing device further includes an adjustment module for adjusting the threshold in the aggregation operation based on the operation element information.

[0066] Optionally, in one embodiment, the file aggregation operation includes a deletion operation, and when the file aggregation operation is a deletion operation, the identification information of the destination file is a predetermined identifier representing an empty file.

[0067] Example 3

[0068] This embodiment also provides a computer device, such as a smartphone, tablet computer, laptop computer, desktop computer, rack server, blade server, tower server, or cabinet server (including a standalone server or a server cluster composed of multiple servers), etc., capable of executing programs. Figure 5 As shown, the computer device 01 in this embodiment includes, but is not limited to, a memory 012 and a processor 011 that can be interconnected via a system bus, such as... Figure 5 As shown. It should be noted that, Figure 5 Only a computer device 01 with component memory 012 and processor 011 is shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0069] In this embodiment, the memory 012 (i.e., the readable storage medium) includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 012 may be an internal storage unit of the computer device 01, such as the hard disk or memory of the computer device 01. In other embodiments, the memory 012 may also be an external storage device of the computer device 01, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 01. Of course, the memory 012 may include both the internal storage unit and its external storage device of the computer device 01. In this embodiment, the memory 012 is typically used to store the operating system and various application software installed on the computer device 01, such as the program code of the file processing device in Embodiment 2. In addition, memory 012 can also be used to temporarily store various types of data that have been output or will be output.

[0070] In some embodiments, processor 011 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. Processor 011 is typically used to control the overall operation of computer device 01. In this embodiment, processor 011 is used to run program code stored in memory 012 or process data, such as file aggregation methods.

[0071] Example 4

[0072] This embodiment also provides a computer-readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, App application store, etc., which stores a computer program. When the program is executed by a processor, it implements the corresponding function. The computer-readable storage medium of this embodiment is used to store a file aggregation processing device, and when executed by a processor, it implements the file aggregation processing method of Embodiment 1.

[0073] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0074] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0075] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0076] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A file aggregation processing method, characterized in that, include: After determining the aggregation operation to aggregate the archived object to the target file, the identification information of the archived object and the identification information of the target file are obtained, wherein the archived object includes a file or a snapshot. Generate the operation ID for the file aggregation operation; Construct a first mapping relationship between the identifier information of the archive object and the operation ID, and a second mapping relationship between the operation ID and the identifier information of the destination archive, wherein the mapping relationship group includes the first mapping relationship and the second mapping relationship; and Backtracking the clustering error based on the mapping relationship group Wherein, when the file aggregation error includes a first snapshot that does not belong to the first file being aggregated into the first file, the step of tracing back the file aggregation error according to the mapping relationship group includes: Step S11: Obtain the current query identifier information, wherein the initial query identifier information is the identifier information of the first captured image; Step S12: Search for the first mapping relationship including the query identifier information in the mapping relationship group to obtain the first result mapping, wherein the operation ID included in the first result mapping is the query operation ID; Step S13: Search for a second mapping relationship that includes the query operation ID in the mapping relationship group to obtain a second result mapping, wherein the identification information of the target file included in the second result mapping is the identification information of the query file; Step S14: Record the identification information of the queried file into the file aggregation path; Step S15: Determine whether the queried file is the first file. If the queried file is not the first file, proceed to step S16. If the queried file is the first file, proceed to step S17. Step S16: Assign the identifier information of the queried file to the query identifier information, and return to step S11; Step S17: Output the aggregated file path.

2. The file aggregation processing method according to claim 1, characterized in that, When the file aggregation error includes aggregating multiple snapshots that do not belong to the same person into the first file, the steps for backtracking the file aggregation error based on the mapping relationship group include: For each of the multiple snapshots, which is taken as the first snapshot, the aggregation path of each snapshot is determined by using the above steps S11 to S17. The multiple capture paths are merged to form a file aggregation tree.

3. The file aggregation processing method according to claim 2, characterized in that, The mapping relationship group also includes a third mapping relationship and a fifth mapping relationship. The method further includes: finding a second mapping relationship for the identification information of the second file; constructing a third mapping relationship between the identification information of the second file and all operation IDs in the found second mapping relationship; after determining the aggregation operation of the aggregation object, obtaining the operation meta-information of the aggregation operation, wherein the operation meta-information includes the parameters used to determine the aggregation operation; and constructing a fifth mapping relationship between the operation ID and the operation meta-information. The steps for backtracking archiving errors based on the mapping relationship group further include: determining the identification information on the target node, wherein the target node is a node in the archiving path or archiving tree; searching in the mapping relationship group for a third mapping relationship that includes the identification information on the target node to obtain a fifth result mapping; searching in the mapping relationship group for a fifth mapping relationship that includes the operation ID in the fifth result mapping to obtain a sixth result mapping; and outputting the operation metadata included in the sixth result mapping.

4. The file aggregation processing method according to claim 3, characterized in that, Also includes: Adjust the threshold in the clustering operation based on the aforementioned operation metadata.

5. The file aggregation processing method according to claim 1, characterized in that, The file aggregation operation includes a deletion operation. When the file aggregation operation is a deletion operation, the identification information of the target file is a predetermined identifier representing an empty file.

6. A method for file aggregation processing, characterized in that, include: After determining the aggregation operation to aggregate the archived object to the target file, the identification information of the archived object and the identification information of the target file are obtained, wherein the archived object includes a file or a snapshot. Generate the operation ID for the file aggregation operation; Construct a first mapping relationship between the identifier information of the archive object and the operation ID, and a second mapping relationship between the operation ID and the identifier information of the destination archive, wherein the mapping relationship group includes the first mapping relationship and the second mapping relationship; and Backtracking the clustering error based on the mapping relationship group The mapping relationship group also includes a third mapping relationship and a fourth mapping relationship. The method further includes: finding a second mapping relationship where the identification information is the identification information of the second file; constructing a third mapping relationship between the identification information of the second file and all operation IDs in the found second mapping relationship; finding a first mapping relationship where the operation ID is the first operation ID; and constructing a fourth mapping relationship between the first operation ID and all identification information in the found first mapping relationship. The steps for backtracking archival errors based on the mapping relationship group include: Step S21: Search for the third mapping relationship including the identification information to be analyzed in the mapping relationship group to obtain the third result mapping, wherein the initial identification information to be analyzed is the identification information of the target file, and the operation ID included in the third result mapping is the operation ID to be analyzed; Step S22: Search for the fourth mapping relationship that includes the ID of the operation to be analyzed in the mapping relationship group to obtain the fourth result mapping, wherein the identification information included in the fourth result mapping is the component identification information; Step S23: Output the component identification information to the cluster tree to complete one recursion; Step S24: Determine whether the current recursion count meets the recursion depth threshold. If the current recursion count does not meet the recursion depth threshold, proceed to step S25. If the current recursion count meets the recursion depth threshold, proceed to step S26. Step S25: Assign the component identification information to the identification information to be analyzed, and return to step S21; Step S26: Output the aggregated tree.

7. The file aggregation processing method according to claim 6, characterized in that, The mapping relationship group also includes a fifth mapping relationship, and the method further includes: after determining the aggregation operation of the aggregation object, obtaining the operation meta-information of the aggregation operation, wherein the operation meta-information includes the parameters used to determine the aggregation operation; and constructing a fifth mapping relationship between the operation ID and the operation meta-information. The steps for backtracking archiving errors based on the mapping relationship group further include: determining the identification information on the target node, wherein the target node is a node in the archiving tree; searching in the mapping relationship group for a third mapping relationship that includes the identification information on the target node to obtain a fifth result mapping; searching in the mapping relationship group for a fifth mapping relationship that includes the operation ID in the fifth result mapping to obtain a sixth result mapping; and outputting the operation metadata included in the sixth result mapping.

8. The file aggregation processing method according to claim 7, characterized in that, Also includes: Adjust the threshold in the clustering operation based on the aforementioned operation metadata.

9. The file aggregation processing method according to claim 6, characterized in that, The file aggregation operation includes a deletion operation. When the file aggregation operation is a deletion operation, the identification information of the target file is a predetermined identifier representing an empty file.

10. A file aggregation processing device, characterized in that, include: The acquisition module is used to acquire the identification information of the archive object and the identification information of the target file after determining the archive operation of archive object to archive target file, wherein the archive object includes file or snapshot; The generation module is used to generate the operation ID for the file aggregation operation; A construction module is used to construct a first mapping relationship between the identification information of the aggregated file object and the operation ID, and a second mapping relationship between the operation ID and the identification information of the destination file, wherein the mapping relationship group includes the first mapping relationship and the second mapping relationship; and The backtracking module is used to backtrack clustering errors based on the mapping relationship group. When the aggregation error includes a first snapshot that does not belong to the first file being aggregated into the first file, the backtracking module performs the following steps: Step S11: Obtain the current query identifier information, wherein the initial query identifier information is the identifier information of the first snapshot; Step S12: Search for a first mapping relationship including the query identifier information in the mapping relationship group to obtain a first result mapping, wherein the operation ID included in the first result mapping is the query operation ID; Step S13: Search for a second mapping relationship including the query operation ID in the mapping relationship group to obtain a second result mapping, wherein the identifier information of the destination file included in the second result mapping is the identifier information of the query file; Step S14: Record the identifier information of the query file in the aggregation path; Step S15: Determine whether the query file is the first file, wherein if the query file is not the first file, execute step S16, and if the query file is the first file, execute step S17; Step S16: Assign the identifier information of the query file to the query identifier information and return to step S11; Step S17: Output the aggregation path.

11. A file aggregation processing device, characterized in that, include: The acquisition module is used to acquire the identification information of the archive object and the identification information of the target file after determining the archive operation of archive object to archive target file, wherein the archive object includes file or snapshot; The generation module is used to generate the operation ID for the file aggregation operation; A construction module is used to construct a first mapping relationship between the identification information of the aggregated file object and the operation ID, and a second mapping relationship between the operation ID and the identification information of the destination file, wherein the mapping relationship group includes the first mapping relationship and the second mapping relationship; and The backtracking module is used to backtrack clustering errors based on the mapping relationship group. The mapping relationship group further includes a third mapping relationship and a fourth mapping relationship. The device further includes: a search module, used to search for a second mapping relationship where the identification information is the identification information of the second file; a construction module, used to construct a third mapping relationship between the identification information of the second file and all operation IDs in the found second mapping relationship; the search module is also used to search for a first mapping relationship where the operation ID is the first operation ID; the construction module is also used to construct a fourth mapping relationship between the first operation ID and all identification information in the found first mapping relationship; the backtracking module is also used to perform the following steps: Step S21: Search for the third mapping relationship including the identification information to be analyzed in the mapping relationship group to obtain a third result mapping, wherein the initial identification information to be analyzed is the target. The file identification information, the third result mapping includes the operation ID to be analyzed; Step S22: Search in the mapping relationship group for the fourth mapping relationship including the operation ID to be analyzed, to obtain the fourth result mapping, wherein the identification information included in the fourth result mapping is the component identification information; Step S23: Output the component identification information to the file aggregation tree, completing one recursion; Step S24: Determine whether the current recursion count meets the recursion depth threshold. If the current recursion count does not meet the recursion depth threshold, execute step S25. If the current recursion count meets the recursion depth threshold, execute step S26; Step S25: Assign the component identification information to the identification information to be analyzed, and return to step S21; Step S26: Output the file aggregation tree.

12. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 9.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9.

Images