Data processing method, device, storage medium and program product

Abstract

The embodiments of the present application provide a data processing method, device, storage medium and program product. In the embodiments of the present application, the currently used data set and the historical recycling rounds in which the recycled garbage data have not been deleted can be determined; and when the historical garbage data recycled in the historical recycling rounds are no longer used, the data to be recovered is determined from the historical garbage data according to the currently used data set and the historical garbage data recycled in the historical recycling rounds, and the data to be recovered is restored to the storage system and the historical garbage data is deleted, which not only realizes the lock-free garbage data recycling, but also solves the problem of accidental deletion caused by the lock-free garbage recycling method by delaying the deletion of historical garbage data.

Description

Technical Field

[0001] The present application relates to the field of data storage technology, and in particular to data processing methods, devices, storage media and program products. Background Art

[0002] With the explosive growth of digital information, the data in the backup system has increased, and the data storage management has become more complicated. The adoption of data deduplication technology has optimized storage efficiency, but it has also brought about the problem of data garbage generated during data deletion. How to clean up these storage garbage is an important technical problem that needs to be solved in the backup system. Summary of the invention

[0003] Multiple aspects of the present application provide a data processing method, device, storage medium and program product to implement data garbage collection.

[0004] The present application provides a data processing method, including:

[0005] determining a first data set currently in use;

[0006] Determine the historical recycling rounds in which the recycled garbage data has not been deleted;

[0007] In a case where the historical garbage data recovered in the historical recycling round is no longer used, determining data to be recovered from the historical garbage data according to the first data set;

[0008] The data to be restored is restored to the storage system; and the historical junk data is deleted.

[0009] The embodiment of the present application further provides a computer device, comprising: a memory and a processor; wherein the memory is used to store a computer program and a storage system;

[0010] The processor is coupled to the memory and is configured to execute the computer program to perform the steps in the above data processing method.

[0011] An embodiment of the present application also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed by one or more processors, the one or more processors are caused to execute the steps in the above-mentioned data processing method.

[0012] An embodiment of the present application also provides a computer program product, including: a computer program; the computer program is executed by a processor to implement the above-mentioned data processing method.

[0013] In an embodiment of the present application, it is possible to determine historical recycling rounds in which the currently used data set and the recycled garbage data have not been deleted; and when the historical garbage data recovered in the historical recycling rounds is no longer used, the data to be recovered is determined from the historical garbage data based on the currently used data set and the historical garbage data recovered in the historical recycling rounds, and the data to be recovered is restored to the storage system and the historical garbage data is deleted. This not only realizes lock-free garbage data recovery, but also solves the problem of accidental deletion caused by the lock-free garbage recovery method by delaying the deletion of historical garbage data. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0015] Figure 1a and Figure 1b A flowchart of a data processing method provided in an embodiment of the present application;

[0016] Figure 2 A schematic diagram of the timeline of GC operation and upload operation provided in an embodiment of the present application;

[0017] Figure 3 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0019] With the explosive growth of digital information, cloud storage applications have emerged. Users can upload local data to backup systems such as cloud disks and network disks for data storage. As the data in the backup system increases, data storage management becomes more complex. The adoption of data deduplication technology optimizes storage efficiency. Among them, data deduplication technology is a data reduction technology that is usually used in disk-based backup systems to reduce the storage capacity used in storage systems. It works by finding repeated variable-size data blocks in different locations in different files within a certain time period. Duplicate data blocks are replaced with indicators.

[0020] Garbage Collection (GC) is to remove data that is no longer referenced in the data deleted by the data deduplication technology. For example, assuming that the data deleted by the data deduplication technology is data set R, garbage collection (GC) is to delete data that is no longer referenced in data set R. Correspondingly, garbage data refers to data that is no longer referenced in the data deleted by the data deduplication technology.

[0021] For the locked garbage collection mechanism, the file backup service needs to be stopped during the execution of locked garbage collection. That is, during the execution of locked GC, users cannot continue to upload data to the backup system, affecting the quality of the backup service. On the other hand, the locked GC method is not applicable to some scenarios that require continuous backup. For example, the incremental backup scenario requires continuous data backup or high-frequency data backup, and the locked GC is not suitable for incremental backup scenarios.

[0022] Based on the above technical problems, the embodiment of the present application provides a lock-free GC method. In the embodiment of the present application, in order to realize garbage data recovery, the currently used data set and the historical recovery rounds in which the recovered garbage data are not deleted can be determined; and when the historical garbage data recovered in the historical recovery rounds are no longer used, the data to be recovered is determined from the historical garbage data according to the currently used data set and the historical garbage data recovered in the historical recovery rounds, and the data to be recovered is restored to the storage system and the historical garbage data is deleted, which not only realizes lock-free garbage data recovery, but also solves the problem of accidental deletion caused by the lock-free garbage recovery method by delaying the deletion of historical garbage data.

[0023] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0024] It should be noted that the same reference numerals denote the same objects in the following drawings and embodiments, and therefore, once an object is defined in one drawing or embodiment, it does not need to be further discussed in the subsequent drawings and embodiments.

[0025] Figure 1a The following is a flow chart of a data processing method provided in an embodiment of the present application. Figure 1a As shown, the method includes:

[0026] 101. Determine the dataset currently in use.

[0027] 102. Determine the historical recycling rounds in which the recycled garbage data has not been deleted.

[0028] 103. When the time information associated with the historical recycling round meets the set conditions, the data to be recovered is determined from the historical garbage data according to the currently used data set and the historical garbage data recycled in the historical recycling round.

[0029] 104. Restore the data to be restored to the storage system.

[0030] 105. Delete the historical garbage data collected in the historical recycling rounds.

[0031] In this embodiment, in order to save storage space of the storage system, duplicate data stored in the storage system may be deleted. The storage system may be a cloud storage system, which may be called a backup system in some embodiments, and is mainly used to store data uploaded by users, such as cloud disk data backup. The recycling round refers to the garbage collection (GC) round. The purpose of GC is to delete data that is no longer referenced in the data deleted by the data deduplication technology. In this embodiment, in order to realize garbage data recycling, in step 101, the currently used data set An may be obtained.

[0032] The currently used data set An is the data set used in the current GC round, specifically the data set currently used by the storage system, that is, the data set formed by the deduplicated data in the storage system. The current GC round may be the nth round; n≥2 and is a positive integer.

[0033] For example, in some embodiments, during the deduplication process, the data stored in the storage system may be divided into blocks to obtain data blocks (chunks). The size of the data blocks may be flexibly set according to actual needs. Deduplication may be performed between different files in the storage system in units of data blocks to achieve deduplication. The data set currently used by the storage system is the data set formed by the data blocks after deduplication. For example, for file X, it can be divided into data blocks X1, X2, and X3; for file Y, it can be divided into data blocks Y1 and Y2. Among them, data block X1 and data block Y1 are duplicate data blocks. The storage system may record that the data blocks actually used by file X are data blocks X1, X2, and X3; and the data blocks actually used by file Y are data blocks X1 and Y2. Based on this, the data set currently used by the storage system is data blocks X1, X2, X3 and data block Y2.

[0034] Based on the above analysis, it can be determined that the data corresponding to the identifier of the actual use data recorded by the storage system is the currently used data; the currently used data forms a data set An. For example, it can be determined that the data block corresponding to the actual use database identifier (chunk-id) recorded by the storage system is the currently used data block, and the data set formed by the currently used data block is determined to be the currently used data set An.

[0035] Furthermore, the garbage data recovery method provided in this embodiment is a lock-free GC method. The lock-free GC method does not affect the upload operation to the storage system during the garbage data recovery process. Therefore, the lock-free GC method provided in this embodiment is suitable for scenarios that require continuous data upload, such as incremental backup scenarios. In this embodiment, during the GC process, users can also upload new data to the storage system, which may cause the newly uploaded data during the GC process to be mistakenly regarded as unused data in the current GC round during garbage data recovery, resulting in the garbage data recovered by GC being accidentally deleted.

[0036] In this embodiment, in order to solve the problem of accidental data deletion caused by the lock-free GC method, the garbage data Ci recovered by each GC is delayed for deletion, and the garbage data Ci will be deleted when it will not be used again in the future. i refers to the i-th time, and i is a positive integer. Based on this, in this embodiment, the garbage data Ci can be determined for each GC. Optionally, the currently used data set Ai and the data set Bi currently stored in the storage system can be determined; and based on the currently used data set Ai and the data set Bi currently stored in the storage system, the garbage data (denoted as Ci) corresponding to the current recycling round (denoted as GCi) is determined. In this embodiment, current refers to the execution period of each GC. For example, for the i-th GC, current refers to the execution period of the i-th GC.

[0037] Optionally, data that does not belong to the currently used data set Ai may be obtained from the data set Bi currently stored in the storage system as garbage data Ci to be recycled in the current recycling round.

[0038] Optionally, the data in the data set Ai can be mapped to the data set Bi; and the data in the data set Bi that is not mapped is determined as the garbage data Ci recovered in the current recycling round. Alternatively, the data in the data set Bi can be queried in the data set Ai, and the data that is not queried in the data set Ai can be determined from the data set Bi as the garbage data Ci recovered in the current recycling round.

[0039] In the embodiments of the present application, the triggering method of GC is not limited. In some embodiments, a recycling cycle can be set, and a timer or counter can be started to time the recycling cycle. Whenever a recycling cycle is reached, a GC operation is performed. Alternatively, a GC operation can also be performed under the condition that data is uploaded or deleted in the storage system; but it is not limited to this.

[0040] In this embodiment, in order to solve the problem of accidental data deletion caused by the lock-free GC method, the garbage data Ci can be deleted after it is no longer used. Based on this, in this embodiment, the garbage data Ci recovered each time is migrated from the storage system to the garbage collection file (GC file), and when the garbage data Ci is no longer used, the garbage data Ci is deleted from the GC file. Among them, the GC file is a waiting deletion area, which can be stored in the physical machine deployed by the storage system, and can also be stored on other physical machines.

[0041] In this embodiment, there is no historical recycling round before the first GC. Therefore, it is impossible to reduce the process of mistaken deletion during the first GC. Therefore, in the following embodiments, the GC method provided in the embodiment of the present application is exemplified by focusing on the above-mentioned nth GC as an example. Among them, the nth time can be any time from the second time to the second time.

[0042] Based on the above analysis, in this embodiment, in order to solve the problem of accidental data deletion caused by lock-free GC, in step 102, the historical recycling rounds in which the garbage data was not deleted can be determined. Assuming that the current recycling round is the nth GC (denoted as GCn), the historical recycling rounds can be one or some of the previous (n-1) GCs, which can be denoted as GC j ; j = 1, 2, ..., n-1. n≥2, and is a positive integer. Regarding the process of determining the recycled garbage data in the historical recycling rounds, please refer to the relevant content of determining the garbage data in each GC mentioned above, which will not be repeated here. The historical recycling rounds described in the following embodiments all refer to historical recycling rounds in which the recycled garbage data has not been deleted. Assuming that the garbage data recycled by the (n-3)th GC, the (n-2)th GC and the (n-1)th GC in the previous (n-1)th GC is not deleted, the historical recycling rounds refer to the (n-3)th GC, the (n-2)th GC and the (n-1)th GC.

[0043] Further, in step 103, the historical garbage data (denoted as C j ) is no longer used, based on the currently used data set An, the historical garbage data C recovered from the historical recycling round j The data to be recovered Dn is determined in the historical recycling round. j ) will be used, and the historical garbage data C collected in the historical collection rounds will continue to be retained j .

[0044] In this embodiment, the historical garbage data C collected in the historical collection round j No longer in use can be divided into at least two of the following situations:

[0045] Case 1: The upload operations during the execution of the historical recycling round are all completed before the current recycling round GCn starts. That is, the start time of the upload operations during the execution of the current recycling round GCn is later than the end time of the historical recycling round.

[0046] Case 2: Historical garbage data C collected in historical collection rounds j The time it takes to be recycled to the GC file exceeds the set time threshold. The time threshold can be set by the user or by the default time, such as 12 hours.

[0047] Based on the above situations 1 and 2, before step 103, the historical garbage data C collected by the historical recycling round can also be determined based on whether the time information associated with the historical recycling round meets the set conditions. j Will it be used? If the time information associated with the historical recycling round meets the set conditions, the historical garbage data C recycled by the historical recycling round is determined. j Correspondingly, if the time information associated with the historical recycling round does not meet the set conditions, the historical garbage data C recycled by the historical recycling round is determined. j Therefore, if the time information associated with the historical collection round does not meet the set conditions, the historical garbage data C stored in the GC file will continue to be retained. j .

[0048] Based on the above situations 1 and 2, it can be determined whether the end time of the historical recycling round is earlier than the start upload time of the current uploaded data; and / or it can be determined whether the time difference between the time when the historical garbage data recovered by the historical recycling round was migrated to the garbage recycling file and the current time exceeds the set time threshold. Further, for embodiments using different judgment methods, the judgment results are different, which are explained one by one below.

[0049] In some embodiments, it may be determined whether the end time of a historical recycling round is earlier than the start upload time of the currently uploaded data; if the determination result is yes, it is determined that the time information associated with the historical recycling round meets the set conditions.

[0050] In other embodiments, it can be determined whether the time difference between the time when historical garbage data recovered in historical recycling rounds is migrated to the garbage collection file and the current time exceeds a set duration threshold; if the judgment result is yes, it is determined that the time information associated with the historical recycling round meets the set conditions.

[0051] In some other embodiments, it can be determined whether the end time of the historical recycling round is earlier than the start upload time of the currently uploaded data, and whether the time difference between the time when the historical garbage data recovered by the historical recycling round is migrated to the garbage collection file and the current time exceeds the set duration threshold; if the judgment results are all yes, it is determined that the time information associated with the historical recycling round meets the set conditions; accordingly, if there is a case where the judgment result is no, that is, the judgment result is that the end time of the historical recycling round is later than the start upload time of the currently uploaded data, and / or the time difference between the time when the historical garbage data recovered by the historical recycling round is migrated to the garbage collection file and the current time is less than the set duration threshold, then it is determined that the time information associated with the historical recycling round does not meet the set conditions.

[0052] Further, for an embodiment in which the time information associated with the historical recycling round meets the set conditions, the historical garbage data C collected from the historical recycling round can be collected according to the currently used data set An. j The data Dn to be recovered is determined in.

[0053] Optionally, the historical garbage data C j The data belonging to the currently used data set An is obtained as the data to be restored Dn. The data to be restored Dn satisfies the following conditions: (Dn∈C j ) and (Dn∈An). For example, the currently used data set An can be added to the historical garbage data C j The dataset An is mapped in the historical garbage data C j The data mapped in is the data to be recovered Dn. Alternatively, the currently used data set An is used to recover the historical garbage data C j , and the historical garbage data C j The data queried in is used as the data to be restored Dn.

[0054] Further, in step 104, the data to be recovered Dn may be recovered to the storage system; and in step 105, the historical garbage data C recovered in the historical recovery rounds may be deleted accordingly. j .

[0055] Optionally, the data to be recovered can be recovered from the garbage collection file to the storage system; and the historical garbage data C recovered in the historical collection rounds can be deleted from the garbage collection file. j .

[0056] In this embodiment, the historical recycling rounds in which the currently used data set and the recycled garbage data have not been deleted can be determined; and when the historical garbage data recovered in the historical recycling rounds is no longer used, the data to be recovered is determined from the historical garbage data based on the currently used data set and the historical garbage data recovered in the historical recycling rounds, and the data to be recovered is restored to the storage system and the historical garbage data is deleted. This not only realizes lock-free garbage data recovery, but also solves the problem of accidental deletion caused by the lock-free garbage recovery method by delaying the deletion of historical garbage data.

[0057] On the other hand, since the lock-free GC method adopted in this embodiment can delay the deletion of garbage data, the GC operation is performed in a lock-free state and there is no time window requirement. Therefore, for the computer device deploying the data processing method, a smaller resource background can be used to run the above data processing method, which helps to reduce the resource consumption of the GC process.

[0058] In order to facilitate understanding of the above data processing process, the data processing method provided in the embodiment of the present application is described below by taking an embodiment of deleting duplicate data blocks as a unit of data blocks as an example.

[0059] Figure 1b FIG. 4 is a flow chart of a data processing method using the first and second GC as examples. Figure 1b As shown, for the first GC, during the first GC process, the used data set A1, the data set B1 stored in the storage system, and the recovered garbage data C1 are all data blocks. Figure 1b As shown in FIG. 1 , the garbage data C1 is a data block to be deleted. Further, the garbage data C1 can be migrated from the storage system to the GC file, that is, Figure 1b As shown in step 1, the data blocks to be deleted are migrated from the storage system to the waiting deletion area. The GC file can be implemented in the form of a directory, that is, a GC directory. The storage system can be implemented as a data block directory.

[0060] During the first GC process, the chunk identifier (chunk-id) corresponding to the garbage data C1 stored in the user's local key-value storage system may also be deleted. This GC is finished; wait for the next GC operation.

[0061] In this embodiment, deleting the data block identifier (chunk-id) corresponding to the garbage data C1 stored in the user's local key-value storage system and migrating the garbage data C1 from the storage system to the GC file can be performed in parallel or in series. If the data block identifier (chunk-id) corresponding to the garbage data C1 stored in the user's local key-value storage (KVstore) system is deleted and the garbage data C1 is migrated from the storage system to the GC file, these two steps are executed in series, and the order of execution is not limited. This is mainly because: in order to maintain the consistency between the local storage and the remote storage system, after deleting the garbage data C1 from the remote storage system, the chunk-id of the garbage data C1 stored in the local KVstore will be deleted accordingly. However, since there may be accidentally deleted data blocks in the garbage data C1, resulting in fewer chunk-ids recorded in the KVstore, when the database corresponding to the missing chunk-id is uploaded later, the corresponding chunk-id will be recorded again in the KVstore, and the chunk-id stored in the KVstore will be corrected. For the remote storage system, since the chunk corresponding to the chunk-id missing in the KVstore is uploaded again, this embodiment also performs deduplication and GC on the newly uploaded data block, so that the database stored in the remote storage system is corrected.

[0062] For the second GC, the dataset A2 used in the second GC process, the dataset B2 stored in the storage system, and the garbage data C2 recovered for the second time can be determined. Furthermore, it can be determined whether the end time of the first GC is earlier than the start time of uploading the data during the second GC process; and whether the time difference between the time when the historical garbage data C1 recovered by the first GC is migrated to the garbage collection file and the current time exceeds the set duration threshold; if the judgment results are all yes, then the data belonging to the dataset A2 used in the second GC process is obtained from the garbage data C1 recovered by the first GC as the data D1 to be recovered. The data D1 to be recovered meets the following conditions: (D1∈C1) and (D1∈A2). Furthermore, the historical garbage data C1 that meets the above judgment conditions can be deleted, that is, Figure 1b As shown in step 2, the data blocks meeting the conditions are deleted; and the to-be-recovered data D1 is restored to the storage system, that is, as shown in step 3, the data blocks (chunks) still in use are rolled back.

[0063] Next, continue to combine Figure 2 The timeline diagram shown in FIG. 1 is used to exemplarily illustrate the data processing method provided in the embodiment of the present application. Figure 2 In the figure, "upload" refers to an upload operation; "GC" refers to a GC operation; the horizontal axis is the time axis, and the time axis goes backward from left to right.

[0064] like Figure 2 As shown, for the first GC, i.e., GC1, the dataset A1 used, the dataset B1 stored in the storage system, and the recovered garbage data C1 can be determined during the GC1 process. For the specific implementation of determining the dataset A1, the dataset B1 stored in the storage system, and the recovered garbage data C1, please refer to the relevant contents of the above embodiment, which will not be repeated here. Figure 2 As shown, during the execution of GC1, there are upload operations "Upload 1" and "Upload 2". Due to the inconsistency problem, the accidentally deleted data contained in the garbage data C1 recovered by GC1 all come from the upload operations "Upload 1" and "Upload 2".

[0065] The GC1 operation can migrate the garbage data C1 from the storage system to the GC file and delete the chunk-id corresponding to the garbage data C1 stored in the local KVstore system. GC1 ends.

[0066] The second GC operation, namely GC2 operation, can be started to determine the data set A2 used, the data set B2 stored in the storage system, and the garbage data C2 recovered during GC1. Further, it is determined whether the start upload time of the uploaded data during the execution of GC2 is earlier than the end time of GC1, such as Figure 2 It can be seen that there is an "upload 3" operation during the execution of GC2, and the start upload time of the "upload 3" operation is later than the end time of GC1. Therefore, the garbage data C1 recovered by GC1 is no longer used. Furthermore, the data belonging to the data set A2 used in the second GC process can be obtained from the garbage data C1 recovered by the first GC as the data D1 to be recovered. The data D1 to be recovered meets the following conditions: (D1∈C1) and (D1∈A2). Furthermore, the data D1 to be recovered can be restored to the storage system; and the garbage data C1 recovered by GC1 can be deleted.

[0067] For GC2, the garbage data C2 recovered by GC2 can also be migrated from the storage system to the GC file, and the chunk-id corresponding to the garbage data C2 stored in the local KVstore system can be deleted. GC2 ends.

[0068] The third GC operation, namely GC3 operation, is started to determine the data set A3 used, the data set B3 stored in the storage system, and the garbage data C3 recovered during GC3. Further, it is determined whether the start upload time of the uploaded data during the execution of GC3 is earlier than the end time of GC2, such as Figure 2 It can be seen that there is an "upload 4" operation during the execution of GC2, and the start upload time of the "upload 4" operation is earlier than the end time of GC2. Therefore, the garbage data C2 recovered by GC2 may be used later. Therefore, the garbage data C2 recovered by GC2 stored in the GC file continues to be retained.

[0069] For GC3, the garbage data C3 recovered by GC3 can also be migrated from the storage system to the GC file, and the chunk-id corresponding to the garbage data C3 stored in the local KVstore system can be deleted. GC3 ends.

[0070] The fourth GC operation, namely GC4 operation, is started to determine the data set A4 used, the data set B4 stored in the storage system, and the garbage data C4 recovered during GC4. Further, it is determined whether the start upload time of the uploaded data during the execution of GC4 is earlier than the end time of GC2 and GC3, such as Figure 3 It can be seen that there is no upload operation during the execution of GC4, and the start upload time of the uploaded data during the execution of GC4 is earlier than the end time of GC2 and GC3. Therefore, the garbage data C2 recovered by GC2 and the garbage data C3 recovered by GC3 are no longer used.

[0071] Furthermore, data D2 belonging to data set A4 used in the fourth GC process can be obtained from garbage data C2 recovered by the second GC; data D2 satisfies the following conditions: (D2∈C2) and (D2∈A4); and data D3 belonging to data set A4 used in the fourth GC process can be obtained from garbage data C3 recovered by the third GC; data D3 satisfies the following conditions: (D3∈C3) and (D3∈A4). Among them, data D2 and D3 are restored to the storage system as data to be recovered; and garbage data C2 and C3 in the GC file are deleted.

[0072] For GC4, the garbage data C4 recovered by GC4 can also be migrated from the storage system to the GC file, and the chunk-id corresponding to the garbage data C4 stored in the local KVstore system can be deleted. GC4 ends.

[0073] Each GC operation is performed according to the data processing method provided in the above embodiment, which not only realizes lock-free garbage data collection, but also solves the problem of accidental deletion caused by the lock-free garbage collection method by delaying the deletion of historical garbage data. The above data processing process only takes 4 GCs as an example to illustrate the processing logic of the data processing method provided in this embodiment, and does not constitute a limitation.

[0074] It is worth noting that in the GC process provided in the embodiment of the present application, in addition to data upload operations, data download operations can also be performed. For data download operations, in response to the download operation, the data to be downloaded can be queried in the storage system; if the data to be downloaded is not queried in the storage system, the data to be downloaded is queried from the garbage collection file; and the data to be downloaded queried in the garbage collection file is provided to the initiator of the download operation.

[0075] It should be noted that the execution subject of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 101 and 102 can be device A; for another example, the execution subject of step 101 can be device A, and the execution subject of step 102 can be device B; and so on.

[0076] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations appearing in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel, and the serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel.

[0077] Accordingly, an embodiment of the present application also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed by one or more processors, the one or more processors are caused to execute the steps in the above-mentioned data processing method.

[0078] The embodiment of the present application also provides a computer program product, including: a computer program; the computer program is executed by a processor to implement the above-mentioned data processing method. The computer program product provided in this embodiment can be implemented as GC software or GC plug-in, and can be applied to a database system (Data Base System, DBS). The computer program product can be deployed on the same physical machine as the remote storage system, or on a different physical machine.

[0079] Figure 3 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. Figure 3 As shown, the computer device includes: a memory 30a and a processor 30b; wherein the memory is used to store computer programs and storage systems;

[0080] The processor is coupled to the memory and is used to execute a computer program for: determining a first data set currently in use; determining a historical recycling round in which the recycled garbage data has not been deleted; when the historical garbage data recycled in the historical recycling round is no longer in use, determining data to be recovered from the historical garbage data based on the first data set; restoring the data to be recovered to a storage system; and deleting the historical garbage data.

[0081] In some embodiments, the memory 30a also stores a garbage collection file. Accordingly, the processor 30b is further configured to retain the historical garbage data in the garbage collection file when the historical garbage data will be used.

[0082] Optionally, the processor 30b is also used to perform the following judgment operation: judging whether the end time of the historical recycling round is earlier than the start upload time of the currently uploaded data; and / or judging whether the time difference between the time when the historical garbage data recovered by the historical recycling round is migrated to the garbage recycling file and the current time exceeds a set duration threshold; if the result of at least one judgment operation is yes, it is determined that the historical garbage data recovered by the historical recycling round is no longer used.

[0083] Optionally, when determining the data to be restored from the junk data, the processor 30b is specifically configured to: obtain data belonging to the first data set from the historical junk data as the data to be restored.

[0084] Accordingly, when restoring the data to be restored to the storage system, the processor 30b is specifically configured to restore the data to be restored from the garbage collection file to the storage system.

[0085] Correspondingly, when deleting historical garbage data, the processor 30b is specifically used to: delete the historical garbage data from the garbage collection file.

[0086] In some embodiments, the processor 30b is further used to: determine a second data set currently stored in the storage system; determine garbage data recovered in the current recycling round based on the first data set and the second data set; and migrate the garbage data recovered in the current recycling round to a garbage collection file.

[0087] Further, when determining the garbage data corresponding to the current recycling round, the processor 30b is specifically configured to: obtain data that does not belong to the first data set from the second data set as the garbage data recycled in the current recycling round.

[0088] Optionally, the processor 30b is also used to: in response to a download operation, query the storage system for data to be downloaded; if the data to be downloaded is not found in the storage system, query the data to be downloaded from the garbage collection file; and provide the data to be downloaded found in the garbage collection file to the initiator of the download operation through the communication component 30c.

[0089] In some optional embodiments, such as Figure 3 As shown, the computer device may also include: power supply component 30d and other components. In the embodiments of the present application, the implementation form of the computer device is not limited. The computer device may be a single server device, a cloud server array, or a virtual machine (VM) running in a cloud server array. In addition, the server device may also refer to other computing devices with corresponding service capabilities, such as computers and other terminal devices (running service programs), etc. For terminal devices, such as Figure 3As shown, the computer device may further include other components such as a display component 30e and an audio component 30f. Figure 3 The components are shown only schematically, and do not mean that the computer equipment must include Figure 3 The components shown do not necessarily mean that the computer equipment can only include Figure 3 Components shown.

[0090] The computer device provided in this embodiment can determine the historical recycling rounds in which the currently used data set and the recycled garbage data have not been deleted; and when the historical garbage data recovered in the historical recycling rounds is no longer used, the data to be recovered is determined from the historical garbage data based on the currently used data set and the historical garbage data recovered in the historical recycling rounds, and the data to be recovered is restored to the storage system and the historical garbage data is deleted. This not only realizes lock-free garbage data recycling, but also solves the problem of accidental deletion caused by the lock-free garbage collection method by delaying the deletion of historical garbage data.

[0091] In an embodiment of the present application, the memory is used to store a computer program and can be configured to store various other data to support operations on the device where it is located. Among them, the processor can execute the computer program stored in the memory to implement the corresponding control logic. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0092] In the embodiment of the present application, the processor can be any hardware processing device that can execute the logic of the above method. Optionally, the processor can be a central processing unit (CPU), a graphics processing unit (GPU) or a microcontroller unit (MCU); it can also be a field programmable gate array (FPGA), a programmable array logic device (PAL), a general array logic device (GAL), a complex programmable logic device (CPLD) and other programmable devices; or an advanced reduced instruction set (RISC) processor (Advanced RISC Machines, ARM) or a system on chip (System on Chip, SOC), etc., but not limited to this.

[0093] In an embodiment of the present application, the communication component is configured to facilitate wired or wireless communication between the device in which it is located and other devices. The device in which the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G, 5G or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component can also be implemented based on near field communication (NFC) technology, radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology or other technologies.

[0094] In an embodiment of the present application, the display component may include a liquid crystal display (LCD) and a touch panel (TP). If the display component includes a touch panel, the display component may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0095] In an embodiment of the present application, a power supply component is configured to provide power to various components of the device in which it is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply component is located.

[0096] In an embodiment of the present application, the audio component may be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC), and when the device where the audio component is located is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal may be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting an audio signal. For example, for a device with a language interaction function, voice interaction with a user can be achieved through an audio component.

[0097] It should be noted that the descriptions such as “first” and “second” in this article are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do they limit “first” and “second” to different types.

[0098] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0099] The present application is described with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, and the combination of the process and / or box in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one process or multiple processes in the flowchart and / or one box or multiple boxes in the block diagram.

[0100] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0101] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0102] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0103] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0104] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0105] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0106] The above is only the embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A data processing method, characterized in that: include: determining a first data set currently in use; Determine the historical recycling rounds in which the recycled garbage data has not been deleted; When the time information associated with the historical recycling round meets a set condition, determining that the historical garbage data recycled in the historical recycling round is no longer used, and determining data to be recovered from the historical garbage data according to the first data set; Restoring the data to be restored to the storage system; And delete the historical junk data.

2. The method according to claim 1, characterized in that The step of determining the data to be restored from the historical garbage data according to the first data set includes: From the historical garbage data, data belonging to the first data set is obtained as the data to be restored.

3. The method according to claim 1, characterized in that The step of restoring the data to be restored to the storage system includes: Restoring the data to be restored from the garbage collection file to the storage system; The deleting of the historical junk data includes: The historical garbage data is deleted from the garbage collection file.

4. The method according to claim 1, characterized in that: Also includes: determining a second data set currently stored in the storage system; Determining garbage data to be recycled in a current recycling round according to the first data set and the second data set; Migrate the garbage data collected in the current collection round to the garbage collection file.

5. The method according to claim 4, characterized in that The determining, according to the first data set and the second data set, garbage data corresponding to the current recycling round includes: Data that does not belong to the first data set is obtained from the second data set as garbage data collected in the current collection round.

6. The method according to claim 1, characterized in that Also includes: In the case where the historical garbage data will be used, the historical garbage data is retained in the garbage collection file.

7. The method according to any one of claims 1 to 6, characterized in that: It also includes performing at least one of the following judgment operations: Determine whether the end time of the historical recycling round is earlier than the start time of uploading the current uploaded data; and / or, Determine whether the time difference between the time when the historical garbage data collected in the historical recycling round was migrated to the garbage collection file and the current time exceeds a set time threshold; If the result of the at least one judgment operation is yes, it is determined that the time information associated with the historical recycling round meets the set condition.

8. The method according to claim 4, characterized in that Also includes: In response to the download operation, querying the storage system for data to be downloaded; If the data to be downloaded is not found in the storage system, searching the garbage collection file for the data to be downloaded; The data to be downloaded found in the garbage collection file is provided to the initiator of the download operation.

9. A computer device, characterized in that: include: Memory and processor; wherein the memory is used to store computer programs and storage systems; The processor is coupled to the memory and is configured to execute the computer program to perform the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium storing computer instructions, characterized in that: When the computer instructions are executed by one or more processors, the one or more processors are caused to execute the steps in the method according to any one of claims 1 to 8.

11. A computer program product, characterized in that include: Computer program; the computer program is executed by a processor to implement the method according to any one of claims 1 to 8.

Images