Over-date backup data cleaning method, computer equipment, readable storage medium and program product

Abstract

The invention relates to an expired backup data cleaning method, computer equipment, a computer readable storage medium and a computer program product, which are applied to the technical field of big data, and the method comprises the following steps: acquiring a backup expiration identification condition for backup data, and according to the backup expiration identification condition, determining the backup expiration identification condition for the backup data; identifying a plurality of pieces of expired backup data in the backup data; according to data distribution information determined by the data sizes of the multiple pieces of expired backup data, the distribution modes corresponding to the multiple pieces of expired backup data are identified, and the data distribution information is used for representing quantity distribution conditions of different data size orders in the multiple pieces of expired backup data; according to the distribution mode, processing threads are distributed to the multiple pieces of expired backup data, and the total amount of thread processing data of all the processing threads is distributed uniformly; and performing parallel cleaning on the expired backup data obtained by respective corresponding distribution through the processing threads. By adopting the method, the cleaning efficiency of the expired backup data can be improved.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and in particular to a method for cleaning expired backup data, a computer device, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the development of information technology and digital economy, data is closely related to the lives of each of us. The massive data generated by computer networks supports the operation of society in the information age. When software system failures, virus invasions, human deletions and other events occur, if the data can be restored to the state before the accident, the impact of the accident can be greatly reduced and the economic losses can be mitigated. Therefore, data is usually backed up.

[0003] However, when there is too much backup data, if the expired backup data is not cleaned up in time, and the space used for data backup is limited, it is easy to fail to back up the newly generated data. Therefore, there is an urgent need for a method to improve the efficiency of cleaning expired backup data. Summary of the invention

[0004] Based on this, it is necessary to provide an expired backup data cleaning method, device, computer equipment, computer-readable storage medium and computer program product that can improve the cleaning efficiency of expired backup data in response to the above technical problems.

[0005] In a first aspect, the present application provides a method for clearing expired backup data, comprising:

[0006] Acquiring a backup expiration identification condition for the backup data, and identifying a plurality of expired backup data in the backup data according to the backup expiration identification condition;

[0007] Identify allocation modes corresponding to the multiple expired backup data according to data distribution information determined by the data sizes of the multiple expired backup data, wherein the data distribution information is used to characterize the quantity distribution status of different data size magnitudes in the multiple expired backup data;

[0008] According to the allocation method, processing threads are respectively allocated to the plurality of expired backup data, wherein the total amount of thread processing data of each processing thread is evenly allocated;

[0009] The processing threads respectively clean up the corresponding allocated expired backup data in parallel.

[0010] In a second aspect, the present application also provides an expired backup data cleaning device, comprising:

[0011] An acquisition module, used for acquiring a backup expiration identification condition for backup data, and identifying a plurality of expired backup data in the backup data according to the backup expiration identification condition;

[0012] an identification module, configured to identify allocation modes corresponding to the plurality of expired backup data according to data distribution information determined by the data sizes of the plurality of expired backup data, wherein the data distribution information is used to characterize the quantity distribution status of different data size magnitudes in the plurality of expired backup data;

[0013] An allocation module, configured to allocate processing threads to the plurality of expired backup data respectively according to the allocation mode, wherein the total amount of thread processing data of each processing thread is evenly allocated;

[0014] The cleaning module is used to clean up the expired backup data corresponding to each processing thread in parallel.

[0015] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0016] Acquiring a backup expiration identification condition for the backup data, and identifying a plurality of expired backup data in the backup data according to the backup expiration identification condition;

[0017] Identify allocation modes corresponding to the multiple expired backup data according to data distribution information determined by the data sizes of the multiple expired backup data, wherein the data distribution information is used to characterize the quantity distribution status of different data size magnitudes in the multiple expired backup data;

[0018] According to the allocation method, processing threads are respectively allocated to the plurality of expired backup data, wherein the total amount of thread processing data of each processing thread is evenly allocated;

[0019] The processing threads respectively clean up the corresponding allocated expired backup data in parallel.

[0020] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0021] Acquiring a backup expiration identification condition for the backup data, and identifying a plurality of expired backup data in the backup data according to the backup expiration identification condition;

[0022] Identify allocation modes corresponding to the multiple expired backup data according to data distribution information determined by the data sizes of the multiple expired backup data, wherein the data distribution information is used to characterize the quantity distribution status of different data size magnitudes in the multiple expired backup data;

[0023] According to the allocation method, processing threads are respectively allocated to the plurality of expired backup data, wherein the total amount of thread processing data of each processing thread is evenly allocated;

[0024] The processing threads respectively clean up the corresponding allocated expired backup data in parallel.

[0025] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0026] Acquiring a backup expiration identification condition for the backup data, and identifying a plurality of expired backup data in the backup data according to the backup expiration identification condition;

[0027] Identify allocation modes corresponding to the multiple expired backup data according to data distribution information determined by the data sizes of the multiple expired backup data, wherein the data distribution information is used to characterize the quantity distribution status of different data size magnitudes in the multiple expired backup data;

[0028] According to the allocation method, processing threads are respectively allocated to the plurality of expired backup data, wherein the total amount of thread processing data of each processing thread is evenly allocated;

[0029] The processing threads respectively clean up the corresponding allocated expired backup data in parallel.

[0030] The above-mentioned expired backup data cleaning method, device, computer equipment, computer-readable storage medium and computer program product obtain backup expiration identification conditions for backup data, and identify multiple expired backup data in the backup data according to the backup expiration identification conditions; identify the allocation mode corresponding to the multiple expired backup data according to data distribution information determined by the data size of the multiple expired backup data, wherein the data distribution information is used to characterize the quantity distribution status of different data size orders of magnitude in the multiple expired backup data; allocate processing threads to the multiple expired backup data according to the allocation mode, wherein the total amount of thread processing data of each processing thread is evenly distributed; and perform parallel cleaning of the corresponding allocated expired backup data by each processing thread, use the data distribution information as the identification basis of the allocation mode, and then perform thread allocation according to the identified allocation mode, so that the expired backup data allocated to each processing thread matches the data distribution status of the expired backup data, and realizes the reasonable and even distribution of the total amount of thread processing data of each processing thread, thereby improving the cleaning efficiency of expired backup data. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1 A schematic diagram of thread allocation for expired backup data in a scenario;

[0033] Figure 2 A schematic diagram of thread allocation for expired backup data in another scenario;

[0034] Figure 3 A schematic diagram of thread allocation of expired backup data when an expired backup data cleaning method is used in a scenario;

[0035] Figure 4 A schematic diagram of thread allocation of expired backup data when an expired backup data cleaning method is used in another scenario;

[0036] Figure 5 A diagram of an application environment of a method for clearing expired backup data in an embodiment;

[0037] Figure 6 A schematic diagram of a flow chart of a method for clearing expired backup data in one embodiment;

[0038] Figure 7A schematic diagram of external communication involved in a method for clearing expired backup data in one embodiment;

[0039] Figure 8 A schematic diagram of a flow chart of steps for obtaining a backup expiration identification condition for backup data in one embodiment;

[0040] Fig. 9 It is a flowchart of a step of performing parallel cleaning of the expired backup data allocated to each processing thread in one embodiment;

[0041] Fig.10 It is a structural block diagram of an expired backup data cleaning device in one embodiment;

[0042] Fig.11 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0044] It should be noted that the information and data involved in this application (including but not limited to data for analysis, stored data, displayed data, etc.) are all information and data authorized by the user or fully authorized by all parties, and the acquisition, transmission, storage, use and processing of relevant data are in compliance with the relevant provisions of national laws and regulations. Content pushed to users (for example, visualization processes for parallel cleanup of processing threads, etc.) can be rejected by users or can be easily rejected by users. In the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be considered as exemplary. Their purpose is only to illustrate the feasibility of the implementation of the technical solution of this application, but it does not mean that the applicant has or will necessarily use the solution.

[0045] It is understandable that currently, the expired backup data is usually randomly assigned to the processing threads available in the system, and each processing thread cleans up the expired backup data corresponding to the assigned data in parallel. When this method is used, the following may occur: Figure 1 or Figure 2 In the case shown in the figure, the categories A, B and C are used to identify expired backup data. From the size of expired backup data of different types in the figure, it can be seen that the data size of expired backup data of category A is smaller than the data size of expired backup data of category B; the data size of expired backup data of category B is smaller than the data size of expired backup data of category C; Figure 1 and Figure 2In the data allocation scenario shown, there is a situation where the total amount of expired backup data allocated to the processing threads 1, 2, ..., N is quite different, that is, the time required for the data processing by the processing threads 1, 2, ..., N is quite different. Then, when the processing threads 1, 2, ..., N are processed in parallel, some processing threads (for example, Figure 1 and Figure 2 The processing thread 1 in has already processed the expired backup data, and the other processing threads (for example, Figure 1 and Figure 2 In the example, the processing thread 2 and the processing thread N in the processing are still processing expired backup data, that is, the total amount of thread processing data allocated between the processing threads of the same processing capability is uneven. At this time, the time required for the overall expired backup data cleaning is the time required for the processing thread N to complete the processing of the corresponding allocated thread processing data. Therefore, it is easy to cause the overall expired backup data cleaning efficiency to be low.

[0046] against Figure 1 In the scenario where the data total amount differences between the different data size levels (class A, class B and class C expired backup data in the figure) are all within the preset difference range, after adopting the backup expired data processing method provided in this embodiment, the first allocation method is adopted to allocate processing threads, referring to Figure 3 , processing thread 1 is only allocated Class A expired backup data, processing thread 2 is only allocated Class B expired backup data, and processing thread N is only allocated Class C expired backup data. That is, the expired backup data allocated to the processing threads all belong to the same data size level, and multiple processing threads complete the processing almost at the same time, so that the total amount of thread processing data allocated between the processing threads belonging to the same processing capacity is evenly distributed, thereby improving the overall expired backup data cleaning efficiency.

[0047] For example Figure 2 In the scenario where the difference in the total amount of data between the different data size levels (class A, class B and class C expired backup data in the figure) is not within the preset difference range, after adopting the backup expired data processing method provided in this embodiment, the second allocation method is adopted to allocate processing threads, referring to Figure 4 , the total amount of thread processing data corresponding to processing threads 1, 2, ..., N of the same processing capability has little difference between each other, so the time required for data processing corresponding to processing threads 1, 2, ..., N respectively has little difference, and multiple processing threads complete processing almost at the same time, so that the total amount of thread processing data allocated to each processing thread is evenly distributed, thereby improving the overall efficiency of clearing expired backup data.

[0048] The expired backup data cleaning method provided in the embodiment of the present application can be applied to Figure 5In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The server 104 obtains the backup expiration identification conditions for the backup data, and identifies multiple expired backup data in the backup data according to the backup expiration identification conditions; according to the data size of the multiple expired backup data, the processing threads are respectively allocated to the multiple expired backup data, wherein the difference in the total amount of thread processing data corresponding to the processing threads of the same processing capacity between the two processing data is less than the preset difference threshold; and the expired backup data corresponding to each processing thread is cleaned up in parallel. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices, and the Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, projection devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The head mounted device may be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The server 104 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0049] In an exemplary embodiment, Figure 6 As shown, a method for cleaning expired backup data is provided, and the method is applied to Figure 1 The server 104 in the example is used as an example to illustrate, including the following steps 202 to 208. Among them:

[0050] Step 202: Acquire a backup expiration identification condition for the backup data, and identify a plurality of expired backup data in the backup data according to the backup expiration identification condition.

[0051] Among them, the backup expiration identification condition in step 202 can be configured by the user as needed, and can also be configured according to the project corresponding to the backup data. There is no limitation here. Each backup data corresponds to a backup data number. The earlier the data backup time corresponding to the backup data, the smaller the data number. The backup expiration identification condition includes at least one of the first identification condition and the second identification condition. The first identification condition is a condition for the backup data number, and the second identification condition is a condition for the data backup time.

[0052] As an embodiment, obtaining the backup expiration identification condition for the backup data includes: obtaining the first identification condition and / or the second identification condition for the backup data configured by the user as needed.

[0053] As another embodiment, obtaining a backup expiration identification condition for backup data includes: configuring a first identification condition and / or a second identification condition for the backup data according to project information of the project corresponding to the backup data, specifically, the first identification condition includes a backup data number range corresponding to the non-expired backup data, the second identification condition includes a data backup time range corresponding to the non-expired backup data, the project information includes at least one of a total amount of project data and a project data update frequency, the larger the total amount of project data, the more backup data numbers are included in the backup data number range, the higher the project data update frequency, and the later the data backup time in the data backup time range.

[0054] As an embodiment, multiple expired backup data in the backup data are identified according to the backup expiration identification condition, including: if the backup expiration identification condition includes a backup data number range corresponding to the non-expired backup data, then when the backup data number of the backup data is not within the backup data number range corresponding to the non-expired backup data, the backup data is identified as expired backup data.

[0055] As another embodiment, if the backup expiration identification condition includes the data backup time range corresponding to the unexpired backup data, the backup data is identified as expired backup data when the data backup time of the backup data is not within the data backup time range corresponding to the unexpired backup data.

[0056] Step 204 , identifying allocation modes corresponding to the multiple expired backup data according to data distribution information determined by the data sizes of the multiple expired backup data, wherein the data distribution information is used to characterize the quantity distribution status of different data size levels in the multiple expired backup data.

[0057] As an embodiment, step 204 includes: if the data distribution information represents that among multiple expired backup data, the corresponding total data amount differences between different data size levels are all within a preset difference range, then the first allocation method is identified as the allocation method corresponding to the multiple expired backup data, wherein, under the first allocation method, the expired backup data allocated to each processing thread all belong to the same data size level; if the data distribution information represents that among multiple expired backup data, the corresponding total data amount differences between different data size levels are not all within the preset difference range, then the second allocation method is identified as the allocation method corresponding to the multiple expired backup data, wherein, under the second allocation method, the expired backup data allocated to each processing thread belongs to at least one data size level.

[0058] Further, step 204 includes: determining the total amount of data levels of each data size level in multiple expired backup data based on the data distribution information; if the total amount differences between the total amount of data levels of each data size level are all within the preset difference range, then identifying the first allocation method as the allocation method corresponding to the multiple expired backup data; if the total amount differences between the total amount of data levels of each data size level are not all within the preset difference range, then identifying the second allocation method as the allocation method corresponding to the multiple expired backup data.

[0059] Step 206 , assigning processing threads to the plurality of expired backup data respectively according to the allocation mode, wherein the total amount of thread processing data of each processing thread is evenly distributed.

[0060] As an embodiment, step 206 includes: if the allocation method is the first allocation method, classifying each processing thread according to the data size levels corresponding to the multiple expired backup data, to obtain a thread classification result; and according to the thread classification results of each processing thread, respectively allocating processing threads to the multiple expired backup data.

[0061] Furthermore, each processing thread is classified according to the data size levels corresponding to the multiple expired backup data to obtain a thread classification result, including: each type of processing thread represented by the thread classification result corresponds to a data size level, for example, Class A processing thread corresponds to expired backup data of Class A data size level.

[0062] As another embodiment, step 206 includes: if the allocation method is the second allocation method, then determining the processing limit information of each processing thread according to the total amount of expired backup data of multiple expired backup data, the total number of processing threads and the total amount of processable data of each processing thread; data selection step: selecting unallocated expired backup data from the expired backup data; allocating processing threads to the unallocated expired backup data according to the data size of the unallocated expired backup data, the current processing occupancy information of each processing thread and the processing limit information of each processing thread, wherein the current processing occupancy information is used to characterize the processing occupancy of the processing thread under the currently allocated expired backup data; returning to the data selection step until there is no unallocated expired backup data in the expired backup data.

[0063] Among them, the total amount of processable data is used to characterize the processing capacity limit of the processing thread, and the processing limit information is used to characterize the current maximum amount of processed data of the processing thread; the unallocated expired backup data is the expired backup data of the currently unallocated processing thread; the current processing occupancy information includes the current processing occupied data amount, the processing limit information includes the current maximum processing data amount, and the current processing occupied data amount is the difference between the current maximum processing data amount and the total amount of allocated data of the expired backup data allocated to the processing thread.

[0064] Further, based on the total amount of expired backup data of multiple expired backup data, the total number of processing threads and the total amount of processable data of each processing thread, the processing limit information of each processing thread is determined, including: determining the ratio between the total amount of expired backup data of multiple expired backup data and the total number of processing threads as the average amount of expired backup data; if the total amount of processable data of each processing thread is greater than or equal to the average amount of expired backup data, then determining the average amount of expired backup data as the processing limit information of each processing thread; if the total amount of processable data of each processing thread is not greater than or equal to the average amount of expired backup data, then selecting the corresponding first threads whose total amount of processable data is less than the average amount of expired backup data from each processing thread, and determining the total amount of processable data of each first thread as the processing limit information of each first thread, and determining the difference between the total amount of expired backup data and the sum of the total amount of processable data of all first threads, and the ratio between the total number of threads of second threads in the processing threads, as the processing limit information of the second thread, wherein the second thread is a thread in the processing thread other than the first thread.

[0065] In this way, it can be ensured that the processing limit information set for the same processing capacity is the same and compatible with the total amount of expired backup data, that is, the current maximum amount of processed data is the same, and the processing limit information is the basis for subsequent data allocation. Therefore, to a certain extent, it can be ensured that the difference between the total amount of expired backup data obtained by the corresponding allocation of the same processing capacity is small.

[0066] As an embodiment, processing threads are allocated to unallocated expired backup data based on the data size of unallocated expired backup data, current processing occupancy information of each processing thread, and processing limit information of each processing thread, including: determining the remaining processing data volume of each processing thread as the difference between the current maximum processing data volume of each processing thread and the current processing occupied data volume of each processing thread; and allocating the unallocated expired backup data to the processing thread whose corresponding remaining processing data volume is greater than the data size of the unallocated expired backup data.

[0067] In this way, it is ensured that the total amount of expired backup data finally allocated to each processing thread matches the processing limit information of each processing thread, and the processing limit information is adapted to the total amount of expired backup data. Therefore, to a certain extent, it can be ensured that the difference between the total amount of expired backup data obtained by the corresponding allocation of the same processing capacity is small.

[0068] Step 208: clearing the corresponding allocated expired backup data in parallel through each processing thread.

[0069] Exemplarily, step 208 includes: clearing the corresponding allocated expired backup data in parallel by each processing thread in the allocated order or random order.

[0070] Optionally, the method also includes: sending the cleaning progress information corresponding to the corresponding allocated expired backup data to the MQ (Message Queue) database through each processing thread, wherein the cleaning progress information is used to characterize the cleaning progress of the expired backup data of each processing thread, which can be a progress percentage, or the amount of expired backup data that has been cleaned (or the amount of expired backup data that has not been cleaned) and the total amount of expired backup data that has been allocated; querying the MQ database to obtain the cleaning progress information corresponding to each processing thread.

[0071] Optionally, the method further comprises: sending the processing status information corresponding to each processing thread to the basic database, wherein the processing status information comprises at least one of the total amount of processable data, current processing occupancy information and processing status, the processing status comprises a data cleaning on state or a data cleaning off state, and the basic database comprises but is not limited to SQL Server (relational database system), (Depend on The basic database is queried to obtain the processing status information corresponding to each processing thread.

[0072] Optionally, the method further includes: sending visualization information obtained through the query to a user interface, wherein the visualization information includes at least one of processing status information and cleaning progress information, so that the user interface can display the received visualization information.

[0073] In this way, the processing status and cleaning progress of the processing thread can be visualized, so that the user can monitor the backup data cleaning process of the processing thread in real time.

[0074] As an example, refer to Figure 7 The thread pool composed of various processing threads (processing threads 1, 2, ..., N in the figure) communicates with the MQ database and the basic database respectively, the MQ database and the basic database communicate with the server respectively, and the server communicates with the user interface.

[0075] In the above-mentioned expired backup data cleaning method, by obtaining the backup expiration identification condition for the backup data, and according to the backup expiration identification condition, multiple expired backup data in the backup data are identified; according to the data distribution information determined by the data size of the multiple expired backup data, the allocation mode corresponding to the multiple expired backup data is identified, wherein the data distribution information is used to characterize the quantity distribution status of different data size orders of magnitude in the multiple expired backup data; according to the allocation mode, processing threads are respectively allocated to the multiple expired backup data, wherein the total amount of thread processing data of each processing thread is evenly distributed; the expired backup data corresponding to each processing thread is cleaned up in parallel respectively by each processing thread, the data distribution information is used as the identification basis of the allocation mode, and then the threads are allocated according to the identified allocation mode, so that the expired backup data allocated to each processing thread matches the data distribution status of the expired backup data, and the reasonable and even distribution of the total amount of thread processing data of each processing thread is achieved, thereby improving the cleaning efficiency of the expired backup data.

[0076] In an exemplary embodiment, Figure 8 As shown, a method for accurately obtaining the backup expiration identification condition for backup data is provided, and obtaining the backup expiration identification condition for backup data includes steps 302 to 306. Among them:

[0077] Step 302, obtaining the total amount of project data and the update frequency of the project data corresponding to the backup data.

[0078] The total amount of project data may be the total amount of data generated by the project, or the total amount of data backed up by the project.

[0079] Exemplarily, step 302 includes: obtaining the project name of the project corresponding to the backup data, querying the project log according to the project name, and obtaining the total amount of project data and the project data update frequency of the project corresponding to the backup data, wherein the project log includes the total amount of project data and the project data update frequency corresponding to each project name.

[0080] Step 304 : evaluating the user's attention to the backup data according to the total amount of project data and the update frequency of the project data, and obtaining attention information corresponding to the backup data.

[0081] As an embodiment, step 304 includes: constructing project attention features based on the total amount of project data and the frequency of project data updates; mapping the project attention features to attention information corresponding to the backup data through a pre-trained attention evaluation model, wherein the attention evaluation model is trained by multiple training samples, and the training samples are composed of training project attention features corresponding to the training data and the corresponding real attention.

[0082] It is understandable that when artificial intelligence model technology is used to assess the degree of user concern for backup data, since the model prediction accuracy of artificial intelligence model technology depends on the richness of samples, when artificial intelligence model technology is used to assess the degree of user concern for backup data, the model training time consumed is too long, resulting in a long preparation phase of the expired backup data cleaning process.

[0083] As another embodiment, step 304 includes: the attention information includes the degree of attention; based on the total amount of project data, the user's attention level for the backup data is evaluated to obtain a first degree value corresponding to the backup data, and, based on the project data update frequency, the user's attention level for the backup data is evaluated to obtain a second degree value corresponding to the backup data; the first degree value and the second degree value corresponding to the backup data are merged to obtain the attention level value corresponding to the backup data.

[0084] Among them, the larger the total amount of project data, the smaller the first degree value; the higher the update frequency of project data, the larger the second degree value.

[0085] It is understandable that when the total amount of project data is larger, the project size of the backup data is larger, then the user is affected by the larger project size and cannot pay much attention to the expired backup data, so the first degree value is set to be lower. When the project data update frequency is high, the project update speed of the backup data is fast, then the user will pay attention to whether the expired backup data is cleared in time to ensure that the newly updated project data can realize data backup, so the second degree value is set to be higher.

[0086] As an embodiment, the first degree value and the second degree value corresponding to the backup data are merged to obtain the attention degree value corresponding to the backup data, including: determining the average value between the first degree value and the second degree value as the attention degree value corresponding to the backup data.

[0087] In this way, the two degree values ​​determined by the two dimensions can jointly participate in determining the decision basis of the attention degree value, thereby improving the accuracy of determining the attention degree value.

[0088] As another embodiment, the first degree value and the second degree value corresponding to the backup data are merged to obtain the attention degree value corresponding to the backup data, including: weighting the first degree value corresponding to the backup data according to a preset first weight to obtain a first weighted degree value, and weighting the second degree value corresponding to the backup data according to a preset second weight to obtain a second weighted degree value; fusing the first weighted degree value with the second weighted degree value to obtain the attention degree value corresponding to the backup data, wherein the weighted processing can be a weighted summation processing or a weighted product processing; the fusion processing of the weighted degree values ​​can be a summation processing or a product processing.

[0089] Among them, the first weight and the second weight can be set by the user as needed, and can also be determined by the values ​​of the total amount of project data and the project data update frequency themselves, respectively. Specifically, the closer the total amount of project data is to the total amount limit value (the limit value can be the maximum value or the minimum value), the greater the first weight, and the closer the project data update frequency is to the frequency limit value, the greater the second weight.

[0090] In this way, the proportion of the two dimensions involved in determining the attention level value can be matched according to user expectations, or according to the values ​​of the dimensions themselves, thereby improving the accuracy of determining the attention level value.

[0091] Step 306: Generate a backup expiration identification condition for the backup data according to the attention information corresponding to the backup data.

[0092] Exemplarily, step 306 includes: obtaining preset expiration identification conditions for backup data; generating adjustment information corresponding to the preset expiration identification conditions based on the attention information of the backup data, wherein the adjustment information includes identification frequency adjustment information and identification tolerable deviation adjustment information; adjusting the preset expiration identification conditions based on the adjustment information corresponding to the preset expiration identification conditions to obtain the backup expiration identification conditions for the backup data.

[0093] Among them, the preset expiration identification condition can be an expiration identification condition set by the user as needed, or it can be obtained according to the project information configuration of the project corresponding to the backup data. The preset expiration identification condition includes at least one of the preset first condition and the preset second condition. Specifically, the specific implementation steps obtained according to the project information configuration of the project corresponding to the backup data can refer to the above-mentioned specific implementation content of obtaining the backup expiration identification condition for the backup data.

[0094] Specifically, the higher the degree of attention represented by the attention information, the shorter the time it takes to identify expired backup data based on the backup expiration identification condition.

[0095] Further, when the adjustment information includes recognition frequency adjustment information, the higher the level of attention represented by the attention information, the higher the target recognition frequency corresponding to the recognition frequency adjustment information, and the target recognition frequency is the expiration recognition detection frequency for backup data; when the adjustment information includes recognition tolerable deviation adjustment information, the higher the level of attention represented by the attention information, the smaller the recognition tolerable deviation range corresponding to the recognition tolerable deviation adjustment information, and when the preset expiration recognition condition includes a preset first recognition condition, the recognition tolerable deviation range includes a number tolerable deviation range; when the preset expiration recognition condition includes a preset second recognition condition, the recognition tolerable deviation range includes a time tolerable deviation range, and at this time, it is easier to identify as expired backup data.

[0096] As an embodiment, the preset expiration identification condition is adjusted according to the adjustment information corresponding to the preset expiration identification condition to obtain the backup expiration identification condition for the backup data, including: when the preset expiration identification condition includes a preset first identification condition, the preset first identification condition is adjusted according to the tolerable deviation range of the number corresponding to the preset first identification condition to obtain the first identification condition for the backup data; when the preset expiration identification condition includes a preset second identification condition, the preset second identification condition is adjusted according to the tolerable deviation range of the time corresponding to the preset second identification condition to obtain the second identification condition for the backup data.

[0097] In this way, the higher the degree of attention represented by the attention information, the more sensitive the user is to the backup data, that is, the more the user cares about the speed of clearing expired backup data, and the user will want the expired backup data to be identified as early as possible, so the expired backup data will be cleared faster.

[0098] In this embodiment, the total amount of project data and the project data update frequency of the project corresponding to the backup data are obtained; based on the total amount of project data and the project data update frequency, the user's attention to the backup data is evaluated to obtain the attention information corresponding to the backup data; based on the attention information corresponding to the backup data, a backup expiration identification condition for the backup data is generated, and the total amount of project data and the project data update frequency are used as the basis for evaluating the attention information, thereby ensuring that the evaluated attention information can accurately represent the user's attention to the backup data. Using the attention information as the basis for generating the backup expiration identification condition can realize the configurability of the process of identifying the backup data as expired backup data, thereby improving the accuracy of data backup identification.

[0099] In an exemplary embodiment, Fig. 9 As shown, in order to provide a method for processing threads to accurately clean up expired backup data, each processing thread cleans up the expired backup data allocated to it in parallel, including steps 402 to 404. Among them:

[0100] Step 402 : for each processing thread, generating a processing priority of the expired backup data assigned to the processing thread according to the attention information corresponding to the expired backup data assigned to the processing thread.

[0101] The higher the degree of attention to the backup data represented by the attention information, the higher the processing priority.

[0102] Step 404 , according to the processing priority of the expired backup data correspondingly allocated to each processing thread, each processing thread cleans up the expired backup data allocated to each processing thread in parallel.

[0103] Exemplarily, step 404 includes: for each processing thread, sorting the processing order of the expired backup data assigned to the processing thread according to the processing priority of the expired backup data assigned to the processing thread, to obtain the processing sorting result corresponding to the processing thread, wherein the higher the processing priority of the expired backup data, the higher the expired backup data is ranked in the processing sorting result; according to the processing sorting result corresponding to each processing thread, each processing thread performs parallel cleaning on the expired backup data assigned to each processing thread.

[0104] In this embodiment, for each processing thread, a processing priority for the expired backup data assigned to the processing thread is generated according to the attention information corresponding to the assigned expired backup data; according to the processing priority of the expired backup data assigned to each processing thread, each processing thread performs parallel cleaning on the expired backup data assigned to each processing thread, thereby ensuring that the expired backup data with the higher degree of user attention is cleaned up sooner, thereby ensuring that the cleaning of the expired backup data is in accordance with the user's expectations, thereby improving the cleaning accuracy of the expired backup data.

[0105] As a detailed embodiment, the total amount of project data and the update frequency of the project data of the project corresponding to the backup data are obtained; based on the total amount of project data, the user's attention to the backup data is evaluated to obtain a first degree value corresponding to the backup data, and, based on the update frequency of the project data, the user's attention to the backup data is evaluated to obtain a second degree value corresponding to the backup data; the first degree value and the second degree value corresponding to the backup data are merged to obtain a degree value corresponding to the backup data; a preset expiration identification condition for the backup data is obtained; based on the attention information of the backup data, adjustment information corresponding to the preset expiration identification condition is generated, wherein the adjustment information includes identification frequency adjustment information and identification tolerable deviation adjustment information; based on the adjustment information corresponding to the preset expiration identification condition, the preset expiration identification condition is adjusted to obtain a backup expiration identification condition for the backup data.

[0106] Furthermore, the processing limit information of each processing thread is determined according to the total amount of expired backup data determined by the data size of multiple expired backup data, the total number of processing threads and the total amount of processable data of each processing thread; data selection step: selecting unallocated expired backup data from the expired backup data; allocating processing threads to the unallocated expired backup data according to the data size of the unallocated expired backup data, the current processing occupancy information of each processing thread and the processing limit information of each processing thread, wherein the current processing occupancy information is used to characterize the processing occupancy of the processing thread under the currently allocated expired backup data; returning to the data selection step until there is no unallocated expired backup data in the expired backup data; for each processing thread, generating the processing priority of the expired backup data allocated to the processing thread according to the attention information corresponding to the allocated expired backup data; according to the processing priority of the expired backup data allocated to each processing thread, each processing thread performs parallel cleaning of the allocated expired backup data respectively.

[0107] In this way, by obtaining the backup expiration identification conditions for the backup data, and according to the backup expiration identification conditions, identifying multiple expired backup data in the backup data; assigning processing threads to the multiple expired backup data according to the data size of the multiple expired backup data, the total number of processing threads and the total amount of data that can be processed by each processing thread; and clearing the corresponding expired backup data in parallel through each processing thread, taking the data size of the multiple expired backup data, the total number of processing threads and the total amount of data that can be processed by each processing thread as the basis for thread allocation, so that the expired backup data assigned to each processing thread matches the corresponding processing capacity of each processing thread, and realizes the reasonable allocation of the total amount of thread processing data of each processing thread, thereby improving the cleaning efficiency of expired backup data.

[0108] Furthermore, the total amount of project data and the frequency of project data updates are used as the basis for evaluating the attention information, ensuring that the attention information obtained through the evaluation can accurately represent the user's attention level to the backup data. The attention information is used as the basis for generating the backup expiration identification conditions, thereby realizing the configurability of the process of identifying the backup data as expired backup data, thereby improving the accuracy of data backup identification; and, it can ensure that the expired backup data with the higher degree of user attention is cleared sooner, thereby ensuring that the clearing of the expired backup data is in accordance with the user's expectations, thereby improving the accuracy of the clearing of the expired backup data.

[0109] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0110] Based on the same inventive concept, the embodiment of the present application also provides an expired backup data cleaning device for implementing the expired backup data cleaning method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the expired backup data cleaning device provided below can refer to the limitations of the expired backup data cleaning method above, and will not be repeated here.

[0111] In an exemplary embodiment, Fig.10 As shown, an expired backup data cleaning device 500 is provided, comprising: an acquisition module 502, an identification module 504, an allocation module 506 and a cleaning module 508, wherein:

[0112] The acquisition module 502 is used to acquire a backup expiration identification condition for the backup data, and identify a plurality of expired backup data in the backup data according to the backup expiration identification condition;

[0113] An identification module 504 is used to identify allocation modes corresponding to the plurality of expired backup data according to data distribution information determined by the data sizes of the plurality of expired backup data, wherein the data distribution information is used to characterize the quantity distribution status of different data size magnitudes in the plurality of expired backup data;

[0114] An allocation module 506 is used to allocate processing threads to the multiple expired backup data according to the allocation mode, wherein the total amount of thread processing data of each processing thread is evenly allocated;

[0115] The cleaning module 508 is used to clean up the expired backup data corresponding to each processing thread in parallel.

[0116] In one of the embodiments, the acquisition module 502 is also used to obtain the total amount of project data and the project data update frequency of the project corresponding to the backup data; based on the total amount of project data and the project data update frequency, the user's attention level to the backup data is evaluated to obtain attention information corresponding to the backup data; based on the attention information corresponding to the backup data, a backup expiration identification condition is generated for the backup data.

[0117] In one of the embodiments, the acquisition module 502 is also used to obtain preset expiration identification conditions for backup data; based on the attention information of the backup data, generate adjustment information corresponding to the preset expiration identification conditions, wherein the adjustment information includes identification frequency adjustment information and identification tolerable deviation adjustment information; based on the adjustment information corresponding to the preset expiration identification conditions, adjust the preset expiration identification conditions to obtain the backup expiration identification conditions for the backup data.

[0118] In one of the embodiments, the identification module 504 is also used to identify the first allocation method as the allocation method corresponding to the multiple expired backup data if the data distribution information represents that the differences in the total amount of data corresponding to different data size levels are all within a preset difference range among multiple expired backup data, wherein, under the first allocation method, the expired backup data allocated to each processing thread all belong to the same data size level; if the data distribution information represents that the differences in the total amount of data corresponding to different data size levels are not all within the preset difference range among multiple expired backup data, then identify the second allocation method as the allocation method corresponding to the multiple expired backup data, wherein, under the second allocation method, the expired backup data allocated to each processing thread belongs to at least one data size level.

[0119] In one of the embodiments, the allocation module 506 is also used to classify each processing thread according to the data size levels corresponding to the multiple expired backup data to obtain a thread classification result if the allocation method is the first allocation method; and allocate processing threads to the multiple expired backup data respectively according to the thread classification results of each processing thread.

[0120] In one of the embodiments, the allocation module 506 is also used to determine the processing limit information of each processing thread according to the total amount of expired backup data determined by the data size of multiple expired backup data, the total number of processing threads and the total amount of processable data of each processing thread if the allocation method is the second allocation method; data selection step: select unallocated expired backup data from the expired backup data; allocate processing threads to the unallocated expired backup data according to the data size of the unallocated expired backup data, the current processing occupancy information of each processing thread and the processing limit information of each processing thread, wherein the current processing occupancy information is used to characterize the processing occupancy of the processing thread under the currently allocated expired backup data; return to the data selection step until there is no unallocated expired backup data in the expired backup data.

[0121] In one of the embodiments, the cleaning module 504 is also used to generate, for each processing thread, a processing priority for the expired backup data assigned to the processing thread according to the attention information corresponding to the assigned expired backup data; and according to the processing priority for the expired backup data assigned to each processing thread, each processing thread cleans up the expired backup data assigned to each processing thread in parallel.

[0122] Each module in the above-mentioned expired backup data cleaning device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module above.

[0123] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Fig.11As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (NFC) or other technologies. When the computer program is executed by the processor, a method for cleaning expired backup data is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0124] Those skilled in the art will understand that Fig.11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0125] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0126] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0127] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0128] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0129] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0130] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for cleaning expired backup data, characterized in that: The method comprises: Acquiring a backup expiration identification condition for the backup data, and identifying a plurality of expired backup data in the backup data according to the backup expiration identification condition; Identify allocation modes corresponding to the multiple expired backup data according to data distribution information determined by the data sizes of the multiple expired backup data, wherein the data distribution information is used to characterize the quantity distribution status of different data size magnitudes in the multiple expired backup data; According to the allocation method, processing threads are respectively allocated to the plurality of expired backup data, wherein the total amount of thread processing data of each processing thread is evenly allocated; The processing threads respectively clean up the corresponding allocated expired backup data in parallel.

2. The method according to claim 1, characterized in that The obtaining of the backup expiration identification condition for the backup data includes: Obtain the total amount of project data and the update frequency of the project data corresponding to the backup data; According to the total amount of the project data and the update frequency of the project data, evaluating the user's attention to the backup data, and obtaining attention information corresponding to the backup data; A backup expiration identification condition for the backup data is generated according to the attention information corresponding to the backup data.

3. The method according to claim 2, characterized in that The generating, according to the attention information corresponding to the backup data, a backup expiration identification condition for the backup data comprises: Obtaining a preset expiration identification condition for the backup data; Generate adjustment information corresponding to the preset expiration identification condition according to the attention information of the backup data, wherein the adjustment information includes identification frequency adjustment information and identification tolerable deviation adjustment information; According to the adjustment information corresponding to the preset expiration identification condition, the preset expiration identification condition is adjusted to obtain the backup expiration identification condition for the backup data.

4. The method according to claim 1, characterized in that: The step of identifying allocation modes corresponding to the plurality of expired backup data according to data distribution information determined by data sizes of the plurality of expired backup data includes: If the data distribution information indicates that the differences in the total amount of data between two different data size levels in the multiple expired backup data are all within a preset difference range, a first allocation method is identified as the allocation method corresponding to the multiple expired backup data, wherein under the first allocation method, the expired backup data allocated to each processing thread all belong to the same data size level; If the data distribution information represents that the differences in the total amount of data corresponding to different data size levels in the multiple expired backup data are not all within a preset difference range, then the second allocation method is identified as the allocation method corresponding to the multiple expired backup data, wherein, under the second allocation method, the expired backup data allocated to each processing thread belongs to at least one data size level.

5. The method according to claim 4, characterized in that The allocating processing threads to the plurality of expired backup data respectively according to the allocation mode comprises: If the allocation method is the first allocation method, classifying each processing thread according to the data size magnitudes respectively corresponding to the plurality of expired backup data to obtain a thread classification result; According to the thread classification results of each of the processing threads, processing threads are respectively allocated to the multiple expired backup data.

6. The method according to claim 4, characterized in that The allocating processing threads to the plurality of expired backup data respectively according to the allocation mode comprises: If the allocation method is the second allocation method, determining the processing limit information of each processing thread according to the total amount of expired backup data determined by the data size of the multiple expired backup data, the total number of processing threads and the total amount of processable data of each processing thread; Data selection step: selecting unallocated expired backup data from the expired backup data; Allocate a processing thread to the unallocated expired backup data according to the data size of the unallocated expired backup data, the current processing occupancy information of each processing thread and the processing limit information of each processing thread, wherein the current processing occupancy information is used to characterize the processing occupancy of the processing thread under the currently allocated expired backup data; Return to the data selection step until there is no unallocated expired backup data in the expired backup data.

7. The method according to claim 1, characterized in that The clearing up the expired backup data respectively allocated to each processing thread in parallel includes: For each processing thread, generating a processing priority of the expired backup data assigned to the processing thread according to the attention information corresponding to the assigned expired backup data; According to the processing priority of the expired backup data correspondingly allocated to each processing thread, each processing thread cleans up the expired backup data allocated to each processing thread in parallel.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images