Data recovery method, electronic equipment and computer readable storage medium

Abstract

The invention discloses a data recovery method, electronic equipment and a computer readable storage medium, and the method is applied to a cloud storage system, and comprises the steps: obtaining data traffic information of the cloud storage system in a current time period, predicting data traffic in a future time period based on the data traffic information, obtaining a data traffic prediction result, and outputting the data traffic prediction result. Determining data recovery time matched with the data traffic prediction result; determining a data recovery bandwidth based on the data traffic prediction result and load information corresponding to the cloud storage system; and in response to data loss of the cloud storage system, performing data recovery on the cloud storage system based on the data recovery time, the data recovery bandwidth and the load information corresponding to the cloud storage system. According to the scheme, the reasonability of data recovery can be improved.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and particularly relates to a data recovery method, an electronic device, and a computer-readable storage medium. Background Art

[0002] Cloud storage systems usually need to run continuously for a long time. Inevitably, file transmission may be lost due to link timeout, network anomaly, hard disk failure, node failure, program anomaly, etc. Therefore, it is necessary to recover damaged data files to ensure data reliability. However, when the system performs a large amount of data recovery, it will cause the system load to be too large, which may further lead to system crashes. Therefore, how to improve the rationality of data recovery has become an urgent problem to be solved. Summary of the Invention

[0003] The main technical problem to be solved by this application is to provide a data recovery method, an electronic device, and a computer-readable storage medium, which can improve the rationality of data recovery.

[0004] To solve the above technical problem, in the first aspect of this application, a data recovery method is provided, which is applied to a cloud storage system. The method includes: obtaining the data traffic information of the cloud storage system in the current time period, predicting the data traffic in the future time period based on the data traffic information to obtain a data traffic prediction result, and determining a data recovery time that matches the data traffic prediction result; determining a data recovery bandwidth based on the data traffic prediction result and the load information corresponding to the cloud storage system; in response to data loss occurring in the cloud storage system, performing data recovery on the cloud storage system based on the data recovery time, the data recovery bandwidth, and the load information corresponding to the cloud storage system.

[0005] To solve the above technical problem, in the second aspect of this application, an electronic device is provided, which includes a memory and a processor coupled to each other. Program instructions are stored in the memory, and the processor is configured to execute the program instructions to implement the method described in the first aspect above.

[0006] To solve the above technical problem, in the third aspect of this application, a computer-readable storage medium is provided, which stores program instructions that can be run by a processor. The program instructions are used to implement the method described in the first aspect above.

[0007] In the above solution, the data traffic information generated during the current time period of the cloud storage system is obtained, the data traffic in the future time period is predicted based on this data traffic information to obtain the corresponding data traffic prediction result, and the data recovery time matching the data traffic prediction result is determined. Based on the obtained data traffic prediction result and the load information corresponding to the cloud storage system, the data recovery bandwidth is determined. When data loss occurs in the cloud storage system, data recovery of the cloud storage system is performed based on the determined data recovery time, data recovery bandwidth, and the load information corresponding to the cloud storage system, which can effectively avoid data recovery during the traffic peak, solve the problem of excessive server load and system crash caused by data recovery, and thus improve the rationality of data recovery. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts. Among them:

[0009] Figure 1 is a flowchart of an embodiment of the data recovery method of the present application;

[0010] Figure 2 is Figure 1 a flowchart of an embodiment corresponding to part of the content in step S101 in;

[0011] Figure 3 is Figure 1 a flowchart of an embodiment corresponding to step S102 in;

[0012] Figure 4 is Figure 1 a flowchart of an embodiment corresponding to step S103 in;

[0013] Figure 5 is a structural diagram of an embodiment of the electronic device of the present application;

[0014] Figure 6 is a structural diagram of an embodiment of the computer-readable storage medium of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0015] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all embodiments, and adaptive combinations can be made between different embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0016] The terms "system" and "network" are often used interchangeably herein. The term "and / or" in this article only describes the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article generally represents an "or" relationship between the preceding and following associated objects. In addition, "plurality" in this article means two or more.

[0017] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of an implementation manner of the data recovery method of the present application.

[0018] Among them, this data recovery method is applied to a cloud storage system and includes:

[0019] S101: Obtain the data traffic information of the cloud storage system within the current time period, predict the data traffic within the future time period based on the data traffic information to obtain a data traffic prediction result, and determine a data recovery time that matches the data traffic prediction result.

[0020] Specifically, obtain the data traffic information generated during the operation of the cloud storage system within the current time period, predict the data traffic within the future time period based on this data traffic information to obtain a corresponding data traffic prediction result, and determine a data recovery time that matches this data traffic prediction result.

[0021] In an application mode, use the trained data traffic prediction model to collect the data traffic information generated during the operation of the cloud storage system within the current time period, and use this data traffic prediction model to predict the data traffic within the future time period based on the collected data traffic information, so as to obtain a corresponding data traffic prediction result, and determine a data recovery time that matches this data traffic prediction result.

[0022] In another application mode, obtain the data traffic information during the current time period actively collected by the cloud storage system, and input the data traffic information into the trained data traffic prediction model. Use the data traffic prediction model to predict the data traffic in the future time period based on the collected data traffic information, so as to obtain the corresponding data traffic prediction result, and determine the data recovery time matching the data traffic prediction result.

[0023] It should be noted that the data traffic prediction results include that the data traffic is at the peak value, the data traffic is at the valley value, etc.

[0024] In an application scenario, the long sequence time series prediction model is an Informer neural network model.

[0025] In other application scenarios, long sequence time series prediction models such as Long Short-Term Memory (LSTM) and Transformer models can also be used for data traffic prediction. This application does not make specific restrictions here.

[0026] S102: Determine the data recovery bandwidth based on the data traffic prediction result and the load information corresponding to the cloud storage system.

[0027] Specifically, determine the data recovery bandwidth based on the obtained data traffic prediction result and the load information corresponding to the cloud storage system.

[0028] In an application mode, by establishing a simulation model corresponding to the cloud storage system, simulate the data recovery process under different data traffic and load conditions, so as to obtain the corresponding data recovery bandwidth required under different conditions.

[0029] In another application mode, according to historical experience and business requirements, preset a set of rules to determine the data recovery bandwidth. For example, set multiple fixed bandwidth values according to the size of the data traffic and the load situation, and form a query table. When the data traffic prediction result and the load information corresponding to the cloud storage system meet the preset rules, use the query table to determine the data recovery bandwidth.

[0030] S103: In response to data loss in the cloud storage system, perform data recovery on the cloud storage system based on the data recovery time, data recovery bandwidth, and load information corresponding to the cloud storage system.

[0031] Specifically, when data loss occurs in the cloud storage system, perform data recovery on the cloud storage system based on the determined data recovery time, data recovery bandwidth, and load information corresponding to the cloud storage system.

[0032] In one application mode, when data loss occurs in the cloud storage system, a data recovery threshold is determined based on the data recovery bandwidth and the load information corresponding to the cloud storage system, and the cloud storage system is data-recovered according to the data recovery threshold within the data recovery time.

[0033] In another application mode, when data loss occurs in the cloud storage system, a data recovery task queue is generated. The data recovery task queue contains a corresponding number of data recovery tasks, and each data recovery task has a corresponding priority. A data recovery threshold is determined based on the data recovery bandwidth and the load information corresponding to the cloud storage system, and the data recovery tasks are processed according to the data recovery threshold and the corresponding priority within the data recovery time.

[0034] In some application scenarios, when the user needs to recover some specific damaged data, the priority of these damaged data can be manually increased and added to the data recovery task queue. When the cloud storage system is in the data recovery time, these data recovery tasks are preferentially processed.

[0035] In the above solution, the data traffic information generated during the operation of the cloud storage system in the current time period is obtained, the data traffic in the future time period is predicted based on this data traffic information to obtain the corresponding data traffic prediction result, and the data recovery time matching the data traffic prediction result is determined. Based on the obtained data traffic prediction result and the load information corresponding to the cloud storage system, the data recovery bandwidth is determined. When data loss occurs in the cloud storage system, the cloud storage system is data-recovered based on the determined data recovery time, data recovery bandwidth and the load information corresponding to the cloud storage system, which can effectively avoid data recovery during the traffic peak, solve the problem of excessive server load and system crash caused by data recovery, and thus improve the rationality of data recovery.

[0036] In one implementation mode, the data traffic prediction result is obtained by using a long sequence time series prediction model, and the long sequence time series prediction model is trained by using a training sample set composed of data traffic information in the historical time period.

[0037] Specifically, the data traffic information generated during the operation of the cloud storage system in the historical time period is used as the training sample set to train the long sequence time series prediction model, and the trained long sequence time series prediction model is used to predict the data traffic in the future time period to obtain the data traffic prediction result.

[0038] In one implementation scenario, the informer neural network model is used to predict the data traffic in the future time period to obtain the data traffic prediction result. Compared with other models, the informer model has a fast inference speed, high accuracy, and better universality.

[0039] In an application scenario, the training sample set is updated regularly, and the data recovery method further includes: regularly training the long sequence time series prediction model with the updated training sample set to obtain the trained long sequence time series prediction model.

[0040] Specifically, during the operation of the cloud storage system, the data traffic generated will change with the changes in the collection location points, date changes, etc. Therefore, it is necessary to regularly train the long sequence time series prediction model and update the model parameters to ensure the accuracy of the data traffic prediction results.

[0041] In a specific application scenario, during the operation of the cloud storage system, traffic data is continuously collected, and the latest traffic data is automatically added to the training sample set as a data set. Then, the informer neural network model is trained with the updated training sample set to update the model parameters, and the informer neural network model trained with the old data is replaced with the informer neural network model trained with the latest traffic data, so that the predicted traffic data prediction results are more in line with the current operation state of the cloud storage system.

[0042] Please refer to Figure 2 , Figure 2 which Figure 1 is a schematic flowchart of an implementation manner corresponding to part of the content in step S101 in

[0043] S201: Predict the data traffic in the future time period based on the data traffic information to obtain the data traffic prediction result.

[0044] Specifically, predict the data traffic generated during the operation of the cloud storage system in the future time period based on the data traffic information to obtain the data traffic prediction result.

[0045] S202: Based on the data traffic prediction result, divide the future time period into an idle time period and a busy time period, and determine the idle time period as the data recovery time.

[0046] Specifically, based on the data traffic prediction result, the future time period can be divided into an idle time period and a busy time period. When the data traffic prediction result is that the data traffic is at the peak value, it is determined as the busy time period. When the data traffic prediction result is that the data traffic is at the valley value, it is determined as the idle time period, and the idle time period is determined as the data recovery time. Thus, data recovery can be performed during the idle time period and data recovery can be stopped during the busy time period, avoiding excessive load on the cloud storage system when performing data recovery in a high data traffic peak scenario.

[0047] Please refer to Figure 3 , Figure 3 which Figure 1 is a schematic flowchart of an embodiment corresponding to step S102 in

[0048] S301: Based on the data traffic prediction result, obtain the data recovery weight value.

[0049] Specifically, based on the data traffic prediction result, obtain the corresponding data recovery weight value.

[0050] It can be understood that when the data traffic prediction result indicates that the data traffic is at a trough value, it means that the upcoming time period is an idle time period, and the corresponding data recovery weight value will increase accordingly.

[0051] S302: Based on the load information corresponding to the cloud storage system, determine the remaining data bandwidth of the cloud storage system.

[0052] Specifically, based on the load information corresponding to the cloud storage system, determine the remaining data bandwidth corresponding to the cloud storage system.

[0053] In an implementation scenario, the load information corresponding to the cloud storage system includes node data distribution, storage system scale, and the number of service access devices, etc. Among them, the storage system scale corresponds to the total data bandwidth of the entire cloud storage system. The used data bandwidth can be determined through the node data distribution and the number of service access devices, and thus the remaining data bandwidth can be dynamically calculated based on the total data bandwidth and the used data bandwidth.

[0054] S303: Based on the data recovery weight value and the remaining data bandwidth, determine the data recovery bandwidth.

[0055] Specifically, through the data recovery weight value and the remaining data bandwidth, obtain the data recovery bandwidth, so as to more reasonably allocate network resources, avoid excessive system load, and improve the overall performance.

[0056] Please refer to Figure 4 , Figure 4 which Figure 1 is a schematic flowchart of an embodiment corresponding to step S103 in

[0057] S401: In response to data loss in the cloud storage system, generate a corresponding recovery task queue; where the recovery task queue includes a corresponding number of recovery tasks.

[0058] Specifically, when data loss occurs in the cloud storage system, generate a recovery task queue, and this recovery task queue includes multiple recovery tasks.

[0059] S402: Process the recovery tasks based on the data recovery time, data recovery bandwidth, and the load information corresponding to the cloud storage system.

[0060] Specifically, process the recovery tasks in the recovery task queue based on the data recovery time, data recovery bandwidth, and the load information corresponding to the cloud storage system. By combining the data recovery bandwidth and the load information corresponding to the cloud storage system and processing the recovery tasks within the data recovery time, the stability and reliability of the cloud storage system during the recovery process can be ensured.

[0061] In an implementation scenario, the load information corresponding to the cloud storage system includes the storage space capacity. Step S402 specifically includes: in response to the current time being within the data recovery time, retain the recovery tasks corresponding to the number of data recovery bandwidths that meet the preset bandwidth condition in the recovery task queue, and send the remaining number of recovery tasks to the recovery failure queue; retain the recovery tasks corresponding to the number that meets the storage space capacity in the recovery task queue, and send the remaining number of recovery tasks to the recovery failure queue; process the recovery tasks in the recovery task queue.

[0062] Specifically, after the recovery thread starts, it first determines whether the current time is within the data recovery time. If so, retain the recovery tasks corresponding to the number of data recovery bandwidths that meet the preset bandwidth condition in the recovery task queue, and send the remaining number of recovery tasks to the recovery failure queue. If not, wait until the data recovery time is entered. Then, retain the recovery tasks corresponding to the number that meets the storage space capacity in the recovery task queue, and send the remaining number of recovery tasks to the recovery failure queue. After that, process the recovery tasks finally retained in the recovery task queue. By restricting the number of recovery tasks to be recovered, the reasonable allocation of system resources can be further ensured, avoiding performance degradation or system crashes caused by resource overload, and thus further improving the rationality of data recovery.

[0063] In an implementation scenario, after the step of processing the recovery tasks in the recovery task queue, it further includes: assigning priorities to the recovery tasks in the recovery failure queue; in response to the current time being within the data recovery time, send the recovery tasks in the recovery failure queue to the recovery task queue based on the priorities.

[0064] Specifically, corresponding priorities are assigned to the recovery tasks in the failed queue. If the current time is within the data recovery time, the recovery tasks in the failed queue are sent to the recovery task queue based on the assigned priorities, so that the previous recovery tasks can be preferentially processed during the next data recovery time, ensuring that critical data or services are restored as soon as possible, reducing the time of service interruption, improving the recovery efficiency, and by re-adding the failed recovery tasks to the recovery queue and processing them based on the priorities, the cloud storage system can more effectively handle abnormal situations during the recovery process, improving the overall stability and reliability of the system.

[0065] Optionally, the cloud storage system also provides interfaces for manually setting the data recovery time and data recovery bandwidth. In specific cases, the user may urgently need to recover some damaged data. Therefore, the data recovery time and data recovery bandwidth can be manually set through this interface. However, to avoid system crashes caused by excessive settings by the user, the maximum value of the manually set data recovery bandwidth is still limited according to the system load information and data traffic prediction results.

[0066] Please refer to Figure 5 , Figure 5 FIG. is a schematic structural diagram of an embodiment of an electronic device according to the present application. The electronic device 50 includes a memory 500 and a processor 502 that are coupled to each other. Among them, the memory 500 stores program data (not shown in the figure), and the processor 502 calls the program data to implement the method in any of the above embodiments. For the description of related content, please refer to the detailed description of the above method embodiments and will not be repeated here. Specifically, the electronic device 50 includes, but is not limited to: desktop computers, laptop computers, tablet computers, servers, etc., and is not limited here. In addition, the processor 502 can also be referred to as a CPU (Center Processing Unit, central processing unit). The processor 502 may be an integrated circuit chip with signal processing capabilities. The processor 502 may also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. Additionally, the processor 502 may be implemented jointly by integrated circuit chips.

[0067] Please refer to Figure 6 , Figure 6It is a schematic structural diagram of an embodiment of the computer-readable storage medium of the present application. The computer-readable storage medium 60 stores program data 600. When the program data 600 is executed by a processor, it implements the method in any of the above embodiments. For the description of related content, please refer to the detailed description of the above method embodiments, and details will not be repeated here.

[0068] It should be noted that the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0069] In addition, in each embodiment of the present application, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0070] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods in various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0071] The above is only the embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structural or equivalent process transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present application.

Claims

1. A data recovery method, characterized in that, Applied to a cloud storage system, the method includes: Obtain the data traffic information of the cloud storage system within the current time period, predict the data traffic within the future time period based on the data traffic information, obtain a data traffic prediction result, and determine a data recovery time matching the data traffic prediction result; Determine a data recovery bandwidth based on the data traffic prediction result and the load information corresponding to the cloud storage system; In response to data loss occurring in the cloud storage system, perform data recovery on the cloud storage system based on the data recovery time, the data recovery bandwidth, and the load information corresponding to the cloud storage system.

2. The method according to claim 1, characterized in that, The data traffic prediction result is obtained by using a long-sequence time series prediction model, and the long-sequence time series prediction model is trained by using a training sample set composed of data traffic information within a historical time period.

3. The method according to claim 2, wherein The training sample set is updated regularly, and the method further includes: Regularly train the long-sequence time series prediction model by using the updated training sample set to obtain the trained long-sequence time series prediction model.

4. The method according to claim 1, wherein The predicting the data traffic within the future time period based on the data traffic information, obtaining a data traffic prediction result, and determining a data recovery time matching the data traffic prediction result includes: Predict the data traffic within the future time period based on the data traffic information to obtain the data traffic prediction result; Based on the data traffic prediction result, divide the future time period into an idle time period and a busy time period, and determine the idle time period as the data recovery time.

5. The method according to claim 1, characterized in that, The determining a data recovery bandwidth based on the data traffic prediction result and the load information corresponding to the cloud storage system includes: Obtain a data recovery weight value based on the data traffic prediction result; Determine the remaining data bandwidth of the cloud storage system based on the load information corresponding to the cloud storage system; Determine the data recovery bandwidth based on the data recovery weight value and the remaining data bandwidth.

6. The method according to claim 1, characterized in that The performing data recovery on the cloud storage system in response to data loss occurring in the cloud storage system based on the data recovery time, the data recovery bandwidth, and the load information corresponding to the cloud storage system includes: In response to data loss occurring in the cloud storage system, generate a corresponding recovery task queue; wherein, the recovery task queue includes a corresponding number of recovery tasks; Process the recovery tasks based on the data recovery time, the data recovery bandwidth, and the load information corresponding to the cloud storage system.

7. The method according to claim 6, wherein The load information corresponding to the cloud storage system includes the storage space capacity; The processing the recovery tasks based on the data recovery time, the data recovery bandwidth, and the load information corresponding to the cloud storage system includes: In response to the current time being within the data recovery time, retain the recovery tasks corresponding to the data recovery bandwidth meeting the preset bandwidth condition in the recovery task queue, and send the remaining number of recovery tasks to the recovery failure queue; Retain the recovery tasks in the recovery task queue that meet the corresponding number of the storage space capacity in the recovery task queue, and send the remaining number of recovery tasks to the recovery failure queue; Process the recovery tasks in the recovery task queue.

8. The method according to claim 7, wherein After processing the recovery tasks in the recovery task queue, it further includes: Assign priorities to the recovery tasks in the recovery failure queue; In response to the current time being within the data recovery time, send the recovery tasks in the recovery failure queue to the recovery task queue based on the priorities.

9. An electronic device, characterized in that, It includes a memory and a processor that are mutually coupled. Program instructions are stored in the memory, and the processor is used to execute the program instructions to implement the method according to any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, Program instructions that can be run by a processor are stored, and the program instructions are used to implement the method according to any one of claims 1-8.

Images