Data recovery method and device, electronic equipment and computer readable storage medium

Abstract

The invention discloses a data recovery method and device, electronic equipment, a computer readable storage medium and a computer program product, and the method comprises the steps: determining an initial verification position in cache equipment according to a data recovery request, and reading a preset number of cache objects from the initial verification position; obtaining metadata information of each cache object and metadata information of the corresponding main storage object; verifying each cache object according to each piece of metadata information; and for the cache objects which do not pass the verification, deleting the cache objects and storing the main storage objects corresponding to the cache objects to the cache device so as to realize data recovery of the cache device. And when all the cache objects pass the verification, responding to the data recovery request to determine the next initial verification position, and returning to the step of reading the preset number of cache objects from the initial verification position until all the cache objects in the cache device are verified. According to the scheme, data recovery in the cache equipment can be realized after the equipment is restarted due to failure, and the system performance is effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of storage, in particular to a data recovery method, and relates to a data recovery device, an electronic device and a computer readable storage medium. BACKGROUND

[0002] In order to effectively improve the data reading performance, it has become the consensus in the industry to use high-speed SSD (Solid State Drives) or NVME (Nonvolatile Memory Express) SSD as a cache in a distributed storage system with HDD (Hard Disk Drive) as a main storage disk. By saving hot data in the SSD, the cache data in the SSD can be hit when reading the hot data without reading from the main storage HDD with lower performance, so as to reduce the response time and improve the throughput.

[0003] The distributed storage main storage often realizes the redundancy of data through replication or erasure mode, which can allow part of the nodes to fail (such as node power failure or network fluctuation, etc.), and still guarantee the integrity of the data. As a cache layer for improving performance, due to the capacity limitation of the SSD, only single-copy cache can be used to cache as much hot data as possible. In this case, if the node fails to restart, the main storage still has redundancy and can provide read and write services during the node failure period, but after the cache layer service restarts, it cannot be determined whether the object data cached during the failure period is modified, so it cannot be directly used and needs to be cleared. As a result of losing the cache of hot data, the system performance may be greatly reduced, which will affect the front-end business.

[0004] Therefore, how to realize the data recovery in the cache device after the device failure restarts to effectively improve the system performance and guarantee the normal operation of the front-end business is a problem to be solved by those skilled in the art. SUMMARY

[0005] The purpose of the present application is to provide a data recovery method which can realize data recovery in the cache device after the device failure restarts to effectively improve the system performance and guarantee the normal operation of the front-end business. Another purpose of the present application is to provide a data recovery device, an electronic device, a computer readable storage medium and a computer program product, all of which have the above-mentioned beneficial effects.

[0006] In a first aspect, the present application provides a data recovery method, comprising:

[0007] determining a starting check position in the cache device according to a data recovery request, and reading a preset number of cache objects from the starting check position;

[0008] obtain metadata information of each of the cache objects, and obtain metadata information of main memory objects corresponding to each of the cache objects in the main memory device;

[0009] verify each of the cache objects according to the metadata information of each of the cache objects and the metadata information of each of the main memory objects;

[0010] for a cache object that fails the verification, delete the cache object, and save a main memory object corresponding to the cache object to the cache device to implement data recovery of the cache device;

[0011] when all of the cache objects pass the verification, respond to the data recovery request to determine a starting verification position in the cache device, and return a step of reading a preset number of cache objects starting from the starting verification position until all of the cache objects in the cache device are verified.

[0012] Optionally, determining the starting verification position in the cache device according to the data recovery request comprises:

[0013] determining a position of a cache cursor in the cache device according to the data recovery request, and taking the position of the cache cursor as the starting verification position;

[0014] Correspondingly, after reading the preset number of cache objects starting from the starting verification position, the method further comprises:

[0015] determining an end cache object in the preset number of cache objects, and updating the cache cursor to a next cache object of the end cache object.

[0016] Optionally, the data recovery method further comprises:

[0017] when receiving a service read request, determining a read data object corresponding to the service read request;

[0018] if the read data object hits the cache device and an object name of the read data object is less than the cache cursor, executing the service request by using the cache device;

[0019] if the read data object hits the cache device and the object name of the read data object is not less than the cache cursor, executing the service request by using the main memory device;

[0020] if the read data object does not hit the cache device, executing the service request by using the main memory device.

[0021] Optionally, the data recovery method further comprises:

[0022] If the read data object hits the cache device and the object name of the read data object is not less than the cache cursor, a target cache object corresponding to the read data object is determined in the cache device, and a target data recovery request about the target cache object is responded to;

[0023] A data recovery operation is performed on the target cache object according to the target data recovery request, and the service request is executed by using the recovered target cache object.

[0024] Optionally, the data recovery method further comprises:

[0025] When receiving a service write request, a write data object corresponding to the service write request and metadata information of the write data object are determined;

[0026] The write data object is saved to the main memory device and the cache device;

[0027] The metadata information of the write data object is saved to the main memory database and the cache database.

[0028] Optionally, the data recovery method further comprises:

[0029] An average processing time length of service requests is counted, the service requests including the service read request and the service write request;

[0030] When the average processing time length does not exceed a first preset time length, a response interval time length of the data recovery request is increased by a first preset step length;

[0031] When the average processing time length exceeds the first preset time length, the response interval time length of the data recovery request is decreased by a second preset step length.

[0032] Optionally, the data recovery method further comprises:

[0033] A request processing time length of the data recovery request is obtained;

[0034] When the request processing time length does not exceed a second preset time length, a preset number of values is increased by a third preset step length;

[0035] When the first processing time length exceeds the second preset time length, the preset number of values is decreased by a fourth preset step length.

[0036] In a second aspect, the application further discloses a data recovery device, comprising:

[0037] A reading module is configured to determine a starting check position in a cache device according to a data recovery request, and read a preset number of cache objects from the starting check position;

[0038] an acquisition module, configured to acquire metadata information of each of the cache objects and metadata information of main memory objects corresponding to the cache objects in the main memory device;

[0039] a verification module, configured to verify each of the cache objects according to the metadata information of each of the cache objects and the metadata information of the main memory objects;

[0040] a recovery module, configured to, for a cache object that fails the verification, delete the cache object and save a main memory object corresponding to the cache object to the cache device, so as to realize data recovery of the cache device;

[0041] a cycle module, configured to, when all the cache objects pass the verification, determine a starting verification position in the cache device in response to the data recovery request, and return a step of reading a preset number of cache objects starting from the starting verification position until all the cache objects in the cache device are verified.

[0042] In a third aspect, the present application further discloses an electronic device, comprising:

[0043] a memory, configured to store a computer program;

[0044] a processor, configured to implement the steps of any of the data recovery methods described above when the computer program is executed.

[0045] In a fourth aspect, the present application further discloses a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of any of the data recovery methods described above.

[0046] In a fifth aspect, the present application further discloses a computer program product, comprising computer programs / instructions, and the computer programs / instructions are executed by a processor to implement the steps of any of the data recovery methods described above.

[0047] The application provides a data recovery method, comprising the following steps: determining a starting check position in a cache device according to a data recovery request, and reading a preset number of cache objects from the starting check position; obtaining metadata information of each cache object, and obtaining metadata information of a main memory object corresponding to each cache object in a main memory device; checking each cache object according to the metadata information of each cache object and the metadata information of each main memory object; for a cache object that fails to pass the check, deleting the cache object and saving a main memory object corresponding to the cache object to the cache device, so as to realize data recovery of the cache device; when all the cache objects pass the check, responding to the data recovery request to determine a next starting check position in the cache device, and returning to the step of reading a preset number of cache objects from the starting check position until all the cache objects in the cache device are checked.

[0048] According to the technical scheme provided in the application, the metadata information of each data object saved to the main memory device can be saved at the same time, and the metadata information of each data object saved to the cache device can also be saved at the same time. Therefore, when the cache device needs to be recovered due to a fault restart, the effectiveness of each cache object in the cache device can be checked through the metadata information, the cache object that passes the check can continue to be used for front-end business processing, and the cache object that fails to pass the check can be recovered by using the corresponding main memory object in the main memory device and continue to be used for front-end business processing. Therefore, the technical scheme can effectively identify and recover all invalid cache objects in the cache device, that is, the data recovery of the cache device can be realized after the device is restarted due to a fault, the system performance is effectively improved, and the normal operation of the front-end business is ensured.

[0049] In an embodiment of the application, the execution speed of the data recovery operation is controlled by controlling the amount of data read each time to check the data (i.e., the preset number), if the processing time of one data recovery request is relatively long, the value of the preset number can be reduced, and if the processing time of one data request is relatively short, the value of the preset number can be increased. Therefore, the technical scheme can control the data recovery speed according to the actual situation of the system, can effectively reduce the system business jitter, and further improves the system stability.

[0050] The data recovery device, the electronic device, the computer readable storage medium and the computer program product provided in the application also have the technical effects described above, and details are not described herein again. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the prior art and the technical solutions in the embodiments of the present application, the drawings used in the description of the prior art and the embodiments of the present application will be briefly introduced. Of course, the drawings related to the embodiments of the present application described below are only a part of the embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of the provided drawings, and the obtained other drawings also belong to the protection scope of the present application.

[0052] Figure 1 A flowchart of a data recovery method provided by the present application is shown in the figure.

[0053] Figure 2 A structural diagram of a data recovery system provided by the present application is shown in the figure.

[0054] Figure 3 A flowchart of a data verification method provided by the present application is shown in the figure.

[0055] Figure 4 A structural diagram of a data recovery device provided by the present application is shown in the figure.

[0056] Figure 5 A structural diagram of an electronic device provided by the present application is shown in the figure. DETAILED DESCRIPTION

[0057] The core of the present application is to provide a data recovery method, which can realize data recovery in a cache device after device failure restart, so as to effectively improve system performance and ensure normal operation of front-end business. Another core of the present application is to provide a data recovery device, an electronic device, a computer readable storage medium and a computer program product, all of which have the above beneficial effects.

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be introduced below with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0059] The present application provides a data recovery method.

[0060] Please refer to Figure 1 , Figure 1 A flowchart of a data recovery method provided by the present application is shown in the figure, which can include the following S101-S105.

[0061] S101: Determine a starting check position in the cache device according to the data recovery request, and read a preset number of cache objects from the starting check position.

[0062] The step is to achieve the reading of the cache objects to be checked. Specifically, when receiving the data recovery request, the starting check position in the cache device can be determined first, and then a preset number of cache objects in the cache device are read from the starting check position to identify the effectiveness of the preset number of cache objects (i.e., subsequent data checking) and data recovery. The preset reading number is the number of cache objects to be identified and recovered by the data recovery request, and the specific value does not affect the implementation of the technical solution, which can be set and adjusted according to the actual situation, and the present application does not limit it. It can be understood that if the data recovery request is the first data recovery request received after the cache device fails to restart, the above starting check position is the storage position of the first cache object in the cache device.

[0063] In an embodiment of the present application, determining the starting check position in the cache device according to the data recovery request can include: determining the position of the cache cursor in the cache device according to the data recovery request, and taking the position of the cache cursor as the starting check position.

[0064] Correspondingly, after reading the preset number of cache objects from the starting check position, it can also include: determining the last cache object in the preset number of cache objects, and updating the cache cursor to the next cache object of the last cache object.

[0065] The data recovery method provided in the embodiments of the present application can use the cache cursor to record the position of the cache object that has completed data checking. It can be understood that the position of the cache cursor is the starting check position, and the cache objects saved before the position have completed data checking and data recovery, and the cache objects saved in and after the position have not completed data checking and data recovery. Further, after reading a preset number of cache objects each time, the position of the cache cursor can be updated to provide a reference position for the next data recovery request, and so on, until all cache objects in the cache device are processed.

[0066] S102: Obtain the metadata information of each cache object, and obtain the metadata information of the main memory object corresponding to each cache object in the main memory device.

[0067] The step is to achieve the acquisition of the metadata information of the cache objects and their corresponding main memory objects, so as to achieve the validity check of the cache objects, that is, to determine whether the read cache objects are all valid data. Specifically, for each data object saved to the main memory device, the metadata information thereof can be saved simultaneously, and for each data object saved to the cache device, the metadata information thereof can also be saved simultaneously. For example, the main memory database and the cache database can be used to store the corresponding metadata information, so that the metadata information can be acquired in the two databases.

[0068] Based on this, in an embodiment of the present application, the data recovery method can further include: when receiving a service write request, determining the write data object corresponding to the service write request and the metadata information of the write data object; saving the write data object to the main memory device and the cache device; and saving the metadata information of the write data object to the main memory database and the cache database.

[0069] In a possible implementation, the metadata information can include but is not limited to the data object name, the data object storage range (such as data offset address, data length, data version number, etc.), and data loss hotness information.

[0070] S103: Checking each cache object according to the metadata information of each cache object and the metadata information of each main memory object.

[0071] The step is to achieve the validity identification of each cache object based on the metadata information. It can be understood that when the metadata information of the cache object is consistent with the metadata information of the corresponding main memory object, it can be determined that the cache object is valid data; when the metadata information of the cache object is inconsistent with the metadata information of the corresponding main memory object, it can be determined that the cache object is invalid data.

[0072] S104: For the cache object that fails the check, deleting the cache object and saving the main memory object corresponding to the cache object to the cache device to achieve data recovery of the cache device.

[0073] The step is to achieve the validity recovery of the invalid cache object. As described above, the validity check of each cache object can be achieved according to the metadata information of the data object. Therefore, for the cache object that fails the check, since the corresponding main memory object in the main memory device is necessarily valid, the valid main memory object in the main memory device can be saved to the corresponding position in the cache device after the invalid cache object is deleted, thereby achieving the validity recovery of the invalid cache object in the cache device. Of course, when the validity check of the cache object is passed, the validity check of the next cache object can be continued until the preset number of cache objects read are all checked.

[0074] S105: When all the cache objects pass the verification, the data recovery request is responded to determine the next starting verification position in the cache device, and the step of reading a preset number of cache objects from the starting verification position is returned until all the cache objects in the cache device are verified.

[0075] This step aims to realize the effective identification and data recovery of all the cache objects in the cache device through a loop operation. Specifically, if all the cache objects read by this data recovery request are completely verified and recovered, it can be determined that this data recovery request is executed. However, since the preset number is not the number of cache objects in the cache device, the verification and recovery of the remaining cache objects need to be continued. Therefore, a new data recovery request can be automatically responded to determine a new starting verification position, and the step of reading a preset number of cache objects in S101 is returned. In this way, all the cache objects in the cache device are verified and recovered, and the complete cache device data recovery process is realized.

[0076] It should be noted that the response time interval between data recovery requests can be set according to actual conditions, and it is not necessary to continuously execute data recovery requests. The present application does not limit this.

[0077] In an embodiment of the present application, before determining the starting verification position in the cache device according to the data recovery request, the method can further include: receiving the data recovery request and performing a locking operation on the data recovery request to make the data recovery request and the business request mutually exclusive; the business request can include a business read request and a business write request.

[0078] Correspondingly, before responding to the data recovery request to determine the next starting verification position in the cache device after all the cache objects pass the verification, the method can further include: performing an unlocking operation on the locked data recovery request.

[0079] Therefore, through this implementation manner, the response time interval between any two data recovery requests can continue to execute the front-end business request, ensuring the orderly progress of the front-end business; at the same time, the locking operation also effectively avoids the request conflict problem between the business request and the data recovery request, further ensuring the stability of the system.

[0080] It can be seen that the data recovery method provided by the embodiment of the application can save the metadata information of each data object saved to the main storage device at the same time, and can also save the metadata information of each data object saved to the cache device at the same time. Therefore, when the cache device needs to perform data recovery due to a fault restart, the validity of each cache object in the cache device can be checked through the metadata information. The cache object that passes the check can continue to be used for front-end service processing, and the cache object that fails the check can use the corresponding main storage object in the main storage device to perform data recovery and continue to be used for front-end service processing. It can be seen that the technical solution realizes effective identification and data recovery of all invalid cache objects in the cache device, that is, data recovery in the cache device can be realized after device fault restart, effectively improving system performance and ensuring normal operation of front-end services.

[0081] On the basis of the above embodiments:

[0082] In an embodiment of the application, the data recovery method can further include:

[0083] When receiving a service read request, determining a read data object corresponding to the service read request;

[0084] If the read data object hits the cache device and the object name of the read data object is smaller than the cache cursor, the cache device is used to execute the service request;

[0085] If the read data object hits the cache device and the object name of the read data object is not smaller than the cache cursor, the main storage device is used to execute the service request;

[0086] If the read data object does not hit the cache device, the main storage device is used to execute the service request.

[0087] It can be understood that the data recovery process of the cache device is not completed instantaneously, and the service read request can also be executed synchronously in the implementation process. Therefore, during the data recovery process, if a service read request is received during the time period between two data recovery requests, it can be determined whether the read data object corresponding to the service read request hits the cache device. Obviously, if the read data object does not hit the cache device, the main storage object corresponding to the read data object can be directly read in the main storage device. If the read data object hits the cache device, it is necessary to determine whether the cache object that hits the cache device has been checked. Obviously, if the cache object that hits the read data object has been checked, the cache object can be directly read and returned because the cache object is necessarily valid data. If the cache object that hits the read data object has not been checked, the main storage object corresponding to the read data object needs to be read and returned in the main storage device because the cache object that has not been checked can be invalid data.

[0088] If the object name of the read data object is smaller than the cache cursor, it can be determined that the cache object hit by the read data object has completed data verification. If the object name of the read data object is not smaller than the cache cursor, it can be determined that the cache object hit by the read data object has not completed data verification. Here, comparing the object name of the read data object with the cache cursor is essentially comparing the size of the object name of the read data object with the size of the object name of the cache object pointed to by the cache cursor, and the object name size is used to represent the dictionary order size of the object name (provided that each cache object in the cache device has been pre-verified by dictionary order).

[0089] Further, the data recovery method can further include: if the read data object hits the cache device, and the object name of the read data object is not smaller than the cache cursor, determining a target cache object corresponding to the read data object in the cache device, and responding to a target data recovery request about the target cache object; performing a data recovery operation on the target cache object according to the target data recovery request, and executing the service request using the recovered target cache object. That is, for the case that the read data object hits the cache device and the cache object hit by the read data object has not completed verification, the corresponding main memory object in the main memory device can also be selected not to respond to the data read service, but the cache object hit by the read data object is first verified and recovered, and then the cache object after verification and recovery is used to respond to the data read service. Of course, both implementation methods can exist at the same time, and the user can select according to actual needs in actual use scenarios.

[0090] In an embodiment of the present application, the data recovery method can further include:

[0091] Statistically determining an average processing time length of service requests, the service requests including service read requests and service write requests;

[0092] When the average processing time length does not exceed a first preset time length, the response interval time length of the data recovery request is increased by a first preset step length;

[0093] When the average processing time length exceeds the first preset time length, the response interval time length of the data recovery request is decreased by a second preset step length.

[0094] It can be seen that the embodiment of the application can control the execution speed of the data recovery operation by controlling the response interval length between data recovery requests. If the average processing time length of multiple service requests is relatively long, it indicates that the current front-end service is relatively busy, and the value of the response interval length can be reduced. If the average processing time length of multiple service requests is relatively short, it indicates that the current front-end service is relatively idle, and the value of the response interval length can be increased. It can be seen that the technical solution can control the data recovery speed according to the actual situation of the system, can effectively reduce the system service jitter, and further improves the system stability.

[0095] In an embodiment of the application, the data recovery method can further include:

[0096] obtaining the request processing time length of the data recovery request;

[0097] when the request processing time length does not exceed the second preset time length, increasing the value of the preset number by a third preset step length;

[0098] when the first processing time length exceeds the second preset time length, decreasing the value of the preset number by a fourth preset step length.

[0099] It can be seen that the embodiment of the application can control the execution speed of the data recovery operation by controlling the data amount (i.e. the preset number) of each read cache object for data verification. If the processing time length of a data recovery request is relatively long, the value of the preset number can be reduced. If the processing time of a data request is relatively short, the value of the preset number can be increased. It can be seen that the technical solution can control the data recovery speed according to the actual situation of the system, can effectively reduce the system service jitter, and further improves the system stability.

[0100] It should be pointed out that the specific values of the above-mentioned thresholds (the first preset time length, the second preset time length, the first preset step length, the second preset step length, the third preset step length, and the fourth preset step length) do not affect the implementation of the technical solution, and can be set according to actual needs. The values can be equal and can change at any time, and the application does not limit this.

[0101] On the basis of the above-mentioned embodiments, the embodiment of the application provides another data recovery method. The implementation process of the data recovery method provided by the application can include the following steps.

[0102] First, please refer to Figure 2 , Figure 2 is a structural schematic diagram of a data recovery system provided by the application, which mainly includes the following functional modules.

[0103] 1. Context reconstruction module.

[0104] After the fault recovery, the metadata information ctx (context) of all cache objects stored in the cache device is traversed and reconstructed into the read cache module. The context of the cache object is used for the business to access a certain cache object, to query the data range, data state, and hotness of the cached data on the cache device, etc., which is necessary information for the read cache module to manage the hot cache object. These information is stored in the memory, and the metadata information ctx of the cache object is stored in the hash table objectmap of the read cache for searching.

[0105] The metadata information ctx of the cache object includes:

[0106] (1) Object name: used for recording and querying the cache object;

[0107] (2) Cache data range: the cache logical range of the cache object, recording the data offset off and data length len and data version number version of each data segment;

[0108] (3) Hotness: recording the hotness of the cache object, and for the read cache system using LRU (Least Recently Used, Least Recently Used) for cache replacement, the position of the cache object in the LRU is also recorded.

[0109] 2. Data verification module.

[0110] Responsible for data verification of the cache object whose context has been reconstructed after the cache device fault recovery, determining the validity of the data segment by comparing the version number and the check value of each data segment of the object in the cache device and the main storage device, and deleting the invalid data segment.

[0111] Also responsible for maintaining a cache cursor for recording the object name whose context has been reconstructed, recording the current verified cache object position, and the cache object whose object name is smaller than the cache cursor is the cache object whose context has been verified, otherwise it has not been verified. The size of the object name represents the dictionary sorting size of the object name.

[0112] 3. Speed control module.

[0113] Responsible for controlling the background execution speed of the above cache object context reconstruction and data verification process, dynamically adjusting the number of cache objects obtained each time and the processing time interval of the verification request (data recovery request) by the business processing delay and the time consumption of each verification request.

[0114] Further, based on the above data recovery system, the data recovery method provided by the embodiments of the present application can include the following steps.

[0115] 1. Data version number and check value updating process.

[0116] The original service write process updates the version number and the check value of each piece of data, so in the write cache process, the version number and the check value corresponding to the current write data need to be obtained and written into the cache device.

[0117] (1) The write request generates the version number and the check value of the current data object;

[0118] (2) The data object is written into the system main memory device;

[0119] (3) The main memory object metadata is updated, and the version number and the check value of the current data object are submitted to the main memory database;

[0120] (4) The data object is written into the system cache device;

[0121] (5) The cache object metadata is updated, and the version number and the check value of the current data object are submitted to the cache database.

[0122] 2. Fault switching process.

[0123] Triggered when the OSD (Object Storage Device) service of the cache device restarts and initializes the read cache module.

[0124] (1) Set the cache check object name cursor to the minimum object name MIN;

[0125] (2) Start the cache check task thread and send a cache data check request.

[0126] 3. Cache data check process.

[0127] When the cache data check task thread is started, it will initiate a cache data check request, which will take a lock and be mutually exclusive with the business request, and will scan the cache objects on the cache device for checking. Please refer to Figure 3 , Figure 3 The flowchart of the data check method provided in the present application is as follows:

[0128] (1) Take the object name of the checked cache object cursor as the starting point, obtain the cache object names in the cache device for a certain number (a preset number) of objects, and add them to the temporary cache object queue. The number of cache objects obtained is controlled by the speed module, and the function of obtaining objects will return the next cache object name NEXT of the last obtained cache object, and then the checked cache object cursor will be set to the object name;

[0129] (2) Get a cache object from the tail of the acquired cache object queue, read the metadata information of the cache object, including the saved data logical segment and its corresponding version number and check value, and reconstruct these metadata information into the cache object context;

[0130] (3) Get the metadata information of the main memory object, and sequentially traverse the data segment of the cache object, compare each data segment with the length, version number and check value of the same offset data obtained from the main memory, if they are the same, it means the data is valid, and it is preserved, if they are different, it means the data is invalid, and a delete OP (Operation Request) is constructed for this segment of data for subsequent deletion;

[0131] (4) Repeat steps (2) and (3) to complete the verification of all cache objects in the cache object queue, then empty the cache object queue, and determine whether the verified cache object cursor is the maximum MAX, if so, end the task, otherwise initiate the next cache data verification request to the request queue of the OSD.

[0132] 4. Speed control process.

[0133] After each cache data verification request processing is completed, the speed control module will collect relevant data to adjust the parameters of the cache verification process.

[0134] (1) Get the processing time of this cache verification request (the time when the request processing is completed minus the time when the request processing starts), and determine whether the processing time exceeds the threshold (second preset time length), if so, reduce the number of cache objects obtained each time for verification, otherwise increase, the number of cache objects increased or decreased each time can be set by a configuration item according to actual needs, to avoid blocking business due to long verification request processing time;

[0135] (2) Get the average delay of the current business request processing, if the delay exceeds the threshold (first preset time length), increase the interval time for processing cache verification request each time, otherwise decrease, the time interval increased or decreased each time can be set by a configuration item according to actual needs. In addition, when the OSD processes the cache data verification request, it will determine the time the request stays in the request queue, if it does not exceed the time interval setting threshold, it will not be processed and the request will be re-queued.

[0136] 5. Business processing flow.

[0137] When the original business request reads or updates the data in the cache device, the following process needs to be added:

[0138] (1) Determine whether the object name is less than the verified object cursor, if so, continue to process according to the original process, otherwise proceed to the next step;

[0139] (2) judge whether the object is in the cache object context objectmap, if yes, it means that the data has been checked, then continue to process according to the original process, otherwise go to the next step;

[0140] (3) find the object from the cache device, if not found, skip the read cache module processing, if found, trigger the cache data check of the object, and then read or update after processing according to the process of reconstructing the cache object context and the cache data check.

[0141] The business processing flow can make the cache objects that are not processed by the background cache data check when the business is accessed, and the cache data check can be performed in the business processing process. If the object to be accessed is the object that has been cached before the failure and has not been modified during the failure, it can be directly read from the cache device.

[0142] As can be seen, the data recovery method provided by the embodiment of the application can save the metadata information of each data object saved to the main storage device, and can also save the metadata information of each data object saved to the cache device. Therefore, when the cache device needs to be recovered due to failure restart, the effectiveness of each cache object in the cache device can be checked through the metadata information. The cache object that passes the check can be used for front-end business processing, and the cache object that fails the check can be recovered by using the corresponding main storage object in the main storage device and used for front-end business processing. Therefore, the technical solution can effectively identify and recover all invalid cache objects in the cache device, that is, the data recovery in the cache device can be realized after the device failure restart, which effectively improves the system performance and ensures the normal operation of the front-end business.

[0143] The embodiment of the application provides a data recovery device.

[0144] Please refer to Figure 4 , Figure 4 The structure diagram of the data recovery device provided by the application can include:

[0145] The reading module 1 is configured to determine the starting check position in the cache device according to the data recovery request, and read a preset number of cache objects from the starting check position;

[0146] The acquisition module 2 is configured to acquire the metadata information of each cache object, and acquire the metadata information of the main storage object corresponding to each cache object in the main storage device;

[0147] The check module 3 is configured to check each cache object according to the metadata information of each cache object and the metadata information of each main storage object;

[0148] The recovery module 4 is configured to, for the cache object that fails the verification, delete the cache object and save the main memory object corresponding to the cache object to the cache device to realize data recovery of the cache device;

[0149] The cycle module 5 is configured to, when all the cache objects pass the verification, determine a next starting verification position in the cache device in response to the data recovery request and return a step of reading a preset number of cache objects starting from the starting verification position until all the cache objects in the cache device are verified.

[0150] It can be seen that the data recovery apparatus provided by the embodiment of the present application can save the metadata information of each data object saved to the main memory device and the metadata information of each data object saved to the cache device. Thus, when the cache device needs to be recovered due to a fault restart, the effectiveness of each cache object in the cache device can be verified through the metadata information. The cache object that passes the verification can continue to be used for front-end service processing, and the cache object that fails the verification can be recovered by using the corresponding main memory object in the main memory device and continue to be used for front-end service processing. It can be seen that the technical solution realizes effective identification and data recovery of all invalid cache objects in the cache device, that is, data recovery in the cache device after a device fault restart, effectively improves the system performance and ensures normal operation of the front-end service.

[0151] In an embodiment of the present application, the reading module 1 can be specifically configured to determine the position of the cache cursor in the cache device according to the data recovery request and take the position of the cache cursor as the starting verification position.

[0152] Correspondingly, the data recovery apparatus can further include an updating module configured to, after reading the preset number of cache objects starting from the starting verification position, determine the last cache object in the preset number of cache objects and update the cache cursor to the next cache object of the last cache object.

[0153] In an embodiment of the present application, the data recovery apparatus can further include a read request processing module configured to, when a service read request is received, determine a read data object corresponding to the service read request; if the read data object hits the cache device and the object name of the read data object is smaller than the cache cursor, execute the service request by using the cache device; if the read data object hits the cache device and the object name of the read data object is not smaller than the cache cursor, execute the service request by using the main memory device; and if the read data object does not hit the cache device, execute the service request by using the main memory device.

[0154] In an embodiment of the present application, the read request processing module can be further configured to, if the read data object hits the cache device and the object name of the read data object is not less than the cache cursor, determine a target cache object corresponding to the read data object in the cache device, and respond to a target data recovery request about the target cache object; perform a data recovery operation on the target cache object according to the target data recovery request, and execute the service request by using the recovered target cache object.

[0155] In an embodiment of the present application, the data recovery apparatus can further include a write request processing module configured to, when receiving a service write request, determine a write data object corresponding to the service write request and metadata information of the write data object; save the write data object to the main memory device and the cache device; and save the metadata information of the write data object to the main memory database and the cache database.

[0156] In an embodiment of the present application, the data recovery apparatus can further include a first control module configured to count an average processing time length of service requests, the service requests including service read requests and service write requests; when the average processing time length does not exceed a first preset time length, increase a response interval time length of the data recovery request by a first preset step length; and when the average processing time length exceeds the first preset time length, decrease the response interval time length of the data recovery request by a second preset step length.

[0157] In an embodiment of the present application, the data recovery apparatus can further include a second control module configured to obtain a request processing time length of the data recovery request; when the request processing time length does not exceed a second preset time length, increase a preset number of values by a third preset step length; and when the first processing time length exceeds the second preset time length, decrease the preset number of values by a fourth preset step length.

[0158] The apparatus provided in the embodiments of the present application can refer to the above method embodiments, and the present application will not be described herein.

[0159] An electronic device is provided in the embodiments of the present application.

[0160] Please refer to Figure 5 , Figure 5 A structure schematic diagram of an electronic device provided in the present application can include:

[0161] A memory 11 is configured to store a computer program.

[0162] A processor 10 is configured to execute the computer program to implement the steps of any one of the above data recovery methods.

[0163] As Figure 5As shown in the structural schematic diagram of the electronic device, the electronic device can include a processor 10, a memory 11, a communication interface 12 and a communication bus 13. The processor 10, the memory 11 and the communication interface 12 can complete communication with each other through the communication bus 13.

[0164] In the embodiment of the present application, the processor 10 can be a central processing unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic device, etc. The processor 10 can call a program stored in the memory 11, and specifically, the processor 10 can perform the operations in the embodiment of the data recovery method.

[0165] The memory 11 is used to store one or more programs, and the program can include program code including computer operation instructions. In the embodiment of the present application, the memory 11 at least stores a program for implementing the following functions:

[0166] According to the data recovery request, a starting check position in the cache device is determined, and a preset number of cache objects starting from the starting check position are read;

[0167] Metadata information of each cache object is obtained, and metadata information of a main memory object corresponding to each cache object in the main memory device is obtained;

[0168] Each cache object is checked according to the metadata information of each cache object and the metadata information of each main memory object;

[0169] For the cache object that fails to pass the check, the cache object is deleted, and the main memory object corresponding to the cache object is saved to the cache device to realize data recovery of the cache device;

[0170] When all the cache objects pass the check, the data recovery request is responded to determine a next starting check position in the cache device, and the step of reading a preset number of cache objects starting from the starting check position is returned until all the cache objects in the cache device are checked.

[0171] In a possible implementation, the memory 11 can include a program storage area and a data storage area, wherein the program storage area can store an operating system and at least one application required by a function, etc.; the data storage area can store data created in the use process. In addition, the memory 11 can include a high-speed random access memory, and can also include a non-volatile memory, for example, at least one magnetic disk storage device or other volatile solid-state storage device.

[0172] The communication interface 12 can be an interface of a communication module, used to connect with other devices or systems.

[0173] Of course, it needs to be explained that Figure 5 The structure shown does not constitute a limitation on the electronic device in the embodiments of the present application, and in actual applications, the electronic device can include more or fewer components than Figure 5 shown, or combine certain components.

[0174] The embodiments of the present application provide a computer readable storage medium.

[0175] The computer readable storage medium provided by the embodiments of the present application stores a computer program, and the computer program is executed by a processor to implement the steps of any one of the above data recovery methods.

[0176] The computer readable storage medium can be any available medium that can be stored by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media sets, for example, it can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape, etc.), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk) and various media that can store computer program codes.

[0177] For the computer readable storage medium provided by the embodiments of the present application, please refer to the above method embodiments, which will not be repeated here.

[0178] The embodiments of the present application provide a computer program product.

[0179] The computer program product provided by the embodiments of the present application includes computer programs / instructions, which can implement the steps of any one of the above data recovery methods when executed by a processor.

[0180] Specifically, in the above embodiments, all or part of them can be realized by software, hardware, firmware or any combination thereof. When realized by software, all or part of them can be realized in the form of a computer program product.

[0181] The computer program product can include one or more computer programs / instructions, which when loaded and executed on a computer, can produce all or part of the processes or functions described in the embodiments of the present application. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. Computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line, etc.) or wireless (such as infrared, wireless, microwave, etc.) mode.

[0182] The computer program product provided in the embodiments of the present application is introduced in the above method embodiments, and the present application will not be repeated here.

[0183] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the related parts can be referred to the method part.

[0184] The skilled person can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or combination of both. In order to clearly show the interchangeability of hardware and software, the composition and steps of each example have been described in the above description. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0185] The steps of the method or algorithm described in combination with the embodiments disclosed herein can be directly implemented by hardware, software module executed by a processor, or combination of both. The software module can be placed in random access memory (RAM), internal memory, read only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM or any other form of storage medium known in the art.

[0186] The above describes the technical solutions provided by the present application in detail. The principles and implementation modes of the present application are described by applying specific examples. The above description of the embodiments is only to help understand the method and its core idea of the present application. It should be pointed out that, for the ordinary skilled person in the technical field, without departing from the principles of the present application, the present application can be improved and modified in several ways. These improvements and modifications also fall within the protection scope of the present application.

Claims

1. A data recovery method, characterized by, The method comprises the following steps: determining a starting check position in the cache device according to a data recovery request, and reading a preset number of cache objects from the starting check position; obtaining metadata information of each cache object, and obtaining metadata information of a main memory object corresponding to each cache object in the main memory device; checking each cache object according to the metadata information of each cache object and the metadata information of each main memory object; for the cache object that fails to pass the check, deleting the cache object, and saving the main memory object corresponding to the cache object to the cache device to realize data recovery of the cache device; when all the cache objects pass the check, responding to the data recovery request to determine a next starting check position in the cache device, and returning to the step of reading a preset number of cache objects from the starting check position until all the cache objects in the cache device are checked.

2. The data recovery method of claim 1, wherein, The method comprises the following steps: determining a starting check position in the cache device according to a data recovery request, and reading a preset number of cache objects from the starting check position; correspondingly, after reading the preset number of cache objects from the starting check position, the method further comprises the following steps: determining an end cache object in the preset number of cache objects, and updating the cache cursor to the next cache object of the end cache object.

3. The data recovery method of claim 2, wherein, The method further comprises the following steps: when a service read request is received, determining a read data object corresponding to the service read request; if the read data object hits the cache device and the object name of the read data object is smaller than the cache cursor, executing the service request by using the cache device; if the read data object hits the cache device and the object name of the read data object is not smaller than the cache cursor, executing the service request by using the main memory device; if the read data object does not hit the cache device, executing the service request by using the main memory device.

4. The data recovery method of claim 3, wherein, The method further comprises the following steps: if the read data object hits the cache device and the object name of the read data object is not smaller than the cache cursor, determining a target cache object corresponding to the read data object in the cache device, and responding to a target data recovery request about the target cache object; performing a data recovery operation on the target cache object according to the target data recovery request, and executing the service request by using the recovered target cache object.

5. The data recovery method of claim 3, wherein, The method further comprises the following steps: when a service write request is received, determining a write data object corresponding to the service write request and metadata information of the write data object; saving the write data object to the main memory device and the cache device; saving the metadata information of the write data object to the main memory database and the cache database.

6. The data recovery method of claim 5, wherein, The method further comprises the following steps: statistically obtaining an average processing time length of service requests, wherein the service requests include the service read request and the service write request; when the average processing time length does not exceed a first preset time length, increasing a response interval time length of the data recovery request by a first preset step length; When the average processing duration exceeds the first preset duration, the response interval duration of the data recovery request is decreased by a second preset step.

7. The data recovery method according to any one of claims 1 to 6, characterized by, Further comprising: acquiring a request processing duration of the data recovery request; When the request processing duration does not exceed a second preset duration, increasing the preset number of values by a third preset step; When the first processing duration exceeds the second preset duration, decreasing the preset number of values by a fourth preset step.

8. A data recovery apparatus, characterized by comprising: Comprising: a reading module, configured to determine a starting check position in a cache device according to a data recovery request, and read a preset number of cache objects from the starting check position; an acquiring module, configured to acquire metadata information of each of the cache objects, and acquire metadata information of main memory objects corresponding to each of the cache objects in a main memory device; a checking module, configured to check each of the cache objects according to the metadata information of each of the cache objects and the metadata information of each of the main memory objects; a recovery module, configured to, for a cache object that fails the check, delete the cache object, and save a main memory object corresponding to the cache object to the cache device, so as to realize data recovery of the cache device; a loop module, configured to, when all the cache objects pass the check, respond to the data recovery request to determine a next starting check position in the cache device, and return to the step of reading a preset number of cache objects from the starting check position, until all the cache objects in the cache device are checked.

9. An electronic device, comprising: Comprising: a memory, configured to store a computer program; a processor, configured to implement the steps of the data recovery method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer program is stored on the computer readable storage medium, and the computer program is executed by the processor to implement the steps of the data recovery method according to any one of claims 1 to 7.

Images