Data recovery method, apparatus, device, distributed storage system, and storage medium

Abstract

The present disclosure relates to the technical field of cloud computing, and provides a data recovery method, a distributed storage system, an electronic device, and a storage medium. The data recovery method comprises: when a second chunk server exists in a distributed storage system, in response to scheduling of a metadata server, adding, to a first queue, cloud disk data to be undergone data recovery in the second chunk server; and performing periodic data recovery on the cloud disk data in the first queue on the basis of a preset time period. A first chunk server is a chunk server that has not undergone a shutdown, and the second chunk server is a chunk server that has undergone a shutdown. In the embodiments of the present disclosure, cloud disk data can be recovered in a distributed manner, thereby relieving the pressure of concurrent recovery of cloud disk data, and accelerating the overall recovery efficiency.

Description

[0001]The present disclosure relates to the technical field of cloud computing, and particularly relates to a data recovery method, device, equipment, distributed storage system and storage medium. Background Technology A distributed storage system can provide high-performance storage. When a block server serving as a storage device in the distributed storage system is down, the distributed storage system performs data recovery on cloud disk data of the down block server. In the existing data recovery manner, after a live block server is scheduled to take over the down block server, a large amount of cloud disk data often needs to be recovered, which is time-consuming and slow, and the overall data recovery speed is slow, resulting in weak overall fault recovery capability of the distributed storage system. The present disclosure provides a data recovery method, device, equipment, distributed storage system and storage medium to solve the above technical problems. In a first aspect, the present disclosure provides a data recovery method applied to a first block server in a distributed storage system, which comprises: in the case that there is a second block server in the distributed storage system, adding cloud disk data to be recovered in the second block server to a first queue in response to scheduling of a metadata server; and periodically recovering the cloud disk data in the first queue according to a preset time period. The first block server is a block server that has not been down, and the second block server is a block server that has been down. In a second aspect, the present disclosure provides a data recovery method applied to a metadata server in a distributed storage system, which comprises: in the case that there is a second block server in the distributed storage system, scheduling a first block server to perform the data recovery method provided in the first aspect of the present disclosure. The first block server is a block server that has not been down, and the second block server is a block server that has been down. In a third aspect, the present disclosure provides a distributed storage system, which comprises: a plurality of metadata servers, a plurality of block servers and a distributed file system. Each block server is in communication connection with a corresponding metadata server, and each block server is also in communication connection with the distributed file system. In the case that there is a second block server in the plurality of block servers, the metadata server is configured to perform the data recovery method provided in the second aspect of the present disclosure, and the first block server is configured to perform the data recovery method provided in the first aspect of the present disclosure when not being down. The first block server is a block server that has not been down, and the second block server is a block server that has been down. In a fourth aspect, the present disclosure provides an electronic device that can serve as a block server or a metadata server in a distributed storage system, which comprises a memory, a processor and a computer program stored in the memory. The processor implements the method of any one of the embodiments of the present disclosure when executing the computer program.In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program. The computer program, when executed by a processor, implements any of the methods provided by the embodiments of the present disclosure. In a sixth aspect, a computer program product is provided, which includes a computer program. The computer program, when executed by a processor, implements any of the methods provided by the embodiments of the present disclosure. The technical solutions of the embodiments of the present disclosure have the following advantages: periodic data recovery of cloud disk data is performed by scheduling the first block server that is active, multiple and scattered data recovery can be achieved, compared with the scheme of actively recovering and concurrently recovering data, the scheme of the embodiments of the present disclosure is a passive and triggered recovery strategy, the corresponding data recovery is triggered by a preset time period, which can effectively alleviate the pressure of concurrently recovering a large amount of data, accelerate the overall recovery efficiency of a large amount of cloud disk data, and increase the expansion elasticity of the number of files, thereby improving the fault recovery capability of the distributed storage system and ensuring the data integrity under the conditions of subsequent read operations and fault recovery. The above description is merely a summary of the technical solutions of the present disclosure. In order to more clearly understand the technical means of the present disclosure, the embodiments of the present disclosure can be implemented in accordance with the contents of the description. In order to make the above and other purposes, features and advantages of the present disclosure more apparent and easy to understand, the specific embodiments of the present disclosure are described below. BRIEF DESCRIPTION OF DRAWINGS In the drawings, the same reference signs represent the same or similar components or elements throughout the drawings unless otherwise specified. The drawings are not necessarily drawn to scale. It should be understood that the drawings merely depict some embodiments according to the present disclosure and should not be considered as limiting the scope of the present disclosure. FIG. 1 shows a schematic diagram of a scenario of a data recovery scheme provided in an embodiment of the present disclosure; FIG. 2 shows a storage format of a logical segment of a physical file in an embodiment of the present disclosure; FIG. 3 shows a division of a logical storage unit in an embodiment of the present disclosure; FIG. 4 shows a schematic diagram of a data writing process of a logical segment in an embodiment of the present disclosure; FIG. 5 shows a schematic diagram of the principle of data recovery in the related art; FIG. 6 shows a format of a compressed file; FIG. 7 shows a flowchart of a data recovery method provided in an embodiment of the present disclosure; FIG. 8 shows a schematic diagram of one specific example of a data recovery method provided in an embodiment of the present disclosure; FIG. 9 shows a schematic diagram of the principle of file state conversion of the example shown in FIG. 8; FIG. 10 shows a structural block diagram of a data recovery apparatus provided in an embodiment of the present disclosure; FIG. 11 shows a structural block diagram of another data recovery apparatus provided in an embodiment of the present disclosure; and FIG. 12 shows a structural block diagram of an electronic device for implementing an embodiment of the present disclosure. DETAILED DESCRIPTION In the following, only some exemplary embodiments are simply described.As those skilled in the art can recognize, the described embodiments can be modified in various different ways without departing from the concept or scope of the present disclosure. Therefore, the drawings and the description are considered to be exemplary in nature, rather than limiting. In order to facilitate understanding of the technical solutions of the embodiments of the present disclosure, the related technologies of the embodiments of the present disclosure are described below. The following related technologies can be combined with the technical solutions of the embodiments of the present disclosure in any manner as optional schemes, and all of them belong to the protection scope of the embodiments of the present disclosure. The technical solutions provided by the embodiments of the present disclosure can be applied to the distributed storage system shown in FIG. 1, which can be implemented based on a log structured block device (LSBD). A big feature of the implementation of the log structured block device is that the writing of data is appended, and the reference to the block device writing data does not need to update the reference address of the logical block to the disk every time, but accumulates the read data block index in the memory to a certain extent, and then updates and merges the latest index of the part of the updated logical block in the memory with the full index that has been stored asynchronously, thereby realizing efficient reading of data. The block device can be a computer device using block storage technology, such as a block server. Referring to FIG. 1, the distributed storage system can include a metadata server (Block Master) cluster, a plurality of block servers (Block Server), and a distributed file system. Each block server is in communication connection with the metadata server, and each block server is also in communication connection with the distributed file system. The metadata server cluster can include a plurality of metadata servers, and three metadata servers, metadata server 1, metadata server 2, and metadata server 3, are shown in FIG. 1. A plurality of cloud disks (not shown in FIG. 1) can be deployed in the block server, and the plurality means more than two. In actual applications, there can be thousands or even millions of cloud disks. Three metadata servers (metadata server 1, metadata server 2, and metadata server 3) and three block servers (block server 1, block server 2, and block server 3) are shown in FIG. 1 as examples. Each block server is in communication connection with a corresponding metadata server, and each block server is also in communication connection with the distributed file system.Referring to FIG. 1, the distributed storage system can store logical segments (Device Segment), specifically, the metadata server can store the metadata of the logical segments, which includes configuration information such as the size of the logical segments, and the distributed file system can store the physical files of the logical segments, specifically, can be stored under the corresponding directory of the logical segment in the shared storage service, the logical segment #0 directory and the logical segment #1 directory shown in FIG. 1 can respectively store the physical files of the logical segment #0 and the logical segment #1. The application program of the client can access the physical files of the corresponding logical segment in the distributed file system through the block server to realize the IO (inputOutput, input / output) data access of the cloud disk. Each block server can obtain the metadata of one or more corresponding logical segments in the corresponding metadata server (the obtaining process is not directly labeled in FIG. 1), forming the physical files of the cloud disk logical segment, and then stored in the distributed file system, for example, the block server 1 in FIG. 1 can obtain the metadata of the corresponding logical segment in the metadata server 1, the block server 2 can obtain the metadata in the metadata server 2, and the block server 3 can obtain the metadata in the metadata server 3. FIG. 2 shows the storage format of the physical file of the logical segment, referring to FIG. 1 and FIG. 2, the physical file of the logical segment can include: index file, data file and log file (Txn File, not shown in FIG. 1), etc., all of which are distributed file types that can only be written in an append manner and cannot be overwritten. The data file is responsible for storing the data and logical block address (Logical Block Address, LBA) of the logical segment, and can store the data of the logical segment in the form of data blocks, FIG. 2 shows data block #0, data block #1 and data block #2 as an example. The index file is responsible for recording the correspondence between the logical address range (Logical Block Address Range, LBA Range) of the data file and the physical address range of the data file, which can be embodied by the logical block address, the data file offset (i.e. the offset address of the data file) and the data file ID (Identity Document, identity number). The log file is responsible for recording the transaction log of modifying the logical segment. Referring to FIG. 2, the application program of the client writes to the data file in an append manner, the offset of the file is always increasing, and cannot be overwritten.The index file records the merging of all write positions of the application at a certain time point, the file is loaded into memory when the logical segment is loaded, and the file can indicate the corresponding data file and the offset in the data file, i.e. the data file offset in Fig. 2, when the application reads the logical segment, to perform the actual data access process. The entire logical data length represented by the index file is consistent with the entire logical data length of the logical segment. In the distributed file system, multiple logical segments can constitute a logical segment group, and multiple logical segment groups can constitute a logical storage unit (Logical Unit Number, LUN). Fig. 3 shows a division of the logical storage unit, wherein segment #0, segment #1, segment #2 and segment #3 are four logical segments in the distributed file system, which can constitute a logical segment group, i.e. segment group #0, and other logical segments not shown can constitute another logical segment group, i.e. segment group #1. The metadata server cluster can store logical relationship data of the logical storage unit, such as the relationship between the logical storage unit, the logical segment group and the logical segment. Fig. 4 shows an example of writing data to the logical segment, Fig. 4 shows the correspondence between each logical block address in the logical address interval of the index table (the memory object of the index file) and each data block in the data file, wherein the first logical block address LBA#01 corresponds to the first data block (represented by a small square in Fig. 4) in data file 1, the second logical block address LBA#02 corresponds to the second to fourth data blocks in data file 1, the metadata includes the size information of the data block, and the size of a data block is 4KB, the third logical block address LBA#03 corresponds to the first data block in data file 2, and the fourth logical block address LBA#04 corresponds to the second to third data blocks in data file 2. The following takes the data writing of data file 2 in Fig. 4 as an example to explain the data writing process of the logical segment as follows: the data writing process of the logical segment includes writing a data file, writing a log file, and updating a memory index file.Specifically, in the process of writing a data file, for a write request initiated by an application, application data carried by the write request can be written into the data file 2 in the form of a data block; in the process of writing a log file, a transaction log of the written data block can be recorded in the log file 1; in the process of updating a memory index file, a correspondence between a logical address interval and a physical address interval of the newly written data block can be updated to an index table (a memory object of the index file), which can exist in the form of a compressed file mapping in the memory, so that in the process of updating the memory index table, the correspondence between the logical address interval and the physical address interval of the newly written data block can be updated to the compressed file mapping (Compressed Map), and the index file in the memory can provide conversion of the logical address interval of the data block to the physical address interval of the data block. The index file in the memory, such as the index file 1 and the index file 2 in FIG. 4, can be swapped in and out according to data reading requirements. Referring to the example in FIG. 5, in the related art, when a certain block server (hereinafter referred to as block server 1) has a downtime exception, the metadata server can re-schedule a new active block server (hereinafter referred to as block server 2) to load the compressed file under the logical segment directory corresponding to the block server 1 to perform cloud disk data recovery after detecting that the heartbeat of the block server 1 is lost. In order to solve the problem of storage cost, while balancing the occupation of the central processing unit (Central Processing Unit, CPU) and read amplification, the user data of the written data block can be efficiently compressed, and thus the data file can exist in the form of a compressed file, and the read and write operations on the data file can be realized by read and write operations on the compressed file. The compressed file can be divided into a compressed data zone (Data Zone) and a footer zone (Footer Zone), the compressed data zone can include a plurality of compressed units, three compressed units are shown as an example in FIG. 6, and the footer zone can include a plurality of compressed unit file offsets (i.e., offset addresses of compressed unit files), two compressed unit file offsets are shown as an example in FIG. 6. A certain data block size can be fixed as a compressed unit, for example, in the example in FIG. 6, a data block with a size of 4*4K (kilobyte) can be fixed as a compressed unit.The recovery of the compressed file tail area helps to quickly locate the data part of the data file specified by the compression unit, thereby accelerating the recovery of the data file. The data file is generally provided with a DataFile (data file) OT (Offset Table, offset table) file or a compression unit file OT file (hereinafter referred to as OT file). The OT file only stores the offset of the compression unit file. The recovery of the OT file can recover the offset of the compression unit file, thereby recovering the compressed file tail area, and recovering the compressed file and the data file. The technical solutions of the present disclosure and how the technical solutions of the present disclosure solve the foregoing technical problems will be described in detail below with specific embodiments. Several specific embodiments listed can be combined with each other, and the same or similar concepts or processes can not be described again in some embodiments. The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose authorization or refusal. The embodiments of the present disclosure provide a data recovery method, which can be applied to a metadata server in a distributed storage system. The method can include: in the case that there is a second block server in the distributed storage system, scheduling a first block server to make the first block server execute a corresponding data recovery method. Wherein, the first block server is a block server that has not occurred downtime, and the second block server is a block server that has occurred downtime. Based on the same technical concept, the embodiments of the present disclosure also provide a data recovery method, which can be applied to a first block server in a distributed storage system. Referring to FIG. 7, the method can include steps S701-S702: step S701: in the case that there is a second block server in the distributed storage system, in response to the scheduling of the metadata server, adding cloud disk data to be subjected to data recovery in the second block server to a first queue. The cloud disk data can be a data file of a cloud disk logical segment, and the first queue is a queue to be subjected to data recovery. Step S702: periodically recovering the cloud disk data in the first queue according to a preset time period.For example, in a certain time period, a part of the cloud disk data in the first queue can be recovered, in the next time period, another part of the cloud disk data in the first queue is recovered, and then in multiple time periods, each part of the cloud disk data is recovered in turn until the data recovery of all the cloud disk data to be recovered is completed. How much cloud disk data is recovered in each time period can be set according to actual needs. The above data recovery method provided by the present disclosure can realize multiple and dispersed data recovery by scheduling the active first block server to periodically recover the cloud disk data. Compared with the scheme of actively recovering and concurrently recovering data, the scheme of the present disclosure is a kind of passive trigger recovery strategy, which is triggered by the preset time period to trigger the corresponding data recovery, which can effectively alleviate the pressure of concurrently recovering a large amount of data, speed up the overall recovery efficiency of a large amount of cloud disk data, and increase the expansion elasticity of the number of files, thereby improving the fault recovery capability of the distributed storage system, and ensuring the data integrity under the conditions of subsequent read operation and fault recovery. In a possible implementation manner, the cloud disk data is stored in the form of a compressed file. Correspondingly, in the step S702, periodically recovering the cloud disk data in the first queue according to the preset time period can include: in each preset first time period, for the specified cloud disk data in the first queue, reconstructing the compressed file corresponding to the specified cloud disk data in the memory based on the metadata in the metadata server; and performing read-only setting on the reconstructed compressed file. The first time period and the data that needs to be recovered in the first time period can be determined in advance according to historical data of the pressure of concurrently recovering the cloud disk data, or can be set according to actual needs. The pressure of concurrently recovering the cloud disk data can be represented by the IOPS (Input / Output Operations Per Second) of the metadata server. When the IOPS value is large, a longer time period can be set, and when the IOPS value is small, a shorter time period can be set. For each first time period, the specified cloud disk data can be the cloud disk data that needs to be recovered in the current first time period. In the reconstruction process, the 0T file of the data file of the specified cloud disk data can be loaded from the metadata file of the metadata server to obtain the compressed unit file offset, and the tail region of the compressed file is reconstructed based on the compressed unit file offset, and then the entire compressed file is reconstructed.Reconstruction of the compressed file is a key operation for recovering cloud disk data, involves read and write operations on the metadata server, is a key operation causing concurrent recovery pressure, and based on the first time period, the reconstruction operation of the compressed file is changed from a centralized operation to a periodic decentralized operation, which can delay the key operation of the compressed file reconstruction to the peak and valley of the overall system pressure, effectively alleviate the pressure of concurrent data recovery, and avoid the burst operation flow of the metadata server operation. The read-only setting can make the recovered cloud disk data in a read-only state, prevent other fault ends from writing data to the part of the cloud disk data again, and thus ensure the integrity of the recovered cloud disk data, and facilitate subsequent reading of the complete file. In a possible implementation, the read-only setting of the reconstructed compressed file can include: adding the reconstructed compressed file to a second queue for read-only setting; and periodically setting the read-only of the compressed file in the second queue according to a preset second time period. The second time period can be synchronized with the first time period, or can be unsynchronized. The read-only setting of the reconstructed compressed file is another key operation for recovering cloud disk data, also involves read and write operations on the metadata server, and is also a key operation causing concurrent recovery pressure. Based on the second time period, the read-only setting of the compressed file is changed from a centralized operation to a periodic decentralized operation, which can delay the key operation of the read-only setting of the compressed file to the peak and valley of the overall system pressure, effectively alleviate the pressure of concurrent data recovery, and avoid the burst operation flow of the metadata server operation. In the case that the second time period is synchronized with the first time period, the reconstruction of the compressed file is completed, and the read-only setting of the reconstructed compressed file is also completed in the same time period. In this case, the reconstructed compressed file can also not be added to the second queue, for example, after the reconstruction of a compressed file is completed, the read-only setting of the reconstructed compressed file can be started immediately. In the case that the second time period is unsynchronized with the first time period, the reconstruction of the compressed file can be performed in each first time period, and the read-only setting is periodically performed based on the second time period, and the read-only setting of a part of the reconstructed compressed file is performed in each second time period. In this way, the reconstruction and read-only setting of the compressed file are further dispersed through the two different time periods, i.e., the first time period and the second time period, so that the read and write pressure of the metadata server can be further alleviated.In a possible implementation, the above reconstructing, in the memory, the compressed file corresponding to the specified cloud disk data based on the metadata in the metadata server can include: invoking a compressed file reconstruction task, and reconstructing, in the memory, the compressed file corresponding to the specified cloud disk data based on the metadata in the metadata server in the compressed file reconstruction task. The read-only setting of the reconstructed compressed file can include: invoking a file read-only management task, and performing the read-only setting on the reconstructed compressed file in the file read-only management task. The reconstruction and the read-only setting of the compressed file are implemented by independent tasks, the compressed file reconstruction task is responsible for performing a task of reconstructing the compressed file, for example, a task of periodically reconstructing the compressed file, the file read-only management task is responsible for performing a task of periodically performing the read-only setting on the compressed file, for example, a task of periodically performing the read-only setting on the compressed file, that is, the reconstruction and the read-only setting of the compressed file are decoupled, and can be decoupled from a current task (for example, a logical segment loading task of currently performing overall data recovery), the two tasks can independently perform the reconstruction and the read-only setting, so that the overall efficiency of the reconstruction and the read-only setting is improved, and the overall efficiency of the cloud disk data recovery is improved. In a possible implementation, the compressed file can include a compressed data area and a tail area, the compressed data area can include a compressed unit corresponding to the cloud disk data, and the tail area includes a compressed unit file offset corresponding to the compressed unit. Correspondingly, the reconstructing, in the memory, the compressed file corresponding to the specified cloud disk data based on the metadata in the metadata server can include: loading, from a metadata file in the metadata server, an OT file corresponding to the specified cloud disk data, reconstructing, in the memory, a tail area of a complete compressed file corresponding to the specified cloud disk data based on the OT file; and reconstructing the complete compressed file corresponding to the specified cloud disk data based on the tail area. The reconstruction of the tail area helps to quickly locate a data part of a data file specified by the compressed unit, and then quickly restore the complete compressed file. After the complete compressed file is reconstructed, the OT file can be deleted in the memory to release the compressed unit file offset, and the compressed unit file offset is read in the memory on demand in a subsequent manner, so as to avoid memory occupation. In a possible implementation, the data recovery method provided in the embodiments of the present disclosure can further include: the first block server performs data recovery on target cloud disk data in the first queue in response to a read request sent by the client for the target cloud disk data that has not been recovered.In the process of periodically recovering data, if a read request for target cloud disk data that has not been recovered is received, data recovery can be directly performed on the target cloud disk data without waiting for a subsequent time period, that is, without waiting for the completion of the overall reconstruction and read-only setting to achieve reading; the time of client reading is relatively random and dispersed, and recovery of the target cloud disk data is performed under the triggering of a read request, which can meet the current reading demand on the basis of guaranteeing dispersed and concurrent recovery pressure to speed up the overall recovery progress, and the recovery object is relatively small, and only the target cloud disk data to be read at present needs to be recovered based on the actual reading demand, without the need to recover other cloud disk data in the first queue that is not within the scope of the read request this time, which can greatly improve the speed of this reading. In a possible implementation manner, when data recovery of the target cloud disk data is performed in response to the read request for the target cloud disk data, the data recovery process can include: reconstructing the compressed file corresponding to the target cloud disk data based on the metadata in the metadata server, and performing read-only setting on the reconstructed compressed file. For example, an independent compressed file reconstruction task can be called, the compressed file corresponding to the target cloud disk data is reconstructed in the memory based on the metadata in the metadata server in the compressed file reconstruction task, and an independent file read-only management task is called to perform read-only setting on the reconstructed compressed file. That is, in one reading process, the reconstruction and read-only setting of the compressed file can be first completed, and reading is performed after the reconstruction and read-only setting are completed. In a possible implementation manner, when data recovery of the target cloud disk data is performed in response to the read request for the target cloud disk data, the data recovery process can include: reconstructing the compressed file corresponding to the target cloud disk data in the memory based on the metadata in the metadata server in response to the read request for the target cloud disk data sent by the client; adding the reconstructed compressed file to a second queue for read-only setting; and periodically performing read-only setting on the compressed file in the second queue according to a preset second time period. That is, in one reading process, only the reconstruction of the compressed file can be completed, and reading can be performed after the reconstruction, so that the data reading reaching the backend after passing through the business layer cache is relatively small, the reading delay can be reduced, the reading efficiency is improved, and the task of read-only setting that is prone to cause a long time delay can be delayed and periodically executed to further concentrate the pressure of read-only setting on the metadata server. The second time period can be synchronized with the first time period or can be unsynchronized with the first time period, and can be set according to actual requirements.The reconstruction task of the compressed file corresponding to the target cloud disk data can be performed by an independent compressed file reconstruction task, and the read-only setting task of the compressed file corresponding to the target cloud disk data can be performed by an independent file read-only management task. In a possible implementation, the data recovery method provided by the embodiment of the present disclosure can further include: the first block server transfers the file ownership of the cloud disk data in the second block server from the second block server to the current first block server. The file ownership transfer can be performed before the cloud disk data to be recovered in the second block server is added to the first queue, so as to ensure the data consistency in the subsequent recovery process, so that the data recovery method provided by the embodiment of the present disclosure accelerates the data recovery under the premise of meeting the data consistency, and further accelerates the fault recovery of the distributed storage system. The file ownership transfer can be implemented based on a file grabbing mechanism, for example, the file lock occupied by the second block server in outage can be forcibly released, and the file lock is occupied by the first block server. In a possible implementation, in the process of adding the cloud disk data to be recovered in the second block server to the first queue, it can be first judged whether the cloud disk data to be recovered in the second block server meets the preset addition condition, and if the cloud disk data to be recovered in the second block server meets the preset addition condition, the cloud disk data to be recovered can be added to the first queue for data recovery, otherwise, it does not need to be added to the first queue, and the cloud disk data to be recovered can be directly recovered. The above-mentioned addition condition can be set according to at least one of the number of cloud disk data to be recovered, historical data of the pressure of concurrent recovery of cloud disk data, and the like. For example, it can be set that when the number of cloud disk data to be recovered is greater than a certain preset number threshold, it is added to the first queue. If the number of cloud disk data to be recovered is less than or equal to the number threshold, even if the concurrent recovery, it will not cause too much pressure to the metadata server, and will not cause too much delay to the overall recovery of the cloud disk data, at this time, it can not be added to the first queue for periodic data recovery. For another example, it can be set that when the historical IOPS of the metadata server is greater than a certain IOPS threshold, it is added to the first queue. If the historical IOPS of the metadata server is less than or equal to the IOPS threshold, it can be considered that the historical concurrent read and write amount of the metadata server is small, and periodic data recovery is not needed. FIG. 8 shows a specific example of the data recovery method provided by the embodiment of the present disclosure.Referring to FIG. 5 and FIG. 8, in the case where there is a second chunk server (for example, chunk server 1 in FIG. 5) in the distributed storage system, the first chunk server (for example, server 2 in FIG. 5) can rebuild the data file in the data file mapping in the memory through the logical segment loading task (step 1 in FIG. 8) and set the data file to read-only (step 2 in FIG. 8). In the process of rebuilding the data file, the transfer of ownership can be first performed by substep 1.1 by forcibly releasing the file lock occupied by the downed chunk server 1, and then substep 1.2 is used to determine whether the cloud disk data file to be recovered needs to be rebuilt with delay, that is, whether it needs to be periodically rebuilt, and if so, the file state of the non-empty data file (that is, a non-empty file) to be recovered is set to a to-be-rebuilt state, that is, the file-to-be-rebuilt state in FIG. 8, and the data file to be recovered is inserted into a to-be-rebuilt queue (that is, a first queue) through substep 2.1, and the data files in the to-be-rebuilt queue are all in the to-be-rebuilt state; after being inserted into the to-be-rebuilt queue, an independent compressed file rebuilding task can be created, and step 4 is triggered in the compressed file rebuilding task to periodically rebuild the data file, in addition, step 4 can also be triggered to rebuild the target data file when a read request is received; the process of periodically rebuilding the data file or rebuilding the data file can be implemented through substeps 4.1 to 4.8, first, the data file is opened in the memory, then the OT file (that is, the file of the compressed unit file offset table in FIG. 8) is loaded from the metadata server to obtain the compressed unit file offset therein and then written to the tail area of the compressed file, and based on the tail area, the complete compressed file can be rebuilt, the file state of the rebuilt compressed file is set to an open state, that is, available for client reading, and at the same time, the file state of the complete compressed file after rebuilding is marked as to-be-read-only and inserted into a second queue, that is, available for subsequent read-only setting of the data file, and after the rebuilding is completed, substep 4.8 can also be used to delete the OT file in the memory to release the compressed unit file offset and reduce the occupation of the memory; the read-only setting of the data file can be performed in the file read-only management task, and in the task, the data file in the second queue in the “to-be-read-only setting” state is set to a read-only state. As can be seen from the data recovery process in FIG. 8, the recovery of the compressed file (including rebuilding and read-only setting) can be decoupled from the logical segment loading (that is, step 6), and the recovery of the compressed file (for example, compressed file A) can be delayed until the logical segment loading is completed, thereby improving the overall logical segment reading and loading efficiency; the rebuilding and read-only setting of the compressed file are triggered based on client reading, avoiding the burst operation flow of the metadata server operation.In the case that the non-reading request triggers the recovery instead of the periodic trigger recovery, after the reconstruction, the file state of the complete compressed file after the reconstruction can be marked as the state to be set as the read-only and inserted into the second queue, the file in the second queue is set as the read-only by the file read-only management task, or the complete compressed file after the reconstruction is directly set as the read-only without being inserted into the second queue. As can be seen from the data recovery process in FIG. 8, the data recovery method provided by the embodiment of the present disclosure is a multi-stage and multi-state data recovery method. Referring to FIG. 9, the data recovery process includes two stages of reconstructing the compressed file and setting the read-only, after the stage of reconstructing the compressed file, the non-empty data file can be converted from the state to be reconstructed to the state to be set as the read-only and / or the opened state, after the stage of setting the read-only, the data file can be converted from the state to be set as the read-only to the read-only state, and the file in the opened state can be read by the outside. FIG. 8 is only an example, and does not mean that all the steps in FIG. 8 need to be strictly performed in the actual application. In some scenarios, some steps can be omitted or more steps can be added according to the actual needs. Corresponding to the application scenario of the method provided by the embodiment of the present disclosure and the method, the embodiment of the present disclosure further provides a data recovery apparatus, which can be applied to the metadata server in the distributed storage system as shown in FIG. 1. As shown in FIG. 10, the apparatus 1000 can include a scheduling module 1001, which can be configured to schedule a first block server to perform data recovery in the case that a second block server in the distributed storage system is down, the first block server being a block server that is not down, and the second block server being a block server that is down. The specific recovery method can refer to the related content in the above method embodiments, which will not be described here. Corresponding to the application scenario of the method provided by the embodiment of the present disclosure and the method, the embodiment of the present disclosure further provides a data recovery apparatus, which can be applied to the first block server in the distributed storage system as shown in FIG. 1. As shown in FIG. 11, the apparatus 1100 can include a data adding module 1101 and a data recovery module 1102. The data adding module 1101 is configured to add cloud disk data to be recovered in the second block server to a first queue in response to the scheduling of the metadata server in the case that the second block server exists in the distributed storage system. The data recovery module 1102 is configured to periodically perform data recovery on the cloud disk data in the first queue according to a preset time period.In a possible implementation, the cloud disk data is stored in the form of compressed files. When performing periodic data recovery on the cloud disk data in the first queue according to a preset time period, the data recovery module 1101 can be configured to: in each preset first time period, for specified cloud disk data in the first queue, reconstruct, based on metadata in the metadata server, a compressed file corresponding to the specified cloud disk data in the memory; and perform read-only setting on the reconstructed compressed file. The specified cloud disk data is cloud disk data that needs to be recovered in the current first time period. In a possible implementation, when performing read-only setting on the reconstructed compressed file, the data recovery module 1101 can be configured to: add the reconstructed compressed file to a second queue in which read-only setting is to be performed; and perform periodic read-only setting on the compressed files in the second queue according to a preset second time period. The second time period can be synchronous or asynchronous with the first time period. In a possible implementation, the data recovery module 1102 can be configured to: invoke a compressed file reconstruction task, and reconstruct, based on metadata in the metadata server, a compressed file corresponding to the specified cloud disk data in the memory in the compressed file reconstruction task; and invoke a file read-only management task, and perform read-only setting on the reconstructed compressed file in the file read-only management task. In a possible implementation, the data recovery module 1102 can be further configured to: in response to a read request sent by a client for target cloud disk data in the first queue that has not been recovered, perform data recovery on the target cloud disk data. In a possible implementation, the data recovery module 1102 can be configured to: in response to a read request sent by a client for target cloud disk data in the first queue that has not been recovered, reconstruct, based on metadata in the metadata server, a compressed file corresponding to the target cloud disk data in the memory; add the reconstructed compressed file to a second queue in which read-only setting is to be performed; and perform periodic read-only setting on the compressed files in the second queue according to a preset second time period. In a possible implementation, when adding cloud disk data that needs to be recovered in the second block server to the first queue, the data adding module 1101 can be configured to: add, to the first queue in which data recovery is to be performed, cloud disk data that needs to be recovered in the second block server, if the cloud disk data meets a preset adding condition. The adding condition can be set according to at least one of a quantity of cloud disk data that needs to be recovered and historical data of pressure of concurrently recovered cloud disk data.In a possible implementation, the data recovery apparatus 1100 can further include an ownership transfer module, which can be configured to transfer the file ownership of the cloud disk data in the second server from the second server to the first server before adding the cloud disk data in the second server to be recovered to the first queue. In a possible implementation, the first time period is determined according to historical data of pressure of concurrent recovery of cloud disk data. The functions of the modules in the apparatuses in the embodiments of the present disclosure can refer to the corresponding descriptions in the methods, and have the corresponding beneficial effects, which are not described here again. FIG. 12 is a block diagram of an electronic device for implementing the embodiments of the present disclosure. As shown in FIG. 12, the electronic device includes a memory 1201 and a processor 1202, and the memory 1201 stores a computer program capable of running on the processor 1202. The processor 1202 implements the method in the above embodiments when executing the computer program. The number of the memory 1201 and the processor 1202 can be one or more. The electronic device further includes a communication interface 1203 for communicating with external devices to transmit and receive data. If the memory 1201, the processor 1202 and the communication interface 1203 are independently implemented, the memory 1201, the processor 1202 and the communication interface 1203 can be connected with each other through a bus and complete communication between each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For convenience of representation, only one thick line is used in FIG. 12, but it does not mean that there is only one bus or only one type of bus. Optionally, in a specific implementation, if the memory 1201, the processor 1202 and the communication interface 1203 are integrated on a chip, the memory 1201, the processor 1202 and the communication interface 1203 can complete communication between each other through an internal interface. The embodiments of the present disclosure provide a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the method provided in the embodiments of the present disclosure. The embodiments of the present disclosure provide a computer program product, which includes a computer program, and the program is executed by a processor to implement the method provided in the embodiments of the present disclosure.The chip provided by the embodiments of the present disclosure includes a processor, which is configured to call and run instructions stored in a memory, so that a communication device installed with the chip performs the method provided by the embodiments of the present disclosure. The chip provided by the embodiments of the present disclosure includes an input interface, an output interface, a processor and a memory, the input interface, the output interface, the processor and the memory are connected through internal connection paths, the processor is configured to execute codes in the memory, and when the codes are executed, the processor is configured to perform the method provided by the embodiments of the present disclosure. It should be understood that the processor can be a central processing unit (CPU), or can be another general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or another programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. It should be noted that the processor can be a processor supporting an advanced RISC machine (ARM) architecture. Further, optionally, the memory can include a read-only memory and a random access memory. The memory can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. The non-volatile memory can include a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically EPROM (EEPROM) or a flash memory. The volatile memory can include a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM can be used.For example, Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Sync Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM). oIn the above-described embodiments, all or part can be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed by a computer, all or part generates a process or function according to the present disclosure. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable apparatus. Computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another computer-readable storage medium. In the description of the specification, the description of the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" and the like means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. Also, the specific features, structures, materials, or characteristics described can be combined in any appropriate manner in one or more embodiments or examples. Furthermore, the skilled person in the art can combine and combine the different embodiments or examples described in the specification and the features of the different embodiments or examples without contradiction. In addition, the terms "first", "second" are for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features limited by "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "a plurality of" is two or more, unless specifically limited otherwise. Any process or method described in the flowchart or otherwise described herein can be understood as a module, segment, or portion of code including one or more executable instructions for implementing specific logical functions or processes. And the scope of the preferred embodiments of the present disclosure includes additional implementations, in which the functions can be performed in the order shown or discussed, including in a substantially simultaneous manner or in reverse order according to the functions involved. The logic and / or steps described in the flowchart or otherwise described herein, for example, can be considered as a list of executable instructions for implementing logical functions, which can be specifically implemented in any computer-readable medium for instruction execution system, device or equipment, such as a computer-based system, a system including a processor, or other system that can take instructions from an instruction execution system, device or equipment, or in conjunction with these instruction execution systems, devices or equipment. It should be understood that parts of the present disclosure can be implemented in hardware, software, firmware, or a combination thereof.In the above-mentioned embodiments, the plurality of steps or methods can be implemented by software or firmware stored in the memory and executed by a suitable instruction execution system. All or part of the steps of the above-mentioned embodiment methods can be completed by programs instructing relevant hardware, and the programs can be stored in a computer readable storage medium. When the programs are executed, one or a combination of the steps of the embodiment methods is included. In addition, each functional unit in each embodiment of the present disclosure can be integrated in one processing module, or each unit can exist physically independently, or two or more units can be integrated in one module. The above-mentioned integrated module can be realized in the form of hardware, or in the form of a software functional module. If the above-mentioned integrated module is realized in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer readable storage medium. The storage medium can be a read-only memory, a magnetic disk or an optical disk, etc. The above-mentioned is only an exemplary embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any skilled person in the art can easily think of various changes or replacements within the technical range disclosed in the present disclosure, and these should be covered in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

Claims 1. A data recovery method, applied to the first server in a distributed storage system, the method comprising: In the case of a second server in the distributed storage system, in response to the scheduling of the metadata server, the cloud disk data to be recovered from the second server is added to the first queue; wherein, the first server is a block server that has not crashed, and the second server is a block server that has crashed; according to a preset time period, the cloud disk data in the first queue is periodically recovered.

2. The data recovery method according to claim 1, wherein, The cloud disk data is stored in the form of compressed files; the periodic data recovery of the cloud disk data in the first queue according to a preset time period includes: within each preset first time period, for a specified cloud disk data in the first queue, reconstructing the compressed file corresponding to the specified cloud disk data in memory based on the metadata in the metadata server; The specified cloud disk data is the cloud disk data that needs to be recovered within the current first time period; the reconstructed compressed file is set to read-only.

3. The data recovery method according to claim 2, wherein, Setting the reconstructed compressed file to read-only includes: adding the reconstructed compressed file to the second queue to be set to read-only; According to a preset second time period, the compressed files in the second queue are periodically set to read-only; the second time period may be synchronized with or not synchronized with the first time period.

4. The data recovery method according to claim 2 or 3, wherein, The step of reconstructing the compressed file corresponding to the specified cloud disk data in memory based on the metadata server includes: invoking a compressed file reconstruction task, in which the compressed file reconstruction task reconstructs the compressed file corresponding to the specified cloud disk data in memory based on the metadata server; the step of setting the reconstructed compressed file to read-only includes: invoking a file read-only management task, in which the file read-only management task sets the reconstructed compressed file to read-only.

5. The data recovery method according to any one of claims 1-3, further comprising: In response to a read request sent by the client for target cloud disk data that has not yet been recovered in the first queue, data recovery is performed on the target cloud disk data.

6. The data recovery method according to claim 5, wherein the step of recovering the target cloud disk data in response to a read request sent by the client for the target cloud disk data in the first queue includes: In response to a read request sent by the client for target cloud disk data that has not yet been recovered in the first queue, the compressed file corresponding to the target cloud disk data is reconstructed in memory from the metadata in the metadata server; Add the reconstructed compressed file to the second queue for read-only configuration; According to the preset second time period, the compressed files in the second queue are periodically set to read-only.

7. The data recovery method according to any one of claims 1-3, wherein, Adding the cloud disk data to be recovered in the second server to the first queue includes: adding the cloud disk data to the first queue when the cloud disk data to be recovered in the second server meets the preset addition conditions; the addition conditions are set based on at least one of the historical data of the number of cloud disk data to be recovered and the pressure of concurrent cloud disk data recovery.

8. The data recovery method according to any one of claims 1-3, further comprising, before adding the cloud disk data to be recovered in the second server to the first queue: The ownership of the cloud disk data in the second server is transferred from the second server to the first server.

9. The data recovery method according to any one of claims 1-3, wherein, The first time period is determined based on historical data on the pressure of restoring cloud disk data.

10. A data recovery method, applied to a metadata server in a distributed storage system, the method comprising: In the case of a second server in the distributed storage system, the first server is scheduled to perform the data recovery method as described in any one of claims 1-9; The first block server is a block server that has not experienced a crash, and the second block server is a block server that has experienced a crash.

11. A distributed storage system, comprising: The system comprises multiple metadata servers, multiple block servers, and a distributed file system; each block server is communicatively connected to a corresponding metadata server, and each block server is also communicatively connected to the distributed file system; in the case where a second block server exists among the multiple block servers, the metadata server is configured to execute the data recovery method of claim 10, and the first block server is configured to execute the data recovery method of any one of claims 1-9; the first block server is a block server that has not experienced a crash, and the second block server is a block server that has experienced a crash.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory, wherein the processor, when executing the computer program, implements the data recovery method according to any one of claims 1-10.

13. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the data recovery method according to any one of claims 1-10.

14. A computer program product comprising a computer program that, when executed by a processor, implements the method of any one of claims 1-10.

Images