Disaster recovery method, system, device, equipment and storage medium for IO data

Abstract

The present application relates to a disaster recovery method, system, device, equipment and storage medium for IO data. The disaster recovery method for IO data comprises: in response to a write request for IO data, extracting the offset and data length of the target IO data in the write request in the disk, and writing the target IO data into a cache file according to the offset and data length of the target IO data in the disk, so as to obtain the target IO data from the cache file and transmit it to the disaster recovery end, wherein the cache file is preheated in advance, and the preheating is the process of performing a write operation on the cache file according to the size of the minimum storage unit. The use of this method can avoid the problem of write amplification by preheating the cache file.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a disaster recovery method, system, device, equipment and storage medium for IO data. Background Art

[0002] Cloud computing can provide enterprises with computing resources allocated on demand. Modern enterprise IT infrastructure is gradually migrating from traditional architecture to cloud. Cloud computing can make full use of expensive hardware resources through virtualization technology and can also isolate the dependencies between hardware architecture and software system, improve system security performance, perform data backup, avoid losses, and improve the utilization rate of computing resources. With the development of cloud computing, the scale of cloud data centers continues to increase, and the requirements for data security are getting higher and higher. Disaster recovery systems have emerged. Disaster recovery technology is an environment that can cope with various disasters for computer information systems. When a computer system suffers from an irresistible natural disaster or a man-made disaster such as computer crime, computer virus, power failure, communication failure, hardware or software error, and human operation error, the disaster recovery system will ensure the security of user data (data disaster recovery). Even a more complete disaster recovery system can provide uninterrupted application services (application disaster recovery).

[0003] The original intention of establishing a disaster recovery system is to protect the integrity and security of application data at the most reasonable cost, to restore system operation as soon as possible after a disaster occurs, to reduce business downtime, to minimize interruption or impact on normal business operations, and to minimize the losses caused by disasters to enterprises. In other words, no matter how far apart the two systems are, when a problem occurs in one data center, the other data center can quickly take over the operation, while ensuring both the integrity of business data and the continuity of key businesses.

[0004] In traditional disaster recovery solutions, in order to ensure disk space utilization while taking into account performance requirements, the minimum storage unit usually needs to be configured to be larger. This configuration makes write amplification a problem that is prone to occur in traditional technologies. Summary of the invention

[0005] Based on this, it is necessary to provide a disaster recovery method, system, device, equipment and storage medium for IO data that can preheat cache files to avoid write amplification problems in response to the above technical problems.

[0006] A disaster recovery method for IO data is applied to a source end. The source end is provided with a file system and an IO data storage area. The IO data storage area has a cache file created. The file system is provided with a minimum storage unit. The disaster recovery method for IO data includes:

[0007] In response to a write request for IO data, extract an offset and a data length of the target IO data in the write request in the disk;

[0008] Write the target IO data into the cache file according to the offset and data length of the target IO data in the disk, so as to obtain the target IO data from the cache file and transmit it to the disaster recovery end;

[0009] The cache file is preheated in advance, and the preheating is a process of performing a write operation on the cache file according to the size of the minimum storage unit.

[0010] In one embodiment, in response to a write request for IO data, before extracting the offset and data length of the target IO data in the write request in the disk, the disaster recovery method for IO data further includes:

[0011] An initial write operation is performed on each cache file according to an integer multiple of a minimum storage unit size to preheat the cache file, and the minimum storage unit size is preconfigured.

[0012] In one embodiment, the cache file includes multiple files, and the IO data storage area also creates an index table, which records the usage status of each cache file, including used and unused. The target IO data is written into the cache file according to the offset of the target IO data in the disk and the data length, including:

[0013] Obtain the target cache file according to the usage status of each cache file, and the usage status of the target cache file is unused;

[0014] The target IO data is written into the target cache file according to the offset of the target IO data in the disk and the data length.

[0015] In one embodiment, the granularity of each cache file is the same, and the IO data disaster recovery method further includes:

[0016] Determine the offset of the target cache file in the disk according to the offset of the target IO data in the disk and the granularity of the cache file;

[0017] Add attribute information for the target cache file in the index table, the attribute information including virtual machine information corresponding to the target cache file, disk information, a time period corresponding to the stored IO data, and an offset of the target cache file in the disk.

[0018] In one embodiment, after writing the target IO data into a cache file for use according to the offset and data length of the target IO data in the disk, the IO data disaster recovery method further includes:

[0019] Mark the target cache file as used in the index table;

[0020] When all IO data in the target cache file are transmitted to the disaster recovery end, the mark of the target cache file in the index table is modified to unused, and the corresponding attribute information is deleted.

[0021] In one embodiment, the IO data disaster recovery method further includes:

[0022] Obtain the delivery time of the target IO data, and generate the bitmap information of the target IO data according to the offset and data length of the target IO data in the disk and the delivery time;

[0023] Match the delivery time in the bitmap information with the time period in the index table;

[0024] The target IO data is obtained based on the successfully matched cache file and transmitted to the disaster recovery end.

[0025] In one embodiment, the IO data disaster recovery method further includes:

[0026] When all IO data sent in the current time period are written into the target cache file, the bitmap information corresponding to the IO data in the current time period is written into the pre-created bitmap file, and identification information is added to the bitmap file, where the identification information includes the current time period;

[0027] Disaster recovery methods for IO data also include:

[0028] Obtain the delivery time of the target IO data, and generate the bitmap information of the target IO data according to the offset and data length of the target IO data in the disk and the delivery time;

[0029] Matching the time period corresponding to the text and image file to which the bitmap information belongs with the time period in the index table;

[0030] The target IO data is obtained based on the successfully matched cache file and transmitted to the disaster recovery end.

[0031] In one embodiment, the IO data disaster recovery method further includes:

[0032] The IO data storage area is divided into a data area, a bitmap area and an area where an index table is located. The cache file is stored in the data area, and the bitmap file is stored in the bitmap area.

[0033] In one embodiment, writing the target IO data into the target cache file according to the offset of the target IO data in the disk and the data length includes:

[0034] Determine the offset of the target IO data in the target cache file according to the offset of the target IO data in the disk and the granularity of the cache file;

[0035] The target IO data is written into the target cache file according to the offset of the target IO data in the target cache file.

[0036] In one embodiment, the IO data disaster recovery method further includes:

[0037] Determine the address range of the target cache file according to the offset and granularity of the target IO data in the disk;

[0038] When new IO data is received, it is determined whether the new IO data falls within the address range based on the offset and data length of the new IO data in the disk;

[0039] When new IO data falls within the address range, the new IO data is written to the target cache file.

[0040] In one embodiment, the IO data disaster recovery method further includes:

[0041] When the new IO data does not fall within the address range, a new cache file is obtained and the new IO data is written into the new cache file.

[0042] A disaster recovery system for IO data includes a source end and a disaster recovery end. The source end is provided with a disaster recovery module, a file system and an IO data storage area. The IO data storage area is provided with a cache file.

[0043] The disaster recovery module is used to perform a write operation on the cache file according to the size of the minimum storage unit to preheat the cache file;

[0044] The disaster recovery module is also used to respond to the write request of IO data, extract the offset and data length of the target IO data in the write request, send the offset and data length of the target IO data in the disk to the file system, and write the target IO data into the cache file through the file system;

[0045] The disaster recovery module is also used to generate bitmap information according to the offset of the target IO data in the disk and the data length, so as to obtain the target IO data from the cache file according to the bitmap information and transmit it to the disaster recovery end.

[0046] A disaster recovery device for IO data, the device is applied to a source end, the source end is provided with a file system and an IO data storage area, the IO data storage area has a cache file created, the file system is provided with a minimum storage unit, and the disaster recovery device for IO data includes:

[0047] An extraction module, used for extracting the offset and data length of the target IO data in the write request in the disk in response to the write request of the IO data;

[0048] The write module is used to write the target IO data into the cache file according to the offset and data length of the target IO data in the disk, so as to obtain the target IO data from the cache file and transmit it to the disaster recovery end, wherein the cache file is preheated in advance, and preheating is the process of performing a write operation on the cache file according to the size of the minimum storage unit.

[0049] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0050] In response to a write request for IO data, extract an offset and a data length of the target IO data in the write request in the disk;

[0051] Write the target IO data into the cache file according to the offset and data length of the target IO data in the disk, so as to obtain the target IO data from the cache file and transmit it to the disaster recovery end;

[0052] The cache file is preheated in advance, and the preheating is a process of performing a write operation on the cache file according to the size of the minimum storage unit.

[0053] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0054] In response to a write request for IO data, extract an offset and a data length of the target IO data in the write request in the disk;

[0055] Write the target IO data into the cache file according to the offset and data length of the target IO data in the disk, so as to obtain the target IO data from the cache file and transmit it to the disaster recovery end;

[0056] The cache file is preheated in advance, and the preheating is a process of performing a write operation on the cache file according to the size of the minimum storage unit.

[0057] The above-mentioned IO data disaster recovery method, system, device, equipment and storage medium preheat the cache file by performing the initial write operation on the cache file according to the size of the minimum storage unit, so that the subsequent writing of IO data is written according to the actual data length of the target IO data, which solves the problem that in the traditional technology, no preheating operation is performed, resulting in the first write operation being performed according to the size of the minimum storage unit, and when the target IO data is smaller than the size of the minimum storage unit, it will cause write amplification. Therefore, the IO data disaster recovery method of the present application can preheat the cache file and avoid performance problems caused by write amplification. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1A schematic diagram of a process flow of a disaster recovery method for IO data in one embodiment;

[0059] Figure 2 A schematic diagram of a process for dividing an IO data storage area in one embodiment;

[0060] Figure 3 A schematic diagram of a process of formatting a directory of an IO data storage area in one embodiment;

[0061] Figure 4 A schematic diagram of writing new IO data into a cache file in one embodiment;

[0062] Figure 5 A structural block diagram of a disaster recovery system for IO data in an embodiment;

[0063] Figure 6 A structural block diagram of a disaster recovery device for IO data in one embodiment;

[0064] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0066] In one embodiment, Figure 1 As shown, a disaster recovery method for IO data is provided, the method is applied to a source end, the source end is provided with a file system and an IO data storage area, the IO data storage area is provided with a cache file, the file system is provided with a minimum storage unit, and the method includes:

[0067] Step S11 , in response to a write request for IO data, extracting an offset and a data length of the target IO data in the write request in the disk.

[0068] The source end, also called the production end, refers to the source of IO data. The disaster recovery end refers to the remote server used to store IO data. IO data refers to input and output data. Target IO data refers to IO data that needs to be written to the cache file.

[0069] The file system can be OCFS2 (Oracle Cluster File System 2), which is an open source file system designed for cluster environments. The minimum storage unit is Cluster size, which is the minimum storage unit in the file system and is used to determine the size of the data storage unit on the disk. It is the minimum unit of file storage.

[0070] Cache files are files used to store IO data.

[0071] The target IO data refers to the data to be written into the cache file. The offset of the target IO data in the disk refers to the starting position of the target IO data in the disk.

[0072] Step S12, write the target IO data into the cache file according to the offset and data length of the target IO data in the disk, so as to obtain the target IO data from the cache file and transmit it to the disaster recovery end, wherein the cache file is preheated in advance, and preheating is a process of performing a write operation on the cache file according to the size of the minimum storage unit.

[0073] Specifically, the source end of the present application is deployed with a disaster recovery module, and the disaster recovery method of IO data of the present application is specifically executed in the disaster recovery module of the source end. The source end is also deployed with a virtual machine and a file system, and the source end is also provided with an IO storage area. The disaster recovery module preheats the cache file in advance. When the IO data of the virtual machine of the source end is captured by the disaster recovery module in some way, the disaster recovery module calculates the position of the target IO data in the cache file according to the offset of the target IO data in the disk and the data length, and transmits the calculation result to the file system. The file system writes the target IO data into the IO data storage area according to the calculation result, and further transmits the IO data in the IO data storage area to the IO data storage area of ​​the disaster recovery end. The disaster recovery end then merges the IO data into the disaster recovery virtual machine of the disaster recovery end to complete disaster recovery.

[0074] Therefore, the disaster recovery module in the present application interacts with the file system, and finally writes the IO data into the IO data storage area and transmits it to the disaster recovery end.

[0075] In traditional technology, when the physical performance is met, when the virtual machine processes high-voltage IO, disaster recovery is required to meet the current high-voltage IO processing performance. Therefore, backing up the virtual machine requires high performance requirements for the file system on the storage. In order to ensure the utilization of disk space, the file system also needs to take into account its performance requirements. Therefore, in the OCFS2 file system, in order to meet the above requirements, the minimum storage unit size is usually allocated to 1M. However, this configuration will lead to write amplification problems. The write amplification problem lies in writing a new IO data. The OCFS2 file system will allocate space of the Cluster size at a time for unified writing. If the size of the new IO data is smaller than the Cluster size, a write amplification problem will occur; and when writing new IO data, an erase operation will be performed on the allocated Cluster size to erase dirty data to facilitate the subsequent correct acquisition of valid data.

[0076] Based on this defect in the traditional technology, the present application adds a disaster recovery module, through which the cache file is preheated in advance. By preheating the cache file, in the subsequent execution process, data is no longer written according to the size of the minimum storage unit, but according to the data length of the template IO data, thereby avoiding the problem of write amplification caused when the data block is smaller than the size of the minimum storage unit. Preheating refers to performing the initial write operation on the cache file, and specifically, the cache file can be filled with data according to the size of the minimum storage unit, such as selecting a data block with the same size as the minimum storage unit to fill the cache file with data.

[0077] In one of the embodiments, in response to a write request for IO data, before extracting the offset and data length of the target IO data in the write request on the disk, the disaster recovery method for IO data may also include: performing an initial write operation on each cache file according to an integer multiple of the minimum storage unit size to preheat the cache file, and the minimum storage unit size is pre-configured.

[0078] The present application pre-configures the minimum storage unit size, and illustratively, the minimum storage unit size may be configured as 1 M. Then the present application may fill data in each cache file according to an integer multiple of 1 M to implement the initial write operation.

[0079] Since a write amplification problem will occur if the IO data size is smaller than the minimum storage unit size, the present application selects an integer multiple of the minimum storage unit size for data filling to avoid the write amplification problem.

[0080] The beneficial effect of this design is that by performing the initial write operation on each cache file, the cache file can be preheated in advance, so that there will be no write amplification affecting performance during subsequent execution.

[0081] In one of the embodiments, the cache files include multiple files, and an index table is also created in the IO data storage area, which records the usage status of each cache file, the usage status includes used and unused, and the target IO data is written to the cache file according to the offset of the target IO data in the disk and the data length, including: obtaining the target cache file according to the usage status of each cache file, the usage status of the target cache file is unused, and writing the target IO data to the target cache file according to the offset of the target IO data in the disk and the data length.

[0082] The index table is used to record information of each cache file. Specifically, the index table can record information such as the identifier of each cache file and the usage status of each cache file. The usage status refers to whether the file is being used.

[0083] The unused state refers to the state where no IO data is written (data written during preheating is not counted as used). Specifically, the present application can create a use state for each cache file, and assign a value to the use state to distinguish which files are unused files and which files are used files. Exemplarily, two states can be set: used and unused, and the used state value can be further set to 1, and the unused state value to 0, then the value of the state is the identifier of the unused file mentioned above.

[0084] Alternatively, the usage status of each cache file is marked as used or unused.

[0085] Furthermore, when data is written, unused files are obtained by querying the index table, and then the target IO data is written into the unused files.

[0086] Alternatively, you can also obtain a file in the used files that has not been filled with data, and determine whether the target IO data can fall into the cache file. If so, directly fall into the used cache file. If not, obtain a new unused cache file for data writing.

[0087] The beneficial effect of this design is that by constructing an index table, the usage status of the cache files can be uniformly managed, so that the cache files suitable for writing IO data can be quickly found according to the index table, thereby improving data writing efficiency.

[0088] In one of the embodiments, the granularity of each cache file is the same, and the disaster recovery method for IO data may also include: determining the offset of the target cache file in the disk based on the offset of the target IO data in the disk and the granularity of the cache file, and adding attribute information for the target cache file in the index table, the attribute information including virtual machine information corresponding to the target cache file, disk information, a time period corresponding to the stored IO data, and the offset of the target cache file in the disk.

[0089] Among them, the granularity size of each cache file refers to the size of each cache file, and the granularity size is pre-set. The present application pre-divides and obtains a plurality of cache files according to the set granularity size.

[0090] The offset of the target cache file in the disk refers to the starting position of the target cache file in the disk. The attribute information refers to information used to describe the virtual machine, disk, time period and a certain address corresponding to the target cache file.

[0091] Specifically, each time the present application performs a write operation on a cache file, if the cache file is an unused file, the cache file does not have attribute information, and the corresponding attribute information will be added to the cache file. Exemplarily, when the cache file P is written to data, if the cache file P is an unused cache file, after writing the IO data, the attribute information will be added to the cache file P according to the target IO data.

[0092] The time period corresponding to the stored IO data refers to the data stored in each cache file corresponding to a certain time period, and the time period is the time period in the attribute information.

[0093] Please refer to Table 1 below, which shows the structure of an index table in an embodiment.

[0094] Table 1

[0095]

[0096] Among them, TIME represents the time period corresponding to the stored IO data, and offset represents the offset of the target cache file in the disk. In Table 1, the file numbers of all unused files are recorded in the same data record, and a corresponding data record is created for each used file.

[0097] The beneficial effect of this design is that by adding the attribute information of each used file to the index table, better management of each cache file is achieved.

[0098] In one of the embodiments, after writing the target IO data into the target cache file according to the offset and data length of the target IO data in the disk, the disaster recovery method for IO data may also include: marking the target cache file as used in the index table, and when all the IO data in the target cache file are transmitted to the disaster recovery end, modifying the mark of the target cache file in the index table to not used, and deleting the corresponding attribute information.

[0099] In this application, after receiving the target IO data, the index table is queried, and the unused cache files and the target cache files are obtained by querying the index table, and the IO data is further written into the target cache file. After writing, the usage status of the target cache file is changed to used.

[0100] Furthermore, the usage status of the target cache file continues to be monitored. When it is detected that all IO data in the target cache file are transmitted to the disaster recovery end, the usage status of the target cache file is restored to an unused state, and the previously created attribute information is deleted at the same time.

[0101] The beneficial effect of this design is that the application can create an index table to record and monitor the usage status of each cache file to facilitate the management of each cache file, and then obtain accurate data based on the index table for data transmission, thereby improving disaster recovery efficiency.

[0102] In one of the embodiments, the disaster recovery method for IO data may also include: obtaining the sending time of the target IO data, and generating bitmap information of the target IO data based on the offset and data length of the target IO data in the disk and the sending time, matching the sending time in the bitmap information with the time period in the index table, and obtaining the target IO data based on the successfully matched cache file and transmitting it to the disaster recovery end.

[0103] The bitmap information refers to information used to record the location of the IO data on the disk and the time of issuance. Specifically, the bitmap information may include information such as the offset of the IO data on the disk, the data length, and the time of issuance. Exemplarily, the content of the bitmap information may be {1.11G.20M}, where 1.11G is the offset of the IO data on the disk, and 20M is the length of the IO data.

[0104] Furthermore, the present application can obtain accurate target IO data according to the bitmap information and the index table, so as to transmit the target IO data to the disaster recovery end.

[0105] Specifically, the present application can match the sending time in the bitmap information with the time period in the attribute information corresponding to each used cache file in the index table. When the match is successful, the offset in the disk corresponding to the successfully matched cache file is obtained, and the offset of the target IO data in the cache file is further calculated based on the offset of the cache file in the disk and the offset of the target IO data in the disk. Further, based on the offset of the target IO data in the cache file, the target IO data is obtained from the cache file and transmitted to the disaster recovery end.

[0106] The beneficial effect of this design is that IO data can be accurately obtained for data transmission based on the index table and bitmap information, thereby improving the accuracy of data transmission.

[0107] In one of the embodiments, the disaster recovery method for IO data may also include: when all IO data issued within the current time period are written into the target cache file, the bitmap information corresponding to the IO data within the current time period is written into a pre-created bitmap file, and identification information is added to the bitmap file, wherein the identification information includes the current time period. The disaster recovery method for IO data may also include obtaining the issuance time of the target IO data, and generating the bitmap information of the target IO data based on the offset and data length of the target IO data in the disk and the issuance time, matching the time period corresponding to the text and image file to which the bitmap information belongs with the time period in the index table, and obtaining the target IO data based on the successfully matched cache file and transmitting it to the disaster recovery end.

[0108] In the present application, a time period is preset, and the bitmap information within the same time period is stored in a corresponding bitmap file, and identification information is added to each bitmap file, such as a file name, and the information indicates the time period corresponding to each bitmap file. In addition, the bitmap information within the same time period can be deduplicated and merged, and the IO data within the same time period can also be deduplicated and merged.

[0109] Furthermore, each cache file also has a corresponding time period, and the IO data in each time period is stored in a corresponding cache file. The attribute information corresponding to each cache file carries the corresponding time period.

[0110] Furthermore, when performing data transmission, the present application can match the time period corresponding to the bitmap file with the time period corresponding to the cache file. When the match is successful, the data to be transmitted can be accurately obtained based on the offset of the cache file on the disk in the attribute information of the successfully matched cache file and the offset of the IO data on the disk in the bitmap information, and transmitted to the disaster recovery end.

[0111] Exemplarily, assuming that the time period of the bitmap file M is consistent with that of the cache file N, the offset of the cache file N in the disk is determined according to the attribute information of the cache file N. Assuming that the bitmap file M includes three IO data, namely IO data 1, IO data 2 and IO data 3, the offset of IO data 1 in the cache file N is determined according to the offset of IO data 1 in the disk in the bitmap information corresponding to IO data 1 and the offset of the cache file N in the disk, and then IO data 1 is obtained from the cache file N according to the offset of IO data 1 in the cache file N and transmitted to the disaster recovery end. Similarly, IO data 2 and IO data 3 can be obtained in the same way and transmitted to the disaster recovery end. When all the data in the cache file N are transmitted to the disaster recovery end, the use status of the cache file N is changed to unused, and the attribute information corresponding to the cache file N is deleted.

[0112] The beneficial effect of this design is that IO data can be accurately obtained for data transmission based on the index table and bitmap information, thereby improving the accuracy of data transmission.

[0113] In one embodiment, the IO data disaster recovery method may further include: dividing the IO data storage area to obtain a data area, a bitmap area, and an area where an index table is located, storing cache files in the data area, and storing bitmap files in the bitmap area.

[0114] The data area refers to an area for storing IO data, the bitmap area refers to an area for storing bitmap information, and the area where the index table is located refers to an area for storing the index table.

[0115] Alternatively, the present application may also divide the IO data storage area into two areas and one file, namely a data area, a bitmap area, and an index table.

[0116] Specifically, when starting the IO data disaster recovery process, the application initializes the area storing IO, and divides it into two areas and one file according to function, namely the data area, the bitmap area, and the index table. In the data area, several cache files are created according to the preset granularity, and write operations are performed on all cache files. After writing, the identifiers of each cache file are loaded into the index table. At this point, all cache files in the data area have been executed for the first time, and there will be no write amplification affecting performance in the subsequent execution process.

[0117] Furthermore, after the source receives the target IO data, it obtains an unused target cache file by checking the index table, and then overwrites the target IO data to the offset position after segmentation according to the granularity size, and records the bitmap information of the target IO data to the bitmap area, updates the cache index table, and marks the attribute information of the cache file. When the data is subsequently transmitted, the corresponding IO data is obtained according to the index table and bitmap information for transmission. The IO data of the entire process obtains accurate information from the bitmap information, and the cache files in the data area are all preheated, so there is no excessive performance loss.

[0118] Please refer to Figure 2 , Figure 2 This is a schematic diagram of a process for dividing an IO data storage area in an embodiment. Figure 2 In the example, the IO data storage area is divided into a data area, an index table and a bitmap area. A cache file is created in the data area, and the cache file is used to store IO data. A bitmap file is created in the bitmap area, and the bitmap file is used to store bitmap information. The index table is used to record the usage status and attribute information of each cache file.

[0119] When dividing the IO data storage area in this application, it is necessary to format the directory of the IO data storage area. Figure 3 , Figure 3 In one embodiment, a schematic diagram of a process for formatting a directory in an IO data storage area is shown. Figure 3 In it, multiple directories are included, among which directory 31 is the directory corresponding to the IO data storage area, directory 32 is the directory corresponding to the bitmap area, directory 33 is the directory corresponding to the data area, directory 34 is the subdirectory under directory 32, directory 35 and directory 36 are subdirectories under directory 33, directory 37 is the directory of bitmap files, directory 37 contains the bitmap information corresponding to each cycle, and directory 38 is the directory corresponding to each cache file.

[0120] In one of the embodiments, target IO data is written to a target cache file according to an offset of the target IO data in the disk and a data length, including: determining an offset of the target IO data in the target cache file according to the offset of the target IO data in the disk and a granularity of the cache file, and writing the target IO data to the target cache file according to the offset of the target IO data in the target cache file.

[0121] The granularity of the cache file refers to the size of the cache file. The offset of the target IO data in the target cache file refers to the starting position of the target IO data in the target cache file.

[0122] Specifically, the present application can take the modulus of the target IO data offset in the disk and the granularity of the cache file to obtain the target IO data offset in the target cache file. For example, 1.12G is taken modulus of 1G, and 0.12G is obtained, which is the target IO data offset in the target cache file.

[0123] For example, it is assumed that virtual machine A sends an IO data D to disk C, the offset of IO data D in disk C is 1.11G, the length is 20M, and the granularity of the cache file is 1G. Specifically, the disaster recovery module at the source end responds to the write request of IO data and queries the unused cache file in the index table. If an unused cache file B is found, the address range of cache file B is (1G, 2G). According to the offset of IO data D in disk C is 1.11G, it is calculated that the offset of IO data D in the cache file is 0.11G, then the IO data is overwritten and written to cache file B at this offset, and at the same time, attribute information is added to cache file B as "virtual machine A_disk C_time period_1G", where 1G in the attribute information is the starting address of cache file B, that is, the offset of cache file B in the disk. Further, bitmap information of IO data D is generated, and the content of the bitmap information is {1.11G.20M}.

[0124] In the present application, since the cache file is preheated in advance, when the IO data is subsequently written, it can be written according to the data length of the IO data, thus avoiding the write amplification problem.

[0125] In one of the embodiments, the disaster recovery method for IO data may also include: determining the address range of the target cache file based on the offset and granularity size of the target IO data in the disk, and when new IO data is received, judging whether the new IO data falls within the address range based on the offset and data length of the new IO data in the disk, and when the new IO data falls within the address range, writing the new IO data into the target cache file.

[0126] Specifically, determining the address range of the target cache file according to the offset and granularity of the target IO data in the disk may include:

[0127] The offset of the target IO data in the disk is modulo the granularity size to obtain the quotient and remainder, and the address range of the target cache file is determined according to the quotient and remainder.

[0128] Continuing with the above example, assume that virtual machine A sends an IO data D to disk C. The offset of IO data D in disk C is 1.11G, the length is 20M, and the granularity of the cache file is 1G. 1.11G modulo 1G, the quotient is 1, the remainder is 0.11G, and the address range is (1G, 2G). If disk C receives another new IO data in the subsequent cycle, its offset in the disk is 1.57G, and the length is 10M, then 1.57G falls within the range (1G, 2G), and the length is suitable, then the new IO data will continue to be written to cache file B, and the bitmap information of the new IO data will be generated as {1.57G, 10M}.

[0129] In one of the embodiments, the IO data disaster recovery method may further include: when the new IO data does not fall within the address range, obtaining a new cache file, and writing the new IO data into the new cache file.

[0130] Continuing with the above example, assume that virtual machine A sends an IO data D to disk C. The offset of IO data D in disk C is 1.11G, the length is 20M, and the granularity of cache file B is 1G. When the new IO data is not within the above address range, a new cache file needs to be allocated. After waiting for all IO data in the current cycle to fall, the bitmap information of each recorded IO data is stored in the corresponding bitmap file, and the bitmap file is named "ser_time period_bitmap", where the time period in the bitmap file is consistent with the time period of the attribute information of cache file B.

[0131] Please refer to Figure 4 , Figure 4 FIG. 1 is a schematic diagram of writing new IO data into a cache file in an embodiment. Figure 4 As shown, when new IO data is received, it is determined whether a new cache file is needed based on the offset and data length of the new IO data in the disk. If it is not needed, the new IO data is written to the previous cache file. If it is needed, a new cache file is obtained and the new IO data is written to the new cache file. Wherein, offset is the offset of the new IO data in the disk, and length is the data length of the new IO data.

[0132] In one embodiment, Figure 5As shown, a disaster recovery system for IO data is provided, including a source end and a disaster recovery end, the source end is provided with a disaster recovery module, a file system and an IO data storage area, the IO data storage area is provided with a cache file, the file system is provided with a minimum storage unit, wherein the disaster recovery module is used to perform a write operation on the cache file according to the size of the minimum storage unit for preheating, the disaster recovery module is also used to respond to a write request for IO data, extract the offset and data length of the target IO data in the write request in the disk, send the target IO data to the file system according to the offset and data length of the target IO data in the disk, write the cache file through the file system, and the disaster recovery module is also used to generate bitmap information according to the offset and data length of the target IO data in the disk, so as to obtain the target IO data from the cache file according to the bitmap information and transmit it to the disaster recovery end.

[0133] Specifically, after the disaster recovery module generates the bitmap information, the bitmap information may be sent to the file system, and the file system obtains the target IO data from the cache file according to the bitmap information and transmits it to the disaster recovery end.

[0134] Based on the traditional technology, this application adds a disaster recovery module, which preheats the cache files in advance through the disaster recovery module. After preheating, it interacts with the file system to write IO data into the cache files, and simultaneously creates an index table to record and manage each cache file. In addition, bitmap information of each IO data is also created so as to accurately obtain the corresponding IO data according to the bitmap information and the index table and transmit it to the disaster recovery end.

[0135] In addition, the disaster recovery module of the present application is not bound to a specific file system and does not require modification of the underlying file system to meet performance requirements, thereby reducing the difficulty of function development and the scope of impact of problems.

[0136] It can be seen that compared with the traditional technology, the present application adds a disaster recovery module. The disaster recovery method of IO data is specifically run in the disaster recovery module. The cache file is preheated through the disaster recovery module. Further, the disaster recovery module writes the target IO data into the IO data storage area through the file system. On the one hand, the present application avoids the write amplification problem by preheating the cache file. On the other hand, the disaster recovery module of the present application is a functional module independent of the file system. The advantage of this design is that it does not need to modify the underlying file system and is not bound to a specific file system. It can meet the performance requirements, which is convenient and fast. Moreover, the disaster recovery module of the present application can support all file systems and has high flexibility.

[0137] In one embodiment, Figure 6 As shown, a disaster recovery device for IO data is provided, the device is applied to a source end, the source end is provided with a file system and an IO data storage area, the IO data storage area is provided with a cache file, the file system is provided with a minimum storage unit, and the device includes:

[0138] An extraction module 61 is used to extract the offset and data length of the target IO data in the write request in the disk in response to the write request of the IO data;

[0139] The write module 62 is used to write the target IO data into the cache file according to the offset and data length of the target IO data in the disk, so as to obtain the target IO data from the cache file and transmit it to the disaster recovery end, wherein the cache file is preheated in advance, and preheating is a process of performing a write operation on the cache file according to the size of the minimum storage unit.

[0140] In one embodiment, the IO data disaster recovery device also includes a preheating module (not shown). The preheating module can perform an initial write operation on each cache file according to an integer multiple of the minimum storage unit size to preheat the cache file. The minimum storage unit size is pre-configured.

[0141] In one embodiment, the cache files include multiple files, and an index table is also created in the IO data storage area. The index table records the usage status of each cache file, and the usage status includes used and unused. The writing module 62 can obtain the target cache file according to the usage status of each cache file. The usage status of the target cache file is unused. The target IO data is written to the target cache file according to the offset of the target IO data in the disk and the data length.

[0142] In one of the embodiments, the granularity of each cache file is the same. The write module 62 can also determine the offset of the target cache file in the disk based on the offset of the target IO data in the disk and the granularity of the cache file, and add attribute information for the target cache file in the index table. The attribute information includes the virtual machine information corresponding to the target cache file, the disk information, the time period corresponding to the stored IO data, and the offset of the target cache file in the disk.

[0143] In one embodiment, the writing module 62 may also mark the target cache file as used in the index table. When all IO data in the target cache file is transmitted to the disaster recovery end, the mark of the target cache file in the index table is modified to unused and the corresponding attribute information is deleted.

[0144] In one of the embodiments, the disaster recovery device for IO data also includes a transmission module (not shown). The transmission module can obtain the sending time of the target IO data, and generate bitmap information of the target IO data based on the offset and data length of the target IO data in the disk and the sending time, match the sending time in the bitmap information with the time period in the index table, and obtain the target IO data based on the successfully matched cache file and transmit it to the disaster recovery end.

[0145] In one of the embodiments, the writing module 62 can also write the bitmap information corresponding to the IO data in the current time period into a pre-created bitmap file when all the IO data issued in the current time period are written into the target cache file, and add identification information to the bitmap file, wherein the identification information includes the current time period, obtains the issuing time of the target IO data, and generates the bitmap information of the target IO data according to the offset and data length of the target IO data in the disk and the issuing time, matches the time period corresponding to the text file to which the bitmap information belongs with the time period in the index table, and obtains the target IO data according to the successfully matched cache file and transmits it to the disaster recovery end.

[0146] In one embodiment, the writing module 62 may further: divide the IO data storage area into a data area, a bitmap area and an area where the index table is located, wherein the cache file is stored in the data area and the bitmap file is stored in the bitmap area.

[0147] In one embodiment, the write module 62 can determine the offset of the target IO data in the target cache file according to the offset of the target IO data in the disk and the granularity of the cache file, and write the target IO data into the target cache file according to the offset of the target IO data in the target cache file.

[0148] In one embodiment, the write module 62 can also: determine the address range of the target cache file based on the offset and granularity size of the target IO data in the disk, and when new IO data is received, determine whether the new IO data falls within the address range based on the offset and data length of the new IO data in the disk, and write the new IO data to the target cache file when the new IO data falls within the address range.

[0149] In one embodiment, the writing module 62 may further: when the new IO data does not fall within the address range, obtain a new cache file, and write the new IO data into the new cache file.

[0150] In one embodiment, a computer device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the IO data disaster recovery method provided in any of the above embodiments are implemented.

[0151] In one embodiment, the computer device may be a server, and its internal structure diagram may be as follows: Figure 7As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a disaster recovery method for IO data is implemented.

[0152] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the IO data disaster recovery method provided in any of the above embodiments are implemented.

[0153] The computer readable storage medium may be Figure 7 A computer-readable storage medium in the computer device shown.

[0154] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the above-mentioned computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0155] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0156] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A disaster recovery method for IO data, characterized in that: The method is applied to a source end, the source end is provided with a file system and an IO data storage area, the IO data storage area is provided with a cache file, and the file system is provided with a minimum storage unit. The method comprises: In response to a write request for IO data, extracting an offset and a data length of target IO data in the write request in the disk; The cache files include multiple ones, and the IO data storage area also creates an index table, which records the usage status of each cache file, and obtains the target cache file according to the usage status of each cache file, and the usage status of the target cache file is unused; Writing the target IO data into the target cache file according to the offset and data length of the target IO data in the disk, so as to obtain the target IO data from the cache file and transmit it to the disaster recovery end; The cache file is preheated in advance, and the preheating is a process of performing a write operation on the cache file according to the size of the minimum storage unit; The granularity of each cache file is the same, and the method further includes: Determine the offset of the target cache file in the disk according to the offset of the target IO data in the disk and the granularity of the cache file; Add attribute information for the target cache file in the index table, the attribute information including virtual machine information corresponding to the target cache file, disk information, a time period corresponding to the stored IO data, and an offset of the target cache file in the disk.

2. The method according to claim 1, characterized in that Before extracting the offset and data length of the target IO data in the write request in response to the write request of the IO data, the method further comprises: An initial write operation is performed on each of the cache files according to an integer multiple of the minimum storage unit size to preheat the cache file, and the minimum storage unit size is preconfigured.

3. The method according to claim 1, characterized in that After writing the target IO data into the unused cache file according to the offset and data length of the target IO data in the disk, the method further includes: Marking the target cache file as used in the index table; When all IO data in the target cache file are transmitted to the disaster recovery end, the mark of the target cache file in the index table is modified to unused, and the corresponding attribute information is deleted.

4. The method according to claim 3, characterized in that The method further comprises: Obtaining the sending time of the target IO data, and generating bitmap information of the target IO data according to the offset and data length of the target IO data in the disk and the sending time; Matching the sending time in the bitmap information with the time period in the index table; The target IO data is obtained according to the successfully matched cache file and transmitted to the disaster recovery end.

5. The method according to claim 3, characterized in that: The method further comprises: When all IO data issued in the current time period are written into the target cache file, the bitmap information corresponding to the IO data in the current time period is written into a pre-created bitmap file, and identification information is added to the bitmap file, wherein the identification information includes the current time period; The method further comprises: Obtaining the sending time of the target IO data, and generating bitmap information of the target IO data according to the offset and data length of the target IO data in the disk and the sending time; Matching the time period corresponding to the text and image file to which the bitmap information belongs with the time period in the index table; The target IO data is obtained according to the successfully matched cache file and transmitted to the disaster recovery end.

6. The method according to claim 5, characterized in that The method further comprises: The IO data storage area is divided into a data area, a bitmap area and an area where the index table is located, the cache file is stored in the data area, and the bitmap file is stored in the bitmap area.

7. The method according to claim 3, characterized in that The step of writing the target IO data into the target cache file according to the offset of the target IO data in the disk and the data length includes: Determine the offset of the target IO data in the target cache file according to the offset of the target IO data in the disk and the granularity of the cache file; The target IO data is written into the target cache file according to the offset of the target IO data in the target cache file.

8. The method according to claim 7, characterized in that The method further comprises: Determine the address range of the target cache file according to the offset of the target IO data in the disk and the granularity; When new IO data is received, judging whether the new IO data falls within the address range according to the offset and data length of the new IO data in the disk; When the new IO data falls within the address range, the new IO data is written into the target cache file.

9. The method according to claim 8, characterized in that The method further comprises: When the new IO data does not fall within the address range, a new cache file is acquired, and the new IO data is written into the new cache file.

10. A disaster recovery system for IO data, characterized in that: It includes a source end and a disaster recovery end, wherein the source end is provided with a disaster recovery module, a file system and an IO data storage area, the IO data storage area is provided with a cache file, and the file system is provided with a minimum storage unit, wherein: The disaster recovery module is used to perform a write operation on the cache file according to the size of the minimum storage unit to preheat the cache file; The disaster recovery module is also used for responding to a write request for IO data, extracting the offset and data length of the target IO data in the write request in the disk, the cache files include a plurality, the IO data storage area also creates an index table, the index table records the use status of each of the cache files, the target cache file is obtained according to the use status of each of the cache files, the use status of the target cache file is unused, and the target IO data is written into the target cache file according to the offset and data length of the target IO data in the disk; The disaster recovery module is also used to generate bitmap information according to the offset and data length of the target IO data in the disk, so as to obtain the target IO data from the cache file according to the bitmap information and transmit it to the disaster recovery end; The granularity of each cache file is the same. The disaster recovery module is also used to determine the offset of the target cache file in the disk according to the offset of the target IO data in the disk and the granularity of the cache file, and add attribute information for the target cache file in the index table. The attribute information includes the virtual machine information corresponding to the target cache file, the disk information, the time period corresponding to the stored IO data, and the offset of the target cache file in the disk.

11. A disaster recovery device for IO data, characterized in that: The device is applied to a source end, the source end is provided with a file system and an IO data storage area, the IO data storage area is provided with a cache file, the file system is provided with a minimum storage unit, and the device comprises: An extraction module, configured to extract, in response to a write request for IO data, an offset and a data length of target IO data in the write request in the disk; A writing module, for the cache files including a plurality of files, the IO data storage area also creates an index table, the index table records the use status of each cache file, obtains a target cache file according to the use status of each cache file, the use status of the target cache file is unused, writes the target IO data into the target cache file according to the offset and data length of the target IO data in the disk, so as to obtain the target IO data from the cache file and transmit it to the disaster recovery end, wherein the cache file is preheated in advance, and the preheating is a process of performing a write operation on the cache file according to the size of the minimum storage unit; The granularity of each cache file is the same. The write module is also used to determine the offset of the target cache file in the disk according to the offset of the target IO data in the disk and the granularity of the cache file, and add attribute information for the target cache file in the index table. The attribute information includes virtual machine information corresponding to the target cache file, disk information, a time period corresponding to the stored IO data, and the offset of the target cache file in the disk.

12. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images