Block incremental backup method based on shared storage cluster, product, medium and equipment

Abstract

The invention relates to a database technology, in particular to a block incremental backup method based on a shared storage cluster, a product, a medium and equipment. The block incremental backup method comprises the steps that each node of a shared storage cluster independently stores block change information of the node by utilizing a respective mapping file, and the block change information comprises information of a modified data block on the node; when the block incremental backup is executed, based on the block change information of each node, integrating to obtain all modified data blocks; and performing incremental backup processing according to the modified data block to realize block incremental backup. According to the block incremental backup method, only the information of the modified data block is stored through the mapping file, and the block incremental backup is performed based on the block change information, so that the problem of large disk space occupation caused by storage of all data block information is avoided, the disk space occupation is reduced, efficient storage of the block change information is realized, and the storage efficiency is improved. And meanwhile, the backup efficiency is improved, so that the overall performance is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to database technology, in particular to a block incremental backup method, product, medium and equipment based on a shared storage cluster. BACKGROUND

[0002] Computer systems inevitably have problems such as internal failures, system failures and hardware failures. These problems can cause transactions in the database to stop abnormally or cause partial data loss, so the database management system must have the ability to restore the database from the error state to a certain known correct state (also known as a consistent state or a complete state), which is called a database backup and recovery mechanism.

[0003] Backup technology is an important part of a database management system and plays a decisive role in system reliability. If a full backup is performed every time, the efficiency is not high, especially in the case of a large data set cluster, copying the entire data set cluster consumes a large amount of time and a large amount of space. Therefore, some block incremental backup schemes have appeared on the market to improve backup efficiency. There are two existing block incremental backup schemes: one is to extract the required data block by parsing the pages contained in the WAL (Write-Ahead Logging, pre-write log) log; the other is to track the page changes in real time when the database is running, and extract the changed data block during backup.

[0004] However, the above two schemes have their own advantages and disadvantages. The advantages of the first scheme are that it does not cause any performance impact on the running database, and the disadvantages are that a large amount of WAL log needs to be retained, a large amount of disk space is occupied, and parsing a large amount of WAL log takes a long time and is low in efficiency. The advantages of the second scheme are that it occupies less disk space, has high backup speed and high efficiency, and the disadvantages are that real-time tracking will bring some slight overhead to the operation of the database server, which will affect the performance. In addition, for a shared storage multi-node cluster, if scheme one is used, the disadvantages of scheme one will be amplified and the efficiency will be further reduced due to the mechanism of generating WAL logs independently by each node in the shared storage multi-node cluster, and if scheme two is used, there is a problem of occupying a large amount of disk space due to setting a storage bit for each data block. SUMMARY

[0005] In view of the above problems, a block incremental backup method, product, medium and equipment based on a shared storage cluster are proposed to overcome the above problems or at least partially solve the above problems.

[0006] An object of the present application is to improve the backup efficiency while reducing the disk space occupation.

[0007] A further object of the present application is to improve the recording accuracy of block change information, thereby improving backup reliability.

[0008] A further object of the present application is to improve the recording accuracy of block change information, thereby improving backup reliability.

[0009] In particular, the present application provides a block incremental backup method based on a shared storage cluster, which comprises:

[0010] Each node of the shared storage cluster independently stores the block change information of the node by using a respective mapping file, wherein the block change information comprises information of data blocks modified on the node;

[0011] During the execution of the block incremental backup, all modified data blocks are integrated based on the block change information of each node;

[0012] The incremental backup processing is performed according to the modified data blocks, so as to realize the block incremental backup.

[0013] Optionally, the step of each node of the shared storage cluster independently storing the block change information of the node by using a respective mapping file comprises:

[0014] Each node constructs the mapping file in the shared memory space during initialization;

[0015] Each node registers a callback function, and the callback function is used to record the information of the modified data blocks in the memory in real time;

[0016] The step of performing the disk flushing operation according to the callback function to write the information of the modified data blocks into the mapping file comprises:

[0017] Optionally, the mapping file comprises an entry array composed of a plurality of entries, and each entry in the entry array is used to store a hash value of a modified data block; and

[0018] The step of performing the disk flushing operation according to the callback function to write the information of the modified data blocks into the mapping file comprises:

[0019] The information of the modified data blocks is read from the memory in a loop, and the hash value of each modified data block is calculated;

[0020] The hash value of each modified data block is written into the entry array respectively.

[0021] Optionally, the step of writing the hash value of each modified data block into the entry array respectively comprises:

[0022] The hash value of each modified data block is taken modulo the length of the entry array, and is stored in the entry array;

[0023] Interchange two bits at a preset position in the hash value of each modified data block, take the transformed hash value modulo the length of the entry array, and store it in the entry array to improve the accuracy of the stored check value.

[0024] Optionally, based on the block change information of each node, the step of integrating all modified data blocks comprises:

[0025] In response to the broadcast signal sent by the initiating node, each active node obtains the block change information from the respective mapping file;

[0026] Each active node generates a modification information array according to the block change information, and sends the modification information array to the initiating node, wherein the modification information array is used to record information including whether each data block is modified;

[0027] The initiating node reads the block change information of the dead node, and generates a modification information array of the dead node according to the block change information of the dead node;

[0028] The initiating node integrates all modification information arrays, and takes the integrated modification information array as the return value;

[0029] The return value is parsed to obtain all modified data blocks.

[0030] Optionally, the step of generating a modification information array by each active node according to the block change information comprises:

[0031] Each active node traverses each data block in the block change information, and performs a hash operation on each data block respectively, and compares the hash value obtained by the operation with the hash value of the data block stored in the block change information;

[0032] Generate a modification information array according to the comparison result of each data block, wherein the information of the data block with consistent comparison result is modified, and the information of the data block with inconsistent comparison result is not modified.

[0033] Optionally, in the modification information array, the value of the information of the modified data block is set to 1, and the value of the information of the unmodified data block is set to 0; and

[0034] The step of integrating all modification information arrays by the initiating node comprises:

[0035] The initiating node performs a bitwise OR operation on all values in each modification information array, so that the information of the data block modified on any node in the integrated modification information array is recorded as modified.

[0036] According to another aspect of the present application, there is also provided a computer program product comprising a computer program which, when executed by a processor, implements the steps of any of the above-mentioned block incremental backup methods based on shared storage cluster.

[0037] According to yet another aspect of the present application, there is also provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of any of the above-mentioned block incremental backup methods based on shared storage cluster.

[0038] According to still another aspect of the present application, there is also provided a computer device comprising a memory, a processor and a computer program stored on the memory, the processor executing the computer program to implement the steps of any of the above-mentioned block incremental backup methods based on shared storage cluster.

[0039] The block incremental backup method based on shared storage cluster of the present application stores the block change information on each node by using the respective mapping file independently, and the block change information comprises the information of the data block modified on the node, that is, the mapping file only stores the information of the modified data block, without setting the corresponding storage bit for the unmodified data block, thereby reducing the disk space occupation. In addition, in the execution of the block incremental backup, based on the block change information of each node, all the modified data blocks are integrated and obtained, and the incremental backup processing is performed according to the modified data blocks, so as to realize the block incremental backup, which realizes the quick finding of all the modified data blocks based on the block change information on all the nodes, thereby improving the backup efficiency. Thus, the block incremental backup method of the present application stores only the information of the modified data block by using the mapping file, and performs the block incremental backup based on the block change information, thereby avoiding the problem of the large disk space occupation due to the storage of all the data block information, reducing the disk space occupation, realizing the efficient storage of the block change information, improving the backup efficiency of the block incremental backup, and thereby improving the overall performance of the shared storage cluster.

[0040] Further, the block incremental backup method based on the shared storage cluster provided by the present application provides a standardized space for the storage of the block change information by constructing a mapping file in the shared memory space at the initialization of each node. The callback function is registered by each node to record the information of the modified data block in the memory in real time, so as to realize the real-time tracking of the block change and ensure the timeliness of the record of the block change information. In addition, each node performs the disk flushing operation according to the callback function to write the information of the modified data block into the mapping file, so as to ensure the persistent storage of the block change information and the independent maintenance of the mapping file. Thus, the block incremental backup method based on the shared storage cluster provided by the present application realizes that each node independently constructs the mapping file to register the callback function, so that each node independently stores the block change information of the node itself by using the respective mapping file and independently maintains the mapping file, ensures the persistence of the block change information and the synchronization of the database state, improves the record accuracy of the block change information, thereby providing a reliable data basis for the incremental backup, and further ensuring the reliability of the backup in the fault recovery.

[0041] Further, the block incremental backup method based on the shared storage cluster provided by the present application realizes that each node independently constructs the mapping file to register the callback function, so that each node independently stores the block change information of the node itself by using the respective mapping file and independently maintains the mapping file, ensures the persistence of the block change information and the synchronization of the database state, improves the record accuracy of the block change information, thereby providing a reliable data basis for the incremental backup, and further ensuring the reliability of the backup in the fault recovery.

[0042] The above and other objects, advantages and features of the present application will become more apparent from the following detailed description of some embodiments thereof, when taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0043] Some embodiments of the present application will now be described in detail in the following text with reference to the accompanying drawings. The same reference numerals in the drawings denote the same or similar components or parts. It should be understood by those skilled in the art that the drawings are not necessarily drawn to scale. In the drawings:

[0044] Figure 1 is a schematic diagram of a block incremental backup method based on a shared storage cluster according to an embodiment of the present application;

[0045] Figure 2is a structure diagram of a mapping file in a block incremental backup method based on a shared storage cluster according to an embodiment of the present application;

[0046] Figure 3 is a control flow diagram of a block incremental backup method based on a shared storage cluster according to an embodiment of the present application;

[0047] Figure 4 is a schematic diagram of a computer program product according to an embodiment of the present application;

[0048] Figure 5 is a schematic diagram of a computer readable storage medium according to an embodiment of the present application;

[0049] Figure 6 is a schematic block diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0050] Exemplary embodiments of the present application will be described herein below with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the drawings, it is understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0051] To solve the above problems, an embodiment of the present application proposes a block incremental backup method based on a shared storage cluster. Figure 1 is a schematic diagram of a block incremental backup method based on a shared storage cluster according to an embodiment of the present application.

[0052] As shown in Figure 1 , the block incremental backup method based on a shared storage cluster of the present embodiment can generally include:

[0053] In step S102, each node of the shared storage cluster independently stores the block change information of the node by using the respective mapping file, wherein the block change information includes the information of the data block modified on the node. That is, each node in the shared storage cluster can independently store the respective block change information, and the mapping file of each node can only store the information of the data block modified on the node.

[0054] In step S104, when performing the block incremental backup, all modified data blocks are integrated based on the block change information of each node. Specifically, when performing the block incremental backup, each node interacts through the network to obtain the changed data blocks on each node, and finally the integration is performed on the backup node to obtain all the modified data blocks.

[0055] In step S106, the incremental backup processing is performed according to the modified data blocks to realize the block incremental backup. That is, for the data in the data set to be sorted, only the data meeting the condition of being added to the heap structure is copied, instead of copying all the data. Specifically, all the data blocks modified are copied to the data set cluster of the full backup to replace the original data blocks, thereby completing the incremental backup.

[0056] It should be noted that the shared storage cluster can include a shared storage database cluster, which means that one database is managed by multiple database management system instances, and each instance runs on a separate physical device node. Each node can independently receive access requests from user clients and independently read and write operations on objects and data thereon. When performing backup, the nodes can communicate through the network to synchronize data modifications, thereby ensuring data consistency between nodes.

[0057] The block incremental backup method based on the shared storage cluster in the embodiment of the application stores the block change information on each node by using the respective mapping file, and the block change information includes the information of the data blocks modified on the node, that is, the mapping file only stores the information of the modified data blocks, without setting a corresponding storage bit for the unmodified data blocks, thereby reducing the disk space occupation. In addition, the block incremental backup method based on the shared storage cluster in the embodiment of the application integrates all the modified data blocks based on the block change information of each node when performing block incremental backup, and performs incremental backup processing according to the modified data blocks to realize block incremental backup, thereby quickly finding all the modified data blocks based on the block change information on all the nodes, thereby improving the backup efficiency.

[0058] Therefore, the block incremental backup method in the embodiment of the application stores only the information of the modified data blocks by using the mapping file and performs block incremental backup based on the block change information, avoids the problem of occupying too much disk space due to storing all the data block information, reduces the disk space occupation, realizes efficient storage of the block change information, improves the backup efficiency of the block incremental backup, and thereby improves the overall performance of the shared storage cluster.

[0059] Figure 2 is a structure diagram of a mapping file in the block incremental backup method based on the shared storage cluster according to one embodiment of the embodiment of the application. As shown in Figure 2 The mapping file (which can be recorded as a map file) can store a magic code, a version number, an initial LSN, block change information, and a CRC check value.

[0060] In this embodiment, each node creates a map file independently, and the node number is used as a suffix for naming. Specifically, as shown in Figure 2As shown, the magic code, the version number and the initial LSN can be stored in the header of the map file in sequence. The length of the magic code can be four bytes, which can be used to quickly verify the file type and format of the map file. The length of the version number can be four bytes, which can be used to identify the current state or version of the map file. The length of the initial LSN can be eight bytes, which can be used to identify the backup starting point of the map file. The CRC check value can be a crc32c type check value, which is stored in the tail of the map file, and is used to check the data integrity.

[0061] As shown in Figure 2 The map file can also include an entry array (denoted as entries array) composed of a plurality of entries (denoted as entries []). It should be noted that the entries array is used to store block change information. Specifically, each entries [] in the entries array is used to store the hash value of a modified data block.

[0062] In addition, the map file is a space of a map_size custom parameter size applied in the shared memory space when each node is initialized, that is, the total length of the map file is map_size. Only the length of the array entries in the map file is not fixed, which is subject to the configuration parameter map_size of the ktrack plug-in. The main change in memory is the entries array storing the hash value, and the length of the array is related to the configuration parameter map_size. The number of bytes occupied by the entries array can be obtained by subtracting the lengths of the magic code, the initial LSN and the CRC check from the total length map_size, that is, the length of the entries array ContentNblocks.

[0063] In some embodiments, the step S102 can include the following steps: each node builds a map file in the shared memory space at initialization; each node registers a callback function, the callback function being used to record information of the modified data block in the memory in real time; and a disk flushing operation is performed according to the callback function to write the information of the modified data block into the map file. That is, in the multi-node host, each node respectively performs initialization at the start of the master process, and applies a space of a map_size self-defined parameter size in the shared memory space to store block change information at the time of initialization. In order to record the change of the data block in time, a callback function is registered when a kernel operation file (copydir, mdextend, mdwrite, ProcessSyncRequests) is operated, and the callback function records the change of the data block in the memory in real time. Thus, each node can independently record the data block change made by the node. When the checkpoint is executed, the dirty page is flushed, and the callback function of the node is registered in the ProcessSyncRequests to perform the disk flushing operation, so as to write the block change information into the file, so as to make the data block change information persistent.

[0064] The block incremental backup method based on the shared storage cluster according to the embodiment of the present application provides a standardized space for the storage of the block change information by building a map file in the shared memory space at the initialization of each node. Each node registers a callback function to record the information of the modified data block in the memory in real time, so as to realize real-time tracking of the block change and ensure the timeliness of the recording of the block change information. In addition, each node performs a disk flushing operation according to the callback function to write the information of the modified data block into the map file, so as to ensure the persistent storage of the block change information and the independent maintenance of the map file.

[0065] Thus, the block incremental backup method based on the shared storage cluster according to the embodiment of the present application registers the callback function by independently building a map file by each node, so as to realize independent storage of the block change information of the node by each node by using the respective map file and independent maintenance of the map file, ensure the persistence of the block change information and the synchronization of the database state, improve the recording accuracy of the block change information, so as to provide a reliable data basis for the incremental backup, and further ensure the reliability of the backup at the time of fault recovery.

[0066] In some embodiments, the step of performing the flushing operation according to the callback function to write the information of the modified data block into the map file can include the following steps: cyclically reading the information of the modified data block from the memory and calculating the hash value of each modified data block; and writing the hash value of each modified data block into the entries array respectively. That is, when performing the flushing operation, the information of the modified data block is written into the map file in an orderly, concise and accurate manner.

[0067] Therefore, the block incremental backup method based on the shared storage cluster according to the embodiments of the present application can greatly reduce the occupied space of the block change information by storing the block change information in the map file in the form of storing specific hash values in the entries array, thereby improving the flushing efficiency and further improving the system running efficiency, while realizing efficient storage and management of the information of the modified data block, facilitating subsequent fast query and comparison, and further improving the system performance of the shared storage cluster.

[0068] In some embodiments, the step of writing the hash value of each modified data block into the entries array can include the following steps: taking the hash value of each modified data block modulo the length of the entries array and storing it into the entries array; and exchanging two bits at a preset position in the hash value of each modified data block, taking the transformed hash value modulo the length of the entries array, and storing it into the entries array to improve the accuracy of the check value storage. It should be noted that the two bits at the preset position can be selected and set according to requirements, for example, the low 32 bits and the high 32 bits of the hash value.

[0069] For example, when a certain data block of a certain table file is changed at LSN1, the information is recorded into the entries array. Specifically, first, the relnode (spcNode, dbNode, relNode), forknum and blocknum structure is hashed, the hash value is taken modulo the length ContentNblocks of the entries array, and the entries [] stores the LSN1 value. At the same time, the low 32 bits and the high 32 bits of the hash value are exchanged, and the entries [] is taken again to store the LSN1 value, that is, the LSN1 value of the modified data block is recorded in two places entries [], thereby improving the storage accuracy.

[0070] Therefore, the block incremental backup method based on the shared storage cluster according to the embodiments of the present application can improve the accuracy of the check value storage by the double hashing mechanism, effectively reduce the hash collision probability, and reduce the misjudgment rate of the data block modification state, thereby ensuring the accuracy of the block change information record, and further providing a guarantee for efficient and safe execution of the block incremental backup.

[0071] In some embodiments, the step of integrating all modified data blocks based on the block change information of each node in step S104 can include the following steps: each active node obtains the block change information from the respective map file in response to the broadcast signal sent by the initiating node; each active node generates a modification information array according to the block change information and sends the modification information array to the initiating node, wherein the modification information array is used to record information including whether each data block is modified; the initiating node reads the block change information of the dead node and generates a modification information array of the dead node according to the block change information of the dead node; the initiating node integrates based on all modification information arrays and sends the integrated modification information array as a return value; and the return value is parsed to obtain all modified data blocks.

[0072] That is, when performing block incremental backup, the initiating node first sends a broadcast to each active node, each active node identifies whether each data block thereon is modified according to the respective block change information, and sends a modification information array to the initiating node, and then the initiating node reads the map file of the dead node to obtain the modification information array of the dead node, and integrates the modification information arrays of all nodes to obtain all modified data blocks.

[0073] Thus, the block incremental backup method based on a shared storage cluster according to the embodiments of the present application ensures that no interaction occurs between nodes during the identification process by having each active node independently identify and sort information of whether each data block thereon is modified and sending the information to the initiating node for integration after the identification, thereby avoiding database overhead caused by real-time tracking and improving system performance. In addition, the initiating node comprehensively obtains the modification information arrays of the active nodes and the dead nodes, efficiently integrates the block change information of all nodes in the cluster during the backup process, ensures the comprehensiveness and accuracy of the incremental backup, and thus improves the fault tolerance of the system and the reliability of the backup.

[0074] In some embodiments, the step of each active node generating a modification information array according to the block change information can include the following steps: each active node iterates through each data block therein, respectively performs a hash operation on each data block, and compares the hash value obtained by the operation with the hash value of the data block stored in the block change information; and a modification information array is generated according to the comparison result of each data block, wherein the information of a data block with a consistent comparison result is modified, and the information of a data block with an inconsistent comparison result is not modified.

[0075] Specifically, after each active node receives the broadcast signal, the global, base, tblspc directory and subdirectory are scanned to obtain a table file list, and then each data block of each table file is traversed to perform hashing respectively, and the corresponding lsn is obtained from the hash table entries in the memory, the two lsn values are compared to obtain the information whether the data block is modified, and finally the information whether each data block is modified is arranged in turn to obtain a modified information array.

[0076] Therefore, the block incremental backup method based on the shared storage cluster provided by the embodiment of the application realizes accurate judgment of the modification state of each data block by traversing the data block and performing hashing operation by each active node, and comparing the hash value in the map file, generates a modified information array according to the comparison result, records the modification state of each data block in a standardized format in order, improves the identification accuracy of the data block modification state, thereby providing an accurate modified data list for subsequent incremental backup, and further ensures the correctness of the backup data.

[0077] In some embodiments, the modified information array can be a binary array, denoted as bitmap, and each bit (0 or 1) represents whether a data block is modified. Specifically, in the bitmap, the value of the information of the modified data block is set to 1, and the value of the information of the unmodified data block is set to 0.

[0078] Based on this, the step of integrating all the bitmaps by the initiating node can include the following steps: the initiating node performs a bitwise OR operation on all the values in each bitmap, so that the information of the data block modified on any node is recorded as modified in the integrated bitmap. For example, the value of a data block in the bitmap of one node is 1, and the value of the data block in the bitmap of other nodes is 0, after the bitwise OR operation, the information of the data block is still recorded as 1 in the integrated bitmap, thereby retaining the information of the data block modified on all nodes.

[0079] Therefore, the block incremental backup method based on the shared storage cluster provided by the embodiment of the application realizes accurate judgment of the modification state of each data block by traversing the data block and performing hashing operation by each active node, and comparing the hash value in the map file, generates a modified information array according to the comparison result, records the modification state of each data block in a standardized format in order, improves the identification accuracy of the data block modification state, thereby providing an accurate modified data list for subsequent incremental backup, and further ensures the correctness of the backup data.

[0080] In one embodiment, the present application divides the block incremental backup process in a shared storage cluster into two main steps: one is to store block change information in time; the other is to select blocks that need to be backed up when backup is needed. In order to reduce the impact on performance, a map file is independently built by each node in the shared memory cluster, and a callback function is independently registered in the operating file place, so as to independently maintain the map file, and the map files between nodes do not interact. When performing block incremental backup, each node interacts through the network to obtain the changed data blocks on each node, and finally integrates on the backup node. The following is a disassembly of the backup process:

[0081] 1. Store block change information:

[0082] (1) Initialization is performed on each node:

[0083] In a multi-node host, each node independently performs initialization, and the initialization is performed when the master process is started. When initializing, a map file is constructed in the shared memory space with a space of map_size self-defined parameter size to store the lsn of the block change.

[0084] (2) Register hook function on each node:

[0085] In order to record the change of the block in time, a callback function is registered when the kernel operates the file (copydir, mdextend, mdwrite, ProcessSyncRequests), which will record the change of the data block in the memory in real time. Each node independently records the data block change of the node.

[0086] (3) Each node persists data block change information:

[0087] When performing checkpoint, the dirty page is flushed, and the callback function of the registered ProcessSyncRequests is used to perform the flushing operation: the main operation is to read the elements of entries from the memory and store them in a temporary buf, and when the buf stores a certain amount, it is written into the map file at one time, and all the contents in the memory are written into the map file. Each node creates a map file independently, and the node number is used as the suffix name.

[0088] 2. Obtain block change information:

[0089] (1) When performing block incremental backup, the sys_get_pagemapset() function is called on the initiating node backup, and the parameter is the lsn of each node, i.e. the startpoint of each node of the last full backup.

[0090] (2) The starting node sends a broadcast to each active node to obtain the changed data block information in the form of a bitmap.

[0091] (3) After each node receives the broadcast signal, the global, base, and tblspc directories and subdirectories are scanned to obtain the table file list, and then each block of each table file is traversed to perform hashing, and the corresponding lsn is obtained from the hash table entries of the memory file, and the two lsn values are compared to set the mapset to generate the respective bitmap.

[0092] (4) Each node sends the obtained bitmap to the starting node through network communication.

[0093] (5) After the starting node receives the bitmap of all active nodes, the map file of the dead node is read, and the bitmap is obtained in the same way.

[0094] (6) The starting node performs a bitwise OR operation on all values in the bitmap of all nodes, and finally takes the calculated bitmap as the return value.

[0095] (7) The returned bitmap value is parsed, and a value of 0 corresponds to no modification, and a value of 1 corresponds to a modified block. All modified blocks are copied to the full backup data set cluster to replace the original blocks. Thus, the shared storage cluster completes the incremental backup.

[0096] Using the above method, the shared storage cluster stores only the modified data block information of each node using the respective map file, and performs block incremental backup based on the block change information, avoiding the problem of occupying a large amount of disk space due to storing all data block information, reducing the disk space occupation, achieving efficient storage of block change information, and improving the backup efficiency of block incremental backup, thereby improving the overall performance of the shared storage cluster.

[0097] Figure 3 is a control flow diagram of a block incremental backup method based on a shared storage cluster according to an embodiment of the present application. The following will be described in combination with Figure 3 The flow steps of the present embodiment will be described in detail.

[0098] In step S302, each node constructs a map file in the shared memory space during initialization. It should be noted that in the shared storage cluster, each node creates a map file independently, and the node number is used as a suffix for naming.

[0099] In step S304, each node registers a callback function, which is used to record the modified data block information in real time.

[0100] Step S306, the information of the modified data block is read from the memory cyclically, and the hash value of each modified data block is calculated.

[0101] Step S308, the hash value of each modified data block is written into the entries array of the map file respectively.

[0102] Step S310, in response to the broadcast signal sent by the initiating node, each active node acquires the block change information from the respective map file.

[0103] Step S312, each active node traverses each data block acquired therein, and respectively performs hash operation on each data block, and compares the hash value obtained by the operation with the hash value of the data block stored in the block change information. It should be noted that the information of the data block with consistent comparison result is modified, and the information of the data block with inconsistent comparison result is not modified.

[0104] Step S314, the bitmap is generated according to the comparison result of each data block, and the bitmap is sent to the initiating node. It should be noted that in the bitmap, the value of the information of the modified data block is set to 1, and the value of the information of the unmodified data block is set to 0.

[0105] Step S316, the initiating node reads the map file of the dead node, and generates the bitmap of the dead node.

[0106] Step S318, the initiating node performs bitwise OR operation on all values in the bitmaps, so that the information of the data block modified on any node is recorded as modified in the integrated bitmap.

[0107] Step S320, the return value is parsed to obtain all modified data blocks.

[0108] Step S322, the incremental backup processing is performed according to the modified data block, so as to realize the block incremental backup. Thus far, the execution process of the block incremental backup is completed, and the current process is ended.

[0109] Therefore, the block incremental backup method of the embodiment of the present application stores only the information of the modified data block through the map file, and performs the block incremental backup based on the block change information, thereby avoiding the problem of occupying much disk space due to storing all data block information, reducing the disk space occupation, realizing the efficient storage of the block change information, improving the backup efficiency of the block incremental backup, and thus improving the overall performance of the shared storage cluster.

[0110] Further, the block incremental backup method based on the shared storage cluster according to the embodiment of the present application registers the callback function by constructing the map file independently on each node, realizes that each node independently stores the block change information of the node and independently maintains the map file by using the respective map file, ensures the persistence of the block change information and the synchronization of the database state, improves the recording accuracy of the block change information, thereby providing a reliable data basis for the incremental backup, and further guarantees the reliability of the backup during the fault recovery.

[0111] Further, the block incremental backup method based on the shared storage cluster according to the embodiment of the present application stores the block change information in the map file by using the entries array to store the specific hash value, greatly reduces the occupied space of the block change information, thereby improving the disk flushing efficiency, further improving the system running efficiency, simultaneously realizes the efficient storage and management of the information of the modified data block, facilitates the subsequent fast query and comparison, and further improves the system performance of the shared storage cluster.

[0112] The embodiment also provides a computer program product 10, a computer readable storage medium 20, and a computer device 30. Figure 4 Fig. 1 is a schematic diagram of the computer program product 10 according to an embodiment of the present application, Figure 5 Fig. 2 is a schematic diagram of the computer readable storage medium 20 according to an embodiment of the present application, Figure 6 Fig. 3 is a schematic block diagram of the computer device 30 according to an embodiment of the present application.

[0113] The computer program product 10 comprises a computer program 11, which, when executed by the processor 32, implements the steps of any of the energy storage scheduling control methods of the energy storage system described above. The computer readable storage medium 20 has the computer program 11 stored thereon, and the computer program 11, when executed by the processor 32, implements the steps of any of the energy storage scheduling control methods of the energy storage system described above. The computer device 30 can comprise a memory 31, a processor 32, and a computer program 11 stored on the memory 31 and running on the processor 32.

[0114] The computer program 11 for performing the operation of the present application can be assembly instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine related instructions, microcode, firmware instructions, state setting data, configuration data of integrated circuits, or source code or object code written in any combination of one or more programming languages and process programming languages.

[0115] The computer program 11 can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate array (FPGA), or programmable logic array (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0116] For the description of this embodiment, the computer program product 10 is a product related to the computer program 11.

[0117] For the description of this embodiment, the computer readable storage medium 20 is a tangible device that can retain and store the computer program 11, which can be any apparatus that can contain, store, communicate, propagate or transport the program 811 for use by or in connection with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer readable storage medium 20 include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, and any suitable combination of the above.

[0118] Up to now, those skilled in the art should recognize that, although the present application has been shown and described in detail in the above embodiments, many other variations or modifications can be directly determined or deduced according to the disclosure of the present application without departing from the spirit and scope of the present application, which conform to the principles of the present application. Therefore, the scope of the present application should be understood and recognized as covering all these other variations or modifications.

Claims

1. A block incremental backup method based on a shared storage cluster, comprising: Each node in the shared storage cluster independently stores the block change information of the node using its own mapping file, wherein the block change information includes information about the modified data blocks on the node; When performing incremental block backup, all modified data blocks are integrated based on the block change information of each node. Incremental backup is performed based on the modified data blocks to achieve block incremental backup.

2. The block incremental backup method based on a shared storage cluster according to claim 1, wherein, The steps of each node in the shared storage cluster independently storing the node's block change information using its own mapping file include: During initialization, each node constructs the mapping file in a shared memory space; Each node registers a callback function, which is used to record information about the modified data block into memory in real time; Perform a disk flushing operation according to the callback function to write the information of the modified data block into the mapping file.

3. The block incremental backup method based on a shared storage cluster according to claim 2, wherein, The mapping file includes an array of entries consisting of multiple entries, each entry in the array being used to store the hash value of a modified data block; and The step of performing a disk flushing operation according to the callback function to write the information of the modified data block into the mapping file includes: Read information about the modified data blocks from memory in a loop and calculate the hash value of each modified data block; Write the hash value of each modified data block into the entry array.

4. The block incremental backup method based on a shared storage cluster according to claim 3, wherein, The step of writing the hash value of each modified data block into the entry array includes: The hash value of each modified data block is modulo the length of the entry array and stored in the entry array; The two bits at preset positions in the hash value of each modified data block are swapped, the transformed hash value is modulo the length of the entry array, and then stored in the entry array to improve the accuracy of the verification value storage.

5. The block incremental backup method based on a shared storage cluster according to claim 3, wherein, The step of integrating all modified data blocks based on the block change information of each node includes: In response to the broadcast signal sent by the initiating node, each active node obtains the block change information from its respective mapping file; Each active node generates a modification information array based on the block change information and sends the modification information array to the initiating node, wherein the modification information array is used to record information including whether each data block has been modified; The initiating node reads the block change information of the dead node and generates a modification information array of the dead node based on the block change information of the dead node; The initiating node integrates all the modified information arrays and returns the integrated modified information array as the return value. The return value is parsed to obtain all the modified data blocks.

6. The block incremental backup method based on a shared storage cluster according to claim 5, wherein, The step of each active node generating a modification information array based on the block change information includes: Each active node traverses and obtains each data block, performs a hash operation on each data block, and compares the hash value obtained with the hash value of the data block stored in the block change information. The modification information array is generated based on the comparison results of each data block, wherein the information of data blocks with consistent comparison results is modified, and the information of data blocks with inconsistent comparison results is not modified.

7. The block incremental backup method based on a shared storage cluster according to claim 5, wherein, In the modified information array, the value of the information of the modified data block is set to 1, and the value of the information of the unmodified data block is set to 0; and The step of the initiating node integrating all the modified information arrays includes: The initiating node performs a bitwise OR operation on all values ​​in each of the modification information arrays, so that the information of the data block modified at any node in the integrated modification information array is recorded as modified.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the block incremental backup method based on a shared storage cluster as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that... When the computer program is executed by the processor, it implements the steps of the block incremental backup method based on a shared storage cluster as described in any one of claims 1 to 7.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the block incremental backup method based on a shared storage cluster as described in any one of claims 1 to 7.

Images