Metadata synchronization method, device and storage system
The status bitmap manages metadata synchronization status, allowing data read and write commands to continue to be processed during the synchronization process, solving the business interruption caused by metadata synchronization and improving the user experience of the storage system.
Patent Information
- Application Number
- CN202411147478.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-21
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2044-08-21
AI Technical Summary
In the storage system, data reading and writing need to be kept silent during the metadata synchronization process, resulting in business interruption and affecting the user experience.
By determining the synchronization status bitmap of the metadata to be updated, data read and write commands are executed only when the synchronization state allows, and metadata is updated during the synchronization process, avoiding synchronization of all data at once.
It realizes metadata synchronization and data reading and writing simultaneously, avoids business interruptions and improves the user experience of the storage system.
Smart Images

Figure CN118656038B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a metadata synchronization method, device and storage system. Background Art
[0002] In the storage system, the remote replication function needs to replicate between different volumes, and metadata is needed to manage the replication progress of the remote replication and the remote replication commands being processed. The metadata used for remote replication needs to be stored on two nodes at the same time. When one node is offline, the metadata stored on the node cannot be used. At this time, the metadata stored on another node can be obtained to continue to be used for remote replication, so that the replication of the two volumes between the two remote replication sites will not be interrupted, and the commands before the failure can continue to be processed.
[0003] In the related technology, when a new node joins the cluster, it is necessary to synchronize the metadata of the original node to the new node without modifying the metadata. When a node joins, it is necessary to first silence the data reading and writing to ensure that the metadata is not modified, and then start synchronizing the metadata from the original node to the newly added node. After the synchronization is completed, resume the normal processing of data reading and writing. In the case of a large amount of metadata, the silence time of data reading and writing is long, which will cause data reading and writing to be interrupted, and the business in the storage system will be interrupted, affecting the user experience of the storage system.
[0004] Therefore, how to avoid data reading and writing silence during metadata synchronization and improve the user experience of the storage system has become a technical problem that the industry needs to solve urgently. Summary of the invention
[0005] The present application provides a metadata synchronization method, device and storage system, which are used to solve the technical problem of how to avoid data reading and writing silence during metadata synchronization and improve the user experience of the storage system.
[0006] The present application provides a metadata synchronization method, comprising:
[0007] Determine the metadata to be updated corresponding to the current data read and write command;
[0008] Determine the status bit corresponding to the metadata to be updated in the synchronization bitmap; the synchronization bitmap is used to indicate the synchronization status of each metadata in the first node in the second node; the first node synchronizes each metadata to the second node in units of metadata corresponding to the status bit;
[0009] In a case where it is determined based on the status bit that the metadata to be updated has been synchronized, executing the current data read and write command;
[0010] Based on the execution result of the current data read and write command, the metadata to be updated is updated in the first node and the second node.
[0011] In some embodiments, the method further comprises:
[0012] In the case where it is determined that the second node is online, determining the synchronization bitmap based on the synchronization status of each metadata in the first node;
[0013] Taking the metadata corresponding to the status bits in the synchronization bitmap as a unit, synchronizing the metadata corresponding to each status bit to the second node;
[0014] The status bits corresponding to the synchronized metadata are adjusted in the synchronization bitmap.
[0015] In some embodiments, determining the synchronization bitmap based on the synchronization status of each metadata in the first node includes:
[0016] Divide each metadata in the first node into blocks to obtain multiple metadata blocks;
[0017] Determining the synchronization bitmap based on the storage location and synchronization status of each metadata block;
[0018] Among them, the status bits in the synchronization bitmap correspond one-to-one to the multiple metadata blocks.
[0019] In some embodiments, determining the synchronization bitmap based on the storage location and synchronization status of each metadata block includes:
[0020] Determine, based on the storage location of each metadata block, the number of the status bit corresponding to each metadata block in the synchronization bitmap;
[0021] Based on the synchronization status of each metadata block, the value of the status bit corresponding to each metadata block in the synchronization bitmap is determined.
[0022] In some embodiments, when the value of the status bit corresponding to any metadata block in the synchronization bitmap is a first preset value, the synchronization state of any metadata block is synchronized; when the value of the status bit corresponding to any metadata block in the synchronization bitmap is a second preset value, the synchronization state of any metadata block is unsynchronized.
[0023] In some embodiments, the step of synchronizing metadata corresponding to each status bit in the synchronization bitmap to the second node in units of metadata corresponding to the status bit, includes:
[0024] Get the value of the current status bit;
[0025] When the value of the current state bit is a second preset value, a synchronization lock is allocated to the current state bit; the synchronization lock is used to control the number of state bits for a single synchronization;
[0026] Determining a current metadata block to be synchronized based on the synchronization lock and the current state bit;
[0027] The metadata block currently to be synchronized is synchronized from the first node to the second node.
[0028] In some embodiments, synchronizing the metadata block to be synchronized from the first node to the second node includes:
[0029] Determining a first node communication message based on the current metadata block to be synchronized;
[0030] Sending the first node communication message to the second node, so that the second node stores the metadata block to be synchronized and generates a second node communication message;
[0031] receiving a second node communication message sent by the second node;
[0032] Based on the second node communication message, it is determined that the metadata block currently to be synchronized has been synchronized.
[0033] In some embodiments, the first node communication message and the second node communication message include metadata synchronization position information and metadata synchronization length information.
[0034] In some embodiments, after synchronizing the metadata block to be synchronized from the first node to the second node, the method further includes:
[0035] Adjusting the value of the current state bit in the synchronization bitmap from the second preset value to the first preset value;
[0036] The synchronization lock is assigned to the next state bit.
[0037] In some embodiments, after obtaining the value of the current state bit, the method further includes:
[0038] When the value of the current state bit is a first preset value, skipping the current state bit;
[0039] Get the value of the next status bit.
[0040] In some embodiments, determining that the metadata to be updated is synchronized based on the status bit includes:
[0041] When it is determined based on the status bit that the metadata to be updated is not synchronized, the execution of the current data read and write command is suspended, and the synchronization of the metadata to be updated is waited until the synchronization of the metadata to be updated is completed.
[0042] The present application provides a metadata synchronization device, comprising:
[0043] A metadata determination unit, used to determine the metadata to be updated corresponding to the current data read and write command;
[0044] A metadata synchronization unit, used to determine the status bit corresponding to the metadata to be updated in a synchronization bitmap; the synchronization bitmap is used to indicate the synchronization status of each metadata in the first node in the second node; the first node synchronizes each metadata to the second node in units of metadata corresponding to the status bit;
[0045] A command execution unit, configured to execute the current data read and write command when it is determined based on the status bit that the metadata to be updated has been synchronized;
[0046] A metadata updating unit is used to update the metadata to be updated in the first node and the second node based on the execution result of the current data read and write command.
[0047] The present application provides a storage system, comprising a first node and a second node;
[0048] The first node is used to execute the metadata synchronization method when detecting that the second node is online.
[0049] The present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the metadata synchronization method when executing the computer program.
[0050] The present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, and the computer program implements the metadata synchronization method when executed by a processor.
[0051] The metadata synchronization method, device and storage system provided by the present application determine the metadata to be updated corresponding to the current data read and write command; determine the status bit corresponding to the metadata to be updated in the synchronization bitmap; the synchronization bitmap is used to indicate the synchronization status of each metadata in the first node in the second node; execute the current data read and write command when it is determined based on the status bit that the metadata to be updated has been synchronized; update the metadata to be updated in the first node and the second node based on the execution result of the current data read and write command; since the first node synchronizes each metadata to the second node in units of metadata corresponding to the status bit, there is no need to synchronize all metadata to the second node at one time, and there is no need to keep the first node silent in data reading and writing; since the first node can continue to process the current data read and write command and update the metadata according to the synchronization status during the metadata synchronization process, metadata synchronization and data reading and writing are carried out simultaneously, thus avoiding business interruption in the storage system and improving the user experience of the storage system. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0053] In order to more clearly illustrate the technical solutions in the present application or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0054] Figure 1 It is a flowchart of the metadata synchronization method provided by this application.
[0055] Figure 2 It is a schematic diagram of the synchronization bitmap provided by this application.
[0056] Figure 3 It is a structural diagram of the metadata synchronization device provided by this application.
[0057] Figure 4 It is a structural diagram of the storage system provided by this application.
[0058] Figure 5 It is a structural schematic diagram of the electronic device provided by this application. DETAILED DESCRIPTION
[0059] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.
[0060] It should be noted that the terms "first", "second" etc. in the present application are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable in appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, the process, method, system, product or equipment comprising a series of steps or units need not be limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.
[0061] Remote replication in storage systems is a data protection and disaster recovery strategy that ensures data reliability, availability, and persistence by replicating data at a physical or logical location away from the primary data storage device. Remote replication is often used to prevent data loss or service interruption caused by a single point of failure or natural disaster by replicating data from one storage system to another.
[0062] The remote replication function requires metadata to manage the replication progress of remote replication and the data read and write commands (also called IO commands) being processed, mark whether the contents of each location in the volume space have been copied to the remote cluster, and record the main metadata of the remote replication command being processed, such as the volume identifier (ID), the data length corresponding to the data read and write, and other information, to continue data reading and writing after failure recovery.
[0063] Metadata refers to data that describes data. It provides detailed information about the data and helps understand, manage, discover and use the data. Metadata itself is not data, but a description of data about data. Remote replication functions usually involve multiple types of metadata, which can help manage and control the process and effect of remote replication. For example, it can include configuration information describing remote replication, such as the identification of source and target devices, replication strategy (synchronous or asynchronous), replication frequency, etc. Another example is recording the current status information of remote replication tasks, including replication progress, the time of the last replication operation, whether the replication is active, etc. In remote replication, data volumes can also be divided into multiple data blocks. By using a bitmap method to record the status of all data blocks, each data block corresponds to one bit.
[0064] Generally, two nodes need to be configured in the storage system to manage and control the reading and writing of data in the storage system. Metadata and bitmaps need to be stored on two nodes and backed up each other to ensure that the normal operation of remote replication services is not affected when a node leaves. When one of the nodes is offline due to a failure or other reasons and a new node is online, the other node needs to synchronize its own stored metadata and bitmap to the newly added node and rebuild the backup relationship. Data read and write silence (IO silence) refers to completing the existing data read and write operations on the data volume and no longer processing new data read and write operations issued by the upper layer. The metadata synchronization method in the related technology requires the node to keep data read and write silent, which causes business interruptions in the storage system and affects the user experience of the storage system.
[0065] In order to solve the shortcomings of related technologies, Figure 1 is a flowchart of the metadata synchronization method provided by this application, such as Figure 1 As shown, the method includes step 110 , step 120 , step 130 and step 140 .
[0066] Step 110: Determine the metadata to be updated corresponding to the current data read / write command.
[0067] Specifically, the execution subject of the metadata synchronization method provided in the embodiment of the present application is a metadata synchronization device. The device can be implemented by software, such as a metadata synchronization program running in a storage system; or it can be a device that executes the metadata synchronization method, such as a desktop computer or server in a storage system.
[0068] The storage system includes at least two nodes. The nodes can be servers or computers in the storage system, etc., which can share data volumes, manage and control the reading and writing of data in the storage system, and are used to execute data reading and writing operations corresponding to relevant commands. The two nodes back up each other and both store metadata corresponding to the remote replication function.
[0069] The application scenario of the method provided in the embodiment of the present application is that any one of the two nodes in the storage system is offline due to a failure or other reasons, and a new node is online. Offline means disconnecting from the storage system; online means establishing a connection with the storage system. The first node refers to a node that is not offline, and the second node refers to a newly online node. In order for the second node to be able to manage and control the reading and writing of data in the storage system after it goes online, the first node needs to synchronize metadata to the second node.
[0070] When a faulty node is offline and a second node is online, the first node continues to receive and process various data processing commands, such as remote replication commands or background replication commands, to provide data service to users.
[0071] After receiving the current data read / write command, the first node can determine whether the metadata needs to be updated according to the specific content of the current data read / write command. The metadata to be updated refers to the metadata that needs to be updated.
[0072] Taking the current data read and write command as a remote replication command as an example, the metadata to be updated may include replication progress, the last replication operation time, and whether the replication is active.
[0073] Step 120: Determine the status bit corresponding to the metadata to be updated in the synchronization bitmap; the synchronization bitmap is used to indicate the synchronization status of each metadata in the first node in the second node. The first node synchronizes each metadata to the second node in units of metadata corresponding to the status bit.
[0074] Specifically, the first node may create a synchronization bitmap (syncbitmap). The synchronization bitmap includes multiple bits, which are used to indicate the synchronization status of each metadata in the first node in the second node, and therefore may also be referred to as a status bit. The value of the bit may be used to indicate the synchronization status, for example, 1 may indicate unsynchronized, and 0 may indicate synchronized.
[0075] A one-to-one correspondence between metadata and status bits may be established. After determining the metadata to be updated, the status bits corresponding to the metadata to be updated in the synchronization bitmap may be determined.
[0076] The unit is used to represent the amount of data that needs to be transmitted each time the first node is synchronized. After the second node goes online, the first node can synchronize the metadata corresponding to each status bit to the second node in sequence, taking the data amount of the metadata corresponding to each status bit as the unit.
[0077] Step 130: When it is determined based on the status bit that the metadata to be updated has been synchronized, execute the current data read and write command.
[0078] Specifically, the synchronization status of the metadata to be updated can be determined according to the value of the status bit. For example, a value of 0 indicates that the metadata to be updated is synchronized, and a value of 1 indicates that the metadata to be updated is not synchronized. After the first node determines that the metadata to be updated has been synchronized to the second node, the current data read and write command can be executed.
[0079] Step 140: Based on the execution result of the current data read and write command, update the metadata to be updated in the first node and the second node.
[0080] Specifically, the first node determines the specific content to be updated according to the execution result of the current data read and write command, thereby updating the metadata to be updated in the first node. In addition, since the metadata to be updated has also been synchronized to the second node, there is a backup in the second node. The first node sends a communication message to the second node, so that the second node updates the stored metadata to be updated.
[0081] The metadata synchronization method provided in the embodiment of the present application determines the metadata to be updated corresponding to the current data read and write command; determines the status bit corresponding to the metadata to be updated in the synchronization bitmap; the synchronization bitmap is used to indicate the synchronization status of each metadata in the first node in the second node; executes the current data read and write command when it is determined based on the status bit that the metadata to be updated has been synchronized; updates the metadata to be updated in the first node and the second node based on the execution result of the current data read and write command; since the first node synchronizes each metadata to the second node in units of metadata corresponding to the status bit, there is no need to synchronize all metadata to the second node at one time, and there is no need to keep the first node silent in data reading and writing; since the first node can continue to process the current data read and write command and update the metadata according to the synchronization status during the metadata synchronization process, metadata synchronization and data reading and writing are carried out simultaneously, thereby avoiding business interruption in the storage system and improving the user experience of the storage system.
[0082] It should be noted that each implementation method of the present application can be freely combined, the order can be changed, or it can be executed separately, and does not need to rely on or depend on a fixed execution order.
[0083] In some embodiments, the method further comprises:
[0084] When it is determined that the second node is online, determining a synchronization bitmap based on the synchronization status of each metadata in the first node;
[0085] Taking the metadata corresponding to the status bit in the synchronization bitmap as a unit, synchronizing the metadata corresponding to each status bit to the second node;
[0086] The status bits corresponding to the synchronized metadata are adjusted in the synchronization bitmap.
[0087] Specifically, when the second node is determined to be online, a memory space can be applied in the first node to create a synchronization bitmap (syncbitmap). Since the synchronization status of each metadata is not synchronized, the value of each status bit in the synchronization bitmap can be set to the value corresponding to the unsynchronized state, for example, 1.
[0088] The first node may synchronize the metadata corresponding to each status bit to the second node in units of metadata corresponding to the status bit in the synchronization bitmap. For example, the metadata corresponding to one status bit may be used as a unit, and the metadata corresponding to each status bit may be synchronized to the second node in sequence according to the numbering order of the status bits. Each time after metadata synchronization is completed, the status bit corresponding to the synchronized metadata may be adjusted in the synchronization bitmap, for example, the value of the status bit may be adjusted from 1 to 0, indicating that the metadata corresponding to the status bit has been synchronized.
[0089] The metadata synchronization method provided in the embodiment of the present application uses the metadata corresponding to the status bit in the synchronization bitmap as a unit to synchronize the metadata corresponding to each status bit to the second node. There is no need to synchronize all the metadata to the second node at one time, and there is no need to keep the first node silent in data reading and writing, thereby avoiding interruption of services in the storage system.
[0090] In some embodiments, determining a synchronization bitmap based on the synchronization status of each metadata in the first node includes:
[0091] Divide each metadata in the first node into blocks to obtain multiple metadata blocks;
[0092] Determining a synchronization bitmap based on the storage location and synchronization status of each metadata block;
[0093] Among them, the status bits in the synchronization bitmap correspond one-to-one to multiple metadata blocks.
[0094] Specifically, each metadata in the first node may be divided into blocks to obtain a plurality of metadata blocks. The data volume of each data block is consistent, for example, 32K (KB, kilobytes). Each status bit in the synchronization bitmap may correspond to a plurality of metadata blocks one by one, so that one status bit represents the synchronization status of one metadata block. The number of the status bit may be determined according to the storage location of the metadata block, and the value of the status bit may be determined according to the synchronization status of the metadata block, thereby determining the synchronization bitmap.
[0095] When each metadata is divided into blocks, the size (data volume) of the metadata block may be determined according to the number of data read and write commands executed by the first node and / or the communication bandwidth between nodes in the storage system.
[0096] For example, when the number of data read and write commands executed by the first node is greater than the preset number, it can be considered that the first node executes more tasks. In order to avoid affecting business services, the data volume of the metadata block can be set to a smaller value; when the number of data read and write commands executed by the first node is less than the preset number, it can be considered that the first node executes fewer tasks. In order to improve the synchronization efficiency of metadata, the data volume of the metadata block can be set to a larger value.
[0097] For example, when the communication bandwidth between nodes in the storage system is greater than the preset bandwidth, it can be considered that the data transmission between the nodes is good. In order to improve the synchronization efficiency of metadata, the data volume of the metadata block can be set to a larger value; when the communication bandwidth between nodes in the storage system is less than the preset bandwidth, it can be considered that the data transmission between the nodes is poor. In order to avoid affecting business services, the data volume of the metadata block can be set to a smaller value.
[0098] The metadata synchronization method provided in the embodiment of the present application divides each metadata in the first node into blocks to obtain multiple metadata blocks, and determines the synchronization bitmap based on the storage location and synchronization status of each metadata block. The metadata can be effectively managed and can be controlled by the size of the metadata block, thereby improving the synchronization efficiency of the metadata, avoiding affecting the data read and write commands being executed in the storage system, avoiding interruption of services in the storage system, and improving the user experience of the storage system.
[0099] In some embodiments, determining a synchronization bitmap based on the storage location and synchronization status of each metadata block includes:
[0100] Based on the storage location of each metadata block, determine the number of the status bit corresponding to each metadata block in the synchronization bitmap;
[0101] Based on the synchronization status of each metadata block, the value of the status bit corresponding to each metadata block in the synchronization bitmap is determined.
[0102] Specifically, the storage location refers to the starting location and the ending location of metadata storage. Figure 2 is a schematic diagram of the synchronization bitmap provided by this application, such as Figure 2 As shown, the metadata corresponding to the remote replication can be divided into three metadata blocks. The data volume of each metadata block is 32K. The storage location of the first metadata block can be expressed as [0, 32K-1], and the storage location of the second metadata block can be expressed as [32K, 32K 2-1], the storage location of the third metadata block can be expressed as [32K 2,32K 3-1]. The status bit corresponding to the first metadata block is Bit0 (status bit number is 0), the status bit corresponding to the second metadata block is Bit1 (status bit number is 1), and the status bit corresponding to the third metadata block is Bit2 (status bit number is 2). Since these metadata blocks are not synchronized, these status bits can all be set to 1.
[0103] The metadata synchronization method provided in the embodiment of the present application determines the number of the status bit according to the storage location of the metadata block, and determines the value of the status bit according to the synchronization status of the metadata block, so that the synchronization bitmap can accurately represent the synchronization status of the metadata.
[0104] In some embodiments, when the value of the status bit corresponding to any metadata block in the synchronization bitmap is a first preset value, the synchronization state of any metadata block is synchronized; when the value of the status bit corresponding to any metadata block in the synchronization bitmap is a second preset value, the synchronization state of any metadata block is unsynchronized.
[0105] Specifically, the first preset value may be 0, and the second preset value may be 1. When the value of the status bit corresponding to any metadata block in the synchronization bitmap is 0, it may indicate that the synchronization state of any metadata block is synchronized; when the value of the status bit corresponding to any metadata block in the synchronization bitmap is 1, it may indicate that the synchronization state of any metadata block is not synchronized.
[0106] The metadata synchronization method provided in the embodiment of the present application represents the synchronization status of the metadata block by a first preset value and a second preset value, and can conveniently determine the synchronization status of the metadata block by the value, thereby improving the synchronization efficiency of the metadata.
[0107] In some embodiments, taking metadata corresponding to the status bits in the synchronization bitmap as a unit, synchronizing metadata corresponding to each status bit to the second node includes:
[0108] Get the value of the current status bit;
[0109] When the value of the current state bit is the second preset value, a synchronization lock is allocated to the current state bit; the synchronization lock is used to control the number of state bits for a single synchronization;
[0110] Based on the synchronization lock and the current state bit, determine the current metadata block to be synchronized;
[0111] Synchronize the metadata block currently to be synchronized from the first node to the second node.
[0112] Specifically, in computer programming, a synchronization lock is a mechanism used to control multiple threads or processes when accessing shared resources. Its main purpose is to ensure that at any time, only one thread or process can access shared resources, thereby avoiding problems caused by data competition and concurrent access.
[0113] In an embodiment of the present application, a synchronization lock may be set to control the number of status bits for a single synchronization. For example, the number of status bits for a single synchronization may be 1, which means that the synchronization lock can ensure that only one metadata block corresponding to one status bit is synchronized between the first node and the second node at the same time. The number of status bits for a single synchronization may also be understood as the volume granularity (grain) corresponding to the synchronization lock. The volume granularity of the synchronization lock is 1, which means that only one metadata block is synchronized at the same time.
[0114] The current state bit can be determined according to the number of each state bit in the synchronization bitmap. For example, starting from 0, state bit 0 can be determined as the current state bit, state bit 1 can be determined as the next state bit, and so on. The judgment is made according to the value of the current state bit.
[0115] If the value of the current status bit is the second preset value, it means that the metadata block corresponding to the current status bit is not synchronized, and a synchronization lock is assigned to the current status bit. According to the synchronization lock and the current status bit, the metadata block to be synchronized is determined. For example, if the volume granularity of the synchronization lock is 1, the metadata block to be synchronized is the data block corresponding to the current status bit. For another example, if the volume granularity of the synchronization lock is 2, the metadata block to be synchronized is the data block corresponding to the current status bit and the next status bit. After determining the metadata block to be synchronized, the metadata block to be synchronized can be synchronized from the first node to the second node.
[0116] The metadata synchronization method provided in the embodiment of the present application controls the number of status bits of a single synchronization by setting a synchronization lock, thereby controlling the data volume of the metadata of a single synchronization, avoiding problems caused by data competition and concurrent access, and improving the efficiency and accuracy of metadata synchronization.
[0117] In some embodiments, synchronizing the metadata block to be synchronized from the first node to the second node includes:
[0118] Determining a first node communication message based on the metadata block currently to be synchronized;
[0119] Sending the first node communication message to the second node, so that the second node stores the metadata block to be synchronized currently, and generates a second node communication message;
[0120] receiving a second node communication message sent by the second node;
[0121] Based on the second node communication message, it is determined that the metadata block to be synchronized currently has been synchronized.
[0122] Specifically, synchronizing the metadata block to be synchronized from the first node to the second node may be performed after the configuration information is updated after the second node comes online.
[0123] After the configuration information of the second node (node2) is updated, the metadata synchronization of the first node (node1) to the second node (node2) begins. The first node (node1) determines the metadata block to be synchronized, for example, starting from the status bit 0 (bit0), and synchronizes in sequence, synchronizing metadata of the size of 1 status bit each time.
[0124] First, status bit 0 obtains the metadata synchronization lock and generates the first node communication message. The first node communication message may include the starting position of the synchronized metadata, the length of the metadata synchronization, the target node and other information. Then, the 32K metadata of status bit 0 is sent to the second node (node2). After the second node (node2) receives the metadata, it copies the metadata sent by the message to the metadata space of this node. After the copy is completed, it sends the second node communication message to the first node (node1). After the first node (node1) receives the second node communication message, it confirms that the metadata of status bit 0 has been copied according to the starting position and length of the metadata in the message, and releases the metadata synchronization lock. Next, status bit 1 obtains the metadata synchronization lock. The synchronization process of status bit 1 is similar to that of status bit 0, and will not be repeated here.
[0125] The metadata synchronization method provided in the embodiment of the present application controls the synchronization of the current metadata block to be synchronized from the first node to the second node through the first node communication message and the second node communication message, thereby improving the efficiency and accuracy of metadata synchronization.
[0126] In some embodiments, the first node communication message and the second node communication message include metadata synchronization position information and metadata synchronization length information.
[0127] Specifically, the metadata synchronization position information refers to information such as the start position and end position of the metadata to be synchronized, etc. The metadata synchronization length information refers to information such as the data size of the metadata to be synchronized, etc.
[0128] The first node communication message and the second node communication message may also include information such as the target node, for example, the target node in the first node communication message is the node address information of the second node, and the target node in the second node communication message is the node address information of the first node.
[0129] In the metadata synchronization method provided by the embodiment of the present application, the first node communication message and the second node communication message include at least metadata synchronization position information and metadata synchronization length information, thereby improving the efficiency and accuracy of metadata synchronization.
[0130] In some embodiments, after synchronizing the metadata block to be synchronized from the first node to the second node, the method further includes:
[0131] Adjusting the value of the current state bit in the synchronization bitmap from the second preset value to the first preset value;
[0132] Assign the sync lock to the next status bit.
[0133] Specifically, after synchronizing the metadata block to be synchronized from the first node to the second node, it is necessary to adjust the value of the current state bit in the synchronization bitmap from the second preset value to the first preset value, that is, to change the synchronization state represented from unsynchronized to synchronized. Then, the synchronization lock is assigned to the next state bit, so that the synchronization of the metadata continues.
[0134] The metadata synchronization method provided in the embodiment of the present application adjusts the value of the current state bit and assigns the synchronization lock to the next state bit after synchronizing the current metadata block to be synchronized from the first node to the second node, thereby improving the efficiency and accuracy of metadata synchronization.
[0135] In some embodiments, after obtaining the value of the current state bit, the method further includes:
[0136] When the value of the current state bit is a first preset value, skipping the current state bit;
[0137] Get the value of the next status bit.
[0138] Specifically, when the value of the current state bit is the first preset value, it means that the metadata corresponding to the current state bit has been synchronized, so the current state bit is skipped, and the value of the next state bit is obtained to synchronize the metadata corresponding to the next state bit.
[0139] The metadata synchronization method provided in the embodiment of the present application skips the current state bit when the value of the current state bit is a first preset value, thereby improving the efficiency of metadata synchronization.
[0140] In some embodiments, determining that the metadata to be updated is synchronized based on the status bit includes:
[0141] When it is determined based on the status bit that the metadata to be updated is not synchronized, the execution of the current data read and write command is suspended, and the synchronization of the metadata to be updated is waited until the synchronization of the metadata to be updated is completed.
[0142] Specifically, if it is determined according to the status bit that the metadata to be updated is not synchronized, the execution of the current data read and write command may be suspended, waiting for the status bit corresponding to the metadata to be updated to obtain the synchronization lock, and completing the synchronization of the metadata to be updated.
[0143] For example, when synchronizing metadata between nodes, the first node receives a remote replication command (IO command) for processing or has a background replication task, and metadata needs to be changed, the specific operations are as follows:
[0144] Step 1: The first node (node1) receives the IO command, and the remote replication relationship receives the IO command and determines that the metadata needs to be updated;
[0145] Step 2: Find the status bit to which the metadata belongs according to the metadata position, and check in the synchronization bitmap whether the status bit has been synchronized to the second node.
[0146] Step 3: Process the next step based on whether the status bit is 1 or 0:
[0147] The synchronization bitmap corresponding to this status bit is 1. The metadata of this status bit has not been synchronized between the two nodes. Therefore, after waiting for the status bit currently being synchronized to complete the synchronization and release the lock, the status bit position where the metadata is synchronized is recorded, and the metadata synchronization lock is obtained for the status bit where the IO command is located. The starting position of the synchronization metadata of this status bit, the length of metadata synchronization, the target node and other communication information are assembled, and the metadata of this status bit is sent to the second node (node2). The second node (node2) copies the metadata to the corresponding space of this node. After the copying is completed, it returns a message to the first node (node1). The first node (node1) sets the bit of this status bit to 0, indicating that the metadata has been transmitted. Release the metadata transmission lock. After the lock is released, if there are still metadata transmission requests initiated due to IO commands to modify metadata, the metadata transmission lock will be given priority to these status bits. If not, continue to search for the next untransmitted metadata in the synchronization bitmap;
[0148] If the metadata corresponding to the status bit is being transmitted, wait for the status bit to be transmitted;
[0149] If this status bit is 0, synchronization has been completed on both nodes and no processing is performed.
[0150] Step 4: Process IO commands.
[0151] Step 5: After the IO command is processed, the remote replication metadata corresponding to the IO command on the first node (node1) and the second node (node2) is updated.
[0152] The metadata synchronization method provided in the embodiment of the present application suspends the execution of the current data read and write command when it is determined based on the status bit that the metadata to be updated is not synchronized, thereby avoiding metadata conflicts caused by executing the current data read and write command and synchronizing the metadata to be updated, and improving the accuracy of metadata synchronization.
[0153] In some embodiments, after all metadata synchronization is completed, the memory space occupied by the synchronization bitmap is released in the first node, marking that the metadata between the two nodes has been completely synchronized;
[0154] Control the first node and the second node to normally process all data read and write commands.
[0155] The embodiment of the present application provides a method for synchronizing metadata between nodes of a storage cluster, which can synchronize metadata from an original node (node1) to a newly added node (node2) while processing IO commands and modifying metadata.
[0156] First, after node2 comes online, only the existing node node1 is allowed to receive and process IO commands. That is, there is node1 in the cluster, and then node2 joins the cluster. After node2 joins the cluster, only node1 receives and processes IO commands. Then a memory space syncbitmap is allocated for metadata on node1 to mark whether the metadata has been synchronized to node2. The size of each metadata transmission is 32K, so each bit (status bit) of syncbitmap manages 32K of remote replication metadata, that is, the size of a sync lock grain is 32K.
[0157] After Node2 is online, the configuration information on Node2 is updated, and a metadata synchronization lock is applied for Node1. A syncbitmap is applied, and all syncbitmap bits are set to 1. Then, Node1 starts to synchronize metadata to Node2 sequentially, synchronizing one grain at a time, that is, 32K. The synchronized grain acquires the lock. After each synchronization of 32K metadata, Node2 returns it to Node1. Node1 updates the corresponding bit on the syncbitmap, changing the bit from 1 to 0, indicating that the grain bitmap has been synchronized, releasing the metadata synchronization lock, and then synchronizing the remote replication metadata with the status bit of 1 in the next syncbitmap.
[0158] When there is an IO command processing, the IO command needs to modify the remote replication metadata. Therefore, before modifying the metadata, the IO command needs to confirm whether the syncbitmap flag is 1 and whether the metadata modified by the IO command has been synchronized to node2. If it has been synchronized to node2, it is represented as 0, and the IO command is processed normally. After the processing is completed, the corresponding remote replication metadata on node1 and node2 is updated. If the corresponding status bit in the query syncbitmap is represented as 1, it means that the 32K metadata has not been synchronized between the two nodes. Then first wait for the metadata synchronization lock. After the current grain metadata synchronization is completed, the lock is given to the grain where the IO command is located. The grain acquires the lock, first synchronizes all the metadata corresponding to the flag to node2, and changes the corresponding status bit in the syncbitmap to 0. Then, the normal processing flow of the IO command is started, and the corresponding remote replication metadata is updated on node1 and node2. The IO command is written successfully.
[0159] The method provided in the embodiment of the present application does not need to synchronize remote replication metadata when silent IO commands are issued, but synchronizes remote replication metadata between nodes in the background when IO commands are processed. This allows new nodes to be available instantly after they are online, without silent IO commands, causing IO command interruptions, and without interrupting services or affecting services.
[0160] The device provided in the embodiment of the present application is described below. The device described below and the method described above can be referenced to each other.
[0161] Figure 3 is a schematic diagram of the structure of the metadata synchronization device provided by this application, such as Figure 3 As shown, the device comprises:
[0162] The metadata determination unit 310 is used to determine the metadata to be updated corresponding to the current data read and write command;
[0163] The metadata synchronization unit 320 is used to determine the status bit corresponding to the metadata to be updated in the synchronization bitmap; the synchronization bitmap is used to indicate the synchronization status of each metadata in the first node in the second node; the first node synchronizes each metadata to the second node in units of metadata corresponding to the status bit;
[0164] A command execution unit 330, configured to execute a current data read and write command when it is determined based on the status bit that the metadata to be updated has been synchronized;
[0165] The metadata updating unit 340 is used to update the metadata to be updated in the first node and the second node based on the execution result of the current data read and write command.
[0166] The metadata synchronization device provided in the embodiment of the present application determines the metadata to be updated corresponding to the current data read and write command; determines the status bit corresponding to the metadata to be updated in the synchronization bitmap; the synchronization bitmap is used to indicate the synchronization status of each metadata in the first node in the second node; executes the current data read and write command when it is determined based on the status bit that the metadata to be updated has been synchronized; updates the metadata to be updated in the first node and the second node based on the execution result of the current data read and write command; since the first node synchronizes each metadata to the second node in units of the metadata corresponding to the status bit, there is no need to synchronize all metadata to the second node at one time, and there is no need to keep the first node silent in data reading and writing; since the first node can continue to process the current data read and write command and update the metadata according to the synchronization status during the metadata synchronization process, metadata synchronization and data reading and writing are carried out simultaneously, thereby avoiding business interruption in the storage system and improving the user experience of the storage system.
[0167] In some embodiments, the metadata synchronization unit is used to:
[0168] When it is determined that the second node is online, determining a synchronization bitmap based on the synchronization status of each metadata in the first node;
[0169] Taking the metadata corresponding to the status bit in the synchronization bitmap as a unit, synchronizing the metadata corresponding to each status bit to the second node;
[0170] The status bits corresponding to the synchronized metadata are adjusted in the synchronization bitmap.
[0171] In some embodiments, the metadata synchronization unit is used to:
[0172] Divide each metadata in the first node into blocks to obtain multiple metadata blocks;
[0173] Determining a synchronization bitmap based on the storage location and synchronization status of each metadata block;
[0174] Among them, the status bits in the synchronization bitmap correspond one-to-one to multiple metadata blocks.
[0175] In some embodiments, the metadata synchronization unit is used to:
[0176] Based on the storage location of each metadata block, determine the number of the status bit corresponding to each metadata block in the synchronization bitmap;
[0177] Based on the synchronization status of each metadata block, the value of the status bit corresponding to each metadata block in the synchronization bitmap is determined.
[0178] In some embodiments, when the value of the status bit corresponding to any metadata block in the synchronization bitmap is a first preset value, the synchronization state of any metadata block is synchronized; when the value of the status bit corresponding to any metadata block in the synchronization bitmap is a second preset value, the synchronization state of any metadata block is unsynchronized.
[0179] In some embodiments, the metadata synchronization unit is used to:
[0180] Get the value of the current status bit;
[0181] When the value of the current state bit is the second preset value, a synchronization lock is allocated to the current state bit; the synchronization lock is used to control the number of state bits for a single synchronization;
[0182] Based on the synchronization lock and the current state bit, determine the current metadata block to be synchronized;
[0183] Synchronize the metadata block currently to be synchronized from the first node to the second node.
[0184] In some embodiments, the metadata synchronization unit is used to:
[0185] Determining a first node communication message based on the metadata block currently to be synchronized;
[0186] Sending the first node communication message to the second node, so that the second node stores the metadata block to be synchronized currently, and generates a second node communication message;
[0187] receiving a second node communication message sent by the second node;
[0188] Based on the second node communication message, it is determined that the metadata block to be synchronized currently has been synchronized.
[0189] In some embodiments, the first node communication message and the second node communication message include metadata synchronization position information and metadata synchronization length information.
[0190] In some embodiments, the metadata synchronization unit is used to:
[0191] Adjusting the value of the current state bit in the synchronization bitmap from the second preset value to the first preset value;
[0192] Assign the sync lock to the next status bit.
[0193] In some embodiments, the metadata synchronization unit is used to:
[0194] When the value of the current state bit is a first preset value, skipping the current state bit;
[0195] Get the value of the next status bit.
[0196] In some embodiments, the command execution unit is used to:
[0197] When it is determined based on the status bit that the metadata to be updated is not synchronized, the execution of the current data read and write command is suspended, and the synchronization of the metadata to be updated is waited until the synchronization of the metadata to be updated is completed.
[0198] Figure 4 is a schematic diagram of the structure of the storage system provided by this application, such as Figure 4 As shown, the storage system 400 includes a first node 410 and a second node 420 .
[0199] The first node is used to execute the metadata synchronization method in the above embodiment when detecting that the second node is online.
[0200] In the storage system provided by the embodiment of the present application, since the first node executes the metadata synchronization method in the above embodiment, and since the first node synchronizes each metadata to the second node in units of metadata corresponding to the status bit, there is no need to synchronize all metadata to the second node at one time, and there is no need to keep the first node silent for data reading and writing; since the first node can continue to process the current data reading and writing commands and update the metadata according to the synchronization status during the metadata synchronization process, metadata synchronization and data reading and writing are achieved simultaneously, thereby avoiding interruption of services in the storage system and improving the user experience of the storage system.
[0201] Figure 5 is a schematic diagram of the structure of the electronic device provided by this application, such as Figure 5 As shown, the electronic device may include: a processor (Processor) 510, a communication interface (Communications Interface) 520, a memory (Memory) 530 and a communication bus (Communications Bus) 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic command in the memory 530 to execute the method described in the above embodiment, for example:
[0202] Determine the metadata to be updated corresponding to the current data read / write command; determine the status bit corresponding to the metadata to be updated in the synchronization bitmap; the synchronization bitmap is used to indicate the synchronization status of each metadata in the first node in the second node; the first node synchronizes each metadata to the second node in units of metadata corresponding to the status bit; execute the current data read / write command when it is determined based on the status bit that the metadata to be updated has been synchronized; based on the execution result of the current data read / write command, update the metadata to be updated in the first node and the second node.
[0203] In addition, the logical commands in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several commands to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.
[0204] The processor in the electronic device provided in the embodiment of the present application can call the logic instructions in the memory to implement the above method. Its specific implementation method is consistent with the implementation method of the aforementioned method and can achieve the same beneficial effects, which will not be repeated here.
[0205] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the methods provided in the above embodiments are implemented.
[0206] Its specific implementation is consistent with the aforementioned method implementation and can achieve the same beneficial effects, so it will not be repeated here.
[0207] An embodiment of the present application provides a computer program product, including a computer program, which implements the above method when executed by a processor.
[0208] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0209] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0210] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A metadata synchronization method, characterized in that: include: Determine the metadata to be updated corresponding to the current data read and write command; Determine a status bit corresponding to the metadata to be updated in a synchronization bitmap; The synchronization bitmap is used to indicate the synchronization status of each metadata in the first node in the second node; The first node synchronizes each metadata to the second node in units of metadata corresponding to the status bit; In a case where it is determined based on the status bit that the metadata to be updated has been synchronized, executing the current data read and write command; Based on the execution result of the current data read and write command, updating the metadata to be updated in the first node and the second node; The method further comprises: In the case where it is determined that the second node is online, determining the synchronization bitmap based on the synchronization status of each metadata in the first node; Taking the metadata corresponding to the status bits in the synchronization bitmap as a unit, synchronizing the metadata corresponding to each status bit to the second node; Adjusting the status bit corresponding to the synchronized metadata in the synchronization bitmap; The step of synchronizing metadata corresponding to each status bit in the synchronization bitmap to the second node in units of metadata corresponding to the status bit, comprises: Get the value of the current status bit; When the value of the current state bit is a second preset value, a synchronization lock is allocated to the current state bit; the synchronization lock is used to control the number of state bits for a single synchronization; Determining a current metadata block to be synchronized based on the synchronization lock and the current state bit; Synchronize the metadata block to be synchronized from the first node to the second node; When the value of the status bit corresponding to any metadata block in the synchronization bitmap is a second preset value, the synchronization status of any metadata block is unsynchronized.
2. The metadata synchronization method according to claim 1, characterized in that: The determining the synchronization bitmap based on the synchronization status of each metadata in the first node includes: Divide each metadata in the first node into blocks to obtain multiple metadata blocks; Determining the synchronization bitmap based on the storage location and synchronization status of each metadata block; Among them, the status bits in the synchronization bitmap correspond one-to-one to the multiple metadata blocks.
3. The metadata synchronization method according to claim 2, characterized in that: The step of determining the synchronization bitmap based on the storage location and synchronization status of each metadata block includes: Determine, based on the storage location of each metadata block, the number of the status bit corresponding to each metadata block in the synchronization bitmap; Based on the synchronization status of each metadata block, the value of the status bit corresponding to each metadata block in the synchronization bitmap is determined.
4. The metadata synchronization method according to claim 3, characterized in that: When the value of the status bit corresponding to any metadata block in the synchronization bitmap is a first preset value, the synchronization status of any metadata block is synchronized.
5. The metadata synchronization method according to claim 1, characterized in that: The step of synchronizing the metadata block to be synchronized from the first node to the second node includes: Determining a first node communication message based on the current metadata block to be synchronized; Sending the first node communication message to the second node, so that the second node stores the metadata block to be synchronized and generates a second node communication message; receiving a second node communication message sent by the second node; Based on the second node communication message, it is determined that the metadata block currently to be synchronized has been synchronized.
6. The metadata synchronization method according to claim 5, characterized in that: The first node communication message and the second node communication message include metadata synchronization position information and metadata synchronization length information.
7. The metadata synchronization method according to claim 1, characterized in that: After synchronizing the metadata block to be synchronized from the first node to the second node, the method further includes: Adjusting the value of the current state bit in the synchronization bitmap from the second preset value to the first preset value; The synchronization lock is assigned to the next state bit.
8. The metadata synchronization method according to claim 1, characterized in that: After obtaining the value of the current state bit, the method further includes: When the value of the current state bit is a first preset value, skipping the current state bit; Get the value of the next status bit.
9. The metadata synchronization method according to any one of claims 1 to 8, characterized in that: The step of determining, based on the status bit, that the metadata to be updated is synchronized includes: When it is determined based on the status bit that the metadata to be updated is not synchronized, the execution of the current data read and write command is suspended, and the synchronization of the metadata to be updated is waited until the synchronization of the metadata to be updated is completed.
10. A metadata synchronization device, characterized in that: include: A metadata determination unit, used to determine the metadata to be updated corresponding to the current data read and write command; A metadata synchronization unit, used to determine a status bit corresponding to the metadata to be updated in a synchronization bitmap; The synchronization bitmap is used to indicate the synchronization status of each metadata in the first node in the second node; The first node synchronizes each metadata to the second node in units of metadata corresponding to the status bit; A command execution unit, configured to execute the current data read and write command when it is determined based on the status bit that the metadata to be updated has been synchronized; a metadata updating unit, configured to update the metadata to be updated in the first node and the second node based on the execution result of the current data read and write command; The metadata synchronization unit is used for: In the case where it is determined that the second node is online, determining the synchronization bitmap based on the synchronization status of each metadata in the first node; Taking the metadata corresponding to the status bits in the synchronization bitmap as a unit, synchronizing the metadata corresponding to each status bit to the second node; Adjusting the status bit corresponding to the synchronized metadata in the synchronization bitmap; The metadata synchronization unit is used for: Get the value of the current status bit; When the value of the current state bit is a second preset value, assigning a synchronization lock to the current state bit; The synchronization lock is used to control the number of status bits for a single synchronization; Determining a current metadata block to be synchronized based on the synchronization lock and the current state bit; Synchronize the metadata block to be synchronized from the first node to the second node; When the value of the status bit corresponding to any metadata block in the synchronization bitmap is a second preset value, the synchronization status of any metadata block is unsynchronized.
11. A storage system, characterized in that: including a first node and a second node; The first node is used to execute the metadata synchronization method according to any one of claims 1 to 9 when detecting that the second node is online.
12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the metadata synchronization method according to any one of claims 1 to 9 is implemented.
13. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the metadata synchronization method according to any one of claims 1 to 9 is implemented.
