Metadata synchronization method, device, controller, storage medium and program product

Abstract

In the field of storage technology, the present application discloses a metadata synchronization method, device, controller, storage medium and program product. In the method, when the controller needs to synchronize the metadata of the entire disk, for the metadata in the controller, its metadata can be divided into ownership information and target information other than the ownership information. The synchronization of the entire disk metadata is divided into data transmission and data merging links. For the data transmission link, two transmission channels are used to transmit the ownership information and the target information respectively. In the transmission link, since two transmission channels are set for parallel transmission, the data transmission can be accelerated, and the target controller can process the storage business normally only after the data transmission is completed. In addition, in the merging stage, only the second storage pool is merged, which can greatly accelerate the efficiency of the merging process, shorten the overall time consumption of the metadata full disk synchronization, enable the controller to join the storage business processing as soon as possible, and reduce the impact on the storage business.

Description

Technical Field

[0001] The present application relates to the field of storage technology, and in particular to a metadata synchronization method, device, controller, storage medium and program product. Background Art

[0002] RAID (Redundant Array of Independent Disks) is a data storage technology. In a RAID system, fault tolerance and recovery management of the underlying disks are crucial. These operations are called soft RAID when implemented at the software level. In the process of maintaining the consistency of disk information, soft RAID will perform a series of background tasks, such as initialization, reconstruction, verification and reconstruction, slow disk repair, and inspection.

[0003] When performing these background tasks, the RAID system records the task execution status by managing a piece of memory space (usually a part of RAM). For tasks involving the operation of the entire array stripe, such as initialization and reconstruction, the system usually uses a bitmap to track the processing status of each stripe.

[0004] In a multi-controller RAID system, each node (or controller) maintains its own bitmap information and metadata. To ensure data redundancy and reliability, when background tasks are executed, the node will synchronize the relevant metadata information to the peer node (partner node). In this way, even if a node fails, its metadata will not be lost because the peer node has a backup. When a node comes back online after a failure or a controller fails, the memory management module will be responsible for merging all metadata information of the failed node to the surviving node. Similarly, when an offline node returns, it will also receive and merge its metadata information from the surviving node to ensure the integrity and accuracy of the metadata.

[0005] In the metadata merging process, in order to ensure the correctness of the merging results and the efficiency of the merging process, bit operations and logical judgments are usually involved to ensure that the merged metadata can accurately reflect the current status of the array. For example, controller 1 transmits a 32k data packet to controller 2, controller 2 merges the data locally, and controller 2 notifies controller 1 to send the next data packet; the above steps are repeated until the data merging is completed, that is, the full data synchronization of controller 2 and controller 1 is completed. Since the data is transmitted serially during the merging process, this process will take a long time and will have a long-term impact on storage services.

[0006] In summary, how to effectively solve problems such as data synchronization of controllers is a technical problem that technical personnel in this field urgently need to solve. Summary of the invention

[0007] The purpose of this application is to provide a metadata synchronization method, device, controller, storage medium and program product, which can effectively accelerate the metadata synchronization efficiency and reduce the impact on storage services.

[0008] In order to solve the above technical problems, this application provides the following technical solutions:

[0009] A metadata synchronization method is applied to a controller managing an independent redundant array of disks, wherein the controller has a metadata synchronization relationship with a target controller managing the independent redundant array of disks, and a transmission link is provided between the controller and the target controller, wherein the transmission link includes a first transmission channel and a second transmission channel; the method comprises:

[0010] In the case where full metadata synchronization is required, the metadata is transmitted from the target controller to the controller in parallel using the first transmission channel and the second transmission channel; wherein the metadata includes attribution information and target information, the attribution information is transmitted through the first transmission channel, and the target information is transmitted through the second transmission channel;

[0011] storing a first data block transmitted through the first transmission channel in a first storage pool, and storing a second data block transmitted through the second transmission channel in a second storage pool;

[0012] Copying metadata in the first storage pool to a first metadata area;

[0013] According to the ownership information in the first metadata area, merging the second data blocks in the second storage pool to obtain a merging result, and copying the merging result to the second metadata area;

[0014] After the data merge is complete, make sure that the metadata is fully synchronized.

[0015] Preferably, merging the second data blocks in the second storage pool according to the ownership information in the first metadata area to obtain a merging result includes:

[0016] Determine a bitmap interval of a belonging node area and a bitmap interval of a non-belonging node area by using the belonging information;

[0017] The second data blocks in the second storage pool are merged according to the merging logic corresponding to different ownership situations to obtain a merging result.

[0018] Preferably, copying the metadata in the first storage pool to the first metadata area includes:

[0019] copying the ownership information from the first storage pool to the first metadata area;

[0020] In the first storage pool, deleting the copied ownership information;

[0021] Correspondingly, according to the ownership information in the first metadata area, the second data blocks in the second storage pool are merged to obtain a merge result, including:

[0022] Determining whether the first storage pool is empty;

[0023] If yes, merging the second data blocks in the second storage pool according to the ownership information in the first metadata area to obtain a merging result;

[0024] If no, continue waiting.

[0025] Preferably, determining that full disk synchronization of metadata is required includes:

[0026] When the controller is in a failure recovery processing stage, or the controller is in a new adding stage, it is determined that full disk synchronization of metadata is required.

[0027] Preferably, when the number of the target controllers exceeds one, transmitting metadata from the target controller to the controller in parallel using the first transmission channel and the second transmission channel comprises:

[0028] Transmitting the attribution information of each part from different target controllers to the controller through the first transmission channel; wherein the attribution information in different target controllers is consistent;

[0029] The target information of each target controller is transmitted to the controller through the second transmission channel.

[0030] Preferably, transmitting the attribution information of each part from different target controllers to the controller through the first transmission channel includes:

[0031] transmitting odd-numbered data blocks from a first target controller to the controller through the first transmission channel;

[0032] transmitting the even-numbered data blocks from the second target controller to the controller through the first transmission channel;

[0033] The ownership information in the first target controller and the second target controller are divided and numbered with the same block size to obtain the first data block, and the odd data block and the even data block are determined according to the parity of the sequence number.

[0034] Preferably, storing the second data block transmitted through the second transmission channel in the second storage pool includes:

[0035] Determine whether the second storage pool has stored a data block in the same area as the current second data block;

[0036] If yes, then hang the current second data block behind the data block in the same area;

[0037] If not, creating a new area in the second storage pool, and hanging the current second data block in the new area;

[0038] Wherein, the target information is divided into several areas.

[0039] Preferably, merging the second data blocks in the second storage pool according to the ownership information in the first metadata area to obtain a merging result includes:

[0040] According to the ownership information in the first metadata area, second data blocks from different target controllers in the second storage pool are merged to obtain the merge result.

[0041] Preferably, merging the second data blocks in the second storage pool according to the ownership information in the first metadata area to obtain a merging result includes:

[0042] Dividing the target information into a plurality of parallel merging regions;

[0043] A plurality of threads corresponding to the parallel merging regions are used to perform merging processing in parallel on the second data blocks corresponding to the parallel merging regions to obtain the merging result.

[0044] A metadata synchronization method is applied to a target controller that manages a redundant array of independent disks, the target controller and a controller that manages the redundant array of independent disks have a metadata synchronization relationship, a transmission link is provided between the controller and the target controller, and the transmission link includes a first transmission channel and a second transmission channel;

[0045] The method comprises:

[0046] Packing the local metadata by region to obtain a first data block corresponding to the attribution information and a second data block corresponding to the target information; wherein the metadata includes the attribution information and the target information;

[0047] In the case where the controller needs to perform full disk synchronization of metadata, the metadata is transmitted to the controller in parallel using the first transmission channel and the second transmission channel; the first data block is transmitted through the first transmission channel, and the second data block is transmitted through the second transmission channel;

[0048] After receiving the notification message of all reception completion fed back by the controller, data transmission is stopped.

[0049] Preferably, the first data block is transmitted through the first transmission channel, including:

[0050] Using a preset block size, dividing and numbering the attribution information to obtain the first data blocks, and sorting the first data blocks;

[0051] Transmitting a first data block with a sequence number of 1 to the controller through the first transmission channel;

[0052] After receiving the receiving response information fed back by the controller, the first data block whose sequence number is the last sent sequence number plus 1 is transmitted to the controller through the first transmission channel until all the first data blocks are sent.

[0053] Preferably, the first data block is transmitted through the first transmission channel, including:

[0054] Using a preset block size, the attribution information is divided and numbered to obtain the first data block, and odd-numbered data blocks and even-numbered data blocks are determined according to the parity of the sequence number;

[0055] Transmitting a first data block with a sequence number of 1 to the controller through the first transmission channel;

[0056] Detecting whether another controller transmits a first data block with a sequence number of 1 to the controller through the first transmission channel;

[0057] If yes, the pre-specified odd data blocks or even data blocks are transmitted to the controller in sequence through the first transmission channel.

[0058] A metadata synchronization device, comprising: a controller used to manage an independent redundant array of disks, the controller having a metadata synchronization relationship with a target controller managing the independent redundant array of disks, a transmission link between the controller and the target controller, the transmission link comprising a first transmission channel and a second transmission channel; the device comprising:

[0059] a data transmission module, configured to transmit metadata from the target controller to the controller in parallel using the first transmission channel and the second transmission channel when full metadata synchronization is required; wherein the metadata includes attribution information and target information, the attribution information is transmitted through the first transmission channel, and the target information is transmitted through the second transmission channel;

[0060] A data storage module, configured to store a first data block transmitted through the first transmission channel in a first storage pool, and store a second data block transmitted through the second transmission channel in a second storage pool;

[0061] a data merging module, configured to copy the metadata in the first storage pool to the first metadata area; merge the second data blocks in the second storage pool according to the ownership information in the first metadata area to obtain a merge result, and copy the merge result to the second metadata area;

[0062] The synchronization completion determination module is used to determine the completion of metadata full disk synchronization after the data merge is completed.

[0063] A metadata synchronization device is applied to a target controller managing an independent redundant array of disks, the target controller and a controller managing the independent redundant array of disks have a metadata synchronization relationship, a transmission link is provided between the controller and the target controller, and the transmission link includes a first transmission channel and a second transmission channel;

[0064] The device comprises:

[0065] A data packaging module, used to package the local metadata by region to obtain a first data block corresponding to the attribution information and a second data block corresponding to the target information; wherein the metadata includes the attribution information and the target information;

[0066] a data sending module, configured to transmit metadata to the controller in parallel by using the first transmission channel and the second transmission channel when the controller needs to perform full disk synchronization of metadata; the first data block is transmitted through the first transmission channel, and the second data block is transmitted through the second transmission channel;

[0067] The transmission completion confirmation module is used to stop data transmission after receiving the notification message of all reception completion fed back by the controller.

[0068] A controller, comprising:

[0069] Memory for storing computer programs;

[0070] A processor is used to implement the steps of the above-mentioned metadata synchronization method when executing the computer program.

[0071] A readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned metadata synchronization method are implemented.

[0072] A computer program product includes a computer program / instruction, which, when executed by a processor, implements the steps of the above-mentioned metadata synchronization method.

[0073] The method provided in the embodiment of the present application can be applied to a controller for managing a redundant array of independent disks, the controller and a target controller for managing a redundant array of independent disks have a metadata synchronization relationship, a transmission link is provided between the controller and the target controller, and the transmission link includes a first transmission channel and a second transmission channel; the method includes:

[0074] In the case where full disk synchronization of metadata is required, the metadata is transmitted from the target controller to the controller in parallel using the first transmission channel and the second transmission channel; wherein the metadata includes ownership information and target information, the ownership information is transmitted through the first transmission channel, and the target information is transmitted through the second transmission channel; the first data block transmitted through the first transmission channel is stored in the first storage pool, and the second data block transmitted through the second transmission channel is stored in the second storage pool; the metadata in the first storage pool is copied to the first metadata area; according to the ownership information in the first metadata area, the second data block in the second storage pool is merged to obtain a merged result, and the merged result is copied to the second metadata area; after completing the data merge, it is determined that the full disk synchronization of metadata is completed.

[0075] In the present application, when the controller needs to synchronize the metadata of the entire disk, parallel data transmission can be performed based on the transmission link between the controller and the target controller with the metadata synchronization relationship, thereby accelerating the synchronization of the entire disk of metadata. Specifically, for the metadata in the controller, its metadata can be divided into ownership information and target information other than the ownership information. For the controller and the target controller with the metadata synchronization relationship, under normal circumstances, the ownership information stored by each is consistent. For the target information, it is necessary to complete the data merge according to the ownership information to obtain the merge result of the local controller. Therefore, in the present application, the synchronization of the entire disk metadata is divided into data transmission and data merging links. For the data transmission link, two transmission channels are used to transmit the ownership information and the target information respectively. Two storage pools are also set up in the controller to store the ownership information and the target information respectively. In the data merging stage, the ownership information is directly copied to the first metadata area; for the target information, the second data block in the second storage pool is merged based on the ownership information in the first metadata area, thereby obtaining the merged result, and the merged result is stored in the second metadata area. After the data merge is completed, it is determined that the synchronization of the entire disk of metadata is completed.

[0076] Technical effect: In the transmission link, since two transmission channels are set for parallel transmission and data merging is performed after data transmission, it can not only accelerate data transmission, but also separate the target controller from the metadata full disk synchronization process after completing data transmission, so that it can handle storage business normally. In addition, in the merging stage, for the first storage pool, it is directly copied and only the second storage pool is merged, which can greatly accelerate the efficiency of the merging process and shorten the overall time of metadata full disk synchronization, so that the controller can join the storage business processing as soon as possible, thereby reducing the impact on the storage business.

[0077] Correspondingly, the embodiments of the present application also provide another metadata synchronization method corresponding to the above-mentioned metadata synchronization method, a metadata synchronization device, an equipment, a readable storage medium and a computer program product, which have the above-mentioned technical effects and are not described in detail here. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0079] Figure 1 This is a flowchart of an implementation method of metadata synchronization in an embodiment of the present application;

[0080] FIG2 is a flowchart of another metadata synchronization method in an embodiment of the present application;

[0081] FIG3 is a schematic diagram of metadata of RAID in an embodiment of the present application;

[0082] FIG4 is a schematic diagram of serial metadata synchronization;

[0083] FIG5 is a schematic diagram of a metadata partition and block division in an embodiment of the present application;

[0084] FIG6 is a flowchart of a specific implementation of a metadata synchronization method in an embodiment of the present application;

[0085] FIG. 7 is a schematic diagram of the contents of storage pool 1 in an embodiment of the present application;

[0086] FIG8 is a schematic diagram of the contents of storage pool 2 in an embodiment of the present application;

[0087] FIG9 is a schematic diagram of the structure of a metadata synchronization device according to an embodiment of the present application;

[0088] FIG10 is a schematic diagram of the structure of another metadata synchronization device according to an embodiment of the present application;

[0089] FIG11 is a schematic diagram of the structure of a controller in an embodiment of the present application;

[0090] FIG. 12 is a schematic diagram of the specific structure of a controller in an embodiment of the present application. DETAILED DESCRIPTION

[0091] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0092] Please refer to Figure 1 , Figure 1 A flowchart of a metadata synchronization method in an embodiment of the present application is applied to a controller managing a redundant array of independent disks, the controller and a target controller managing the redundant array of independent disks have a metadata synchronization relationship, a transmission link is provided between the controller and the target controller, and the transmission link includes a first transmission channel and a second transmission channel;

[0093] The method comprises the following steps:

[0094] S101 . When full disk synchronization of metadata is required, metadata is transmitted from a target controller to a controller in parallel using a first transmission channel and a second transmission channel.

[0095] The metadata includes attribution information and target information. The attribution information is transmitted through a first transmission channel, and the target information is transmitted through a second transmission channel.

[0096] In this application, when the controller needs to perform full disk synchronization of metadata, the metadata can be directly acquired by using the first transmission channel and the second transmission channel. The two transmission channels transmit the metadata existing in the target controller to the controller in parallel, which has a higher transmission efficiency.

[0097] In this embodiment, it is considered that the metadata mainly includes attribution information and target information, wherein the target information is other information in the metadata other than the non-attribution information, such as formatting information, reconstruction information, and the like.

[0098] The first transmission channel may be used to transmit the attribution information, and the second transmission channel may be used to transmit the target information.

[0099] In a specific implementation of the present application, determining that full metadata synchronization is required includes:

[0100] When the controller is in the fault recovery process or the controller is in the new joining stage, it is determined that full metadata synchronization is required. That is, the controller can be back online after a failure, or the controller can be offline. Or when the target controller fails, all metadata information of the failed target controller is merged to the surviving node, that is, the controller.

[0101] After the data transmission is completed, the target controller may send a notification message of the completion of the transmission to the controller, and then the target controller withdraws from the metadata full disk synchronization process of the controller.

[0102] S102: Store a first data block transmitted through a first transmission channel in a first storage pool, and store a second data block transmitted through a second transmission channel in a second storage pool.

[0103] In this embodiment, in order to speed up data merging, a first storage pool and a second storage pool may be set in the controller to store data blocks transmitted from different transmission channels respectively.

[0104] S103: Copy the metadata in the first storage pool to the first metadata area.

[0105] Since the data stored in the first storage pool is ownership information, and the ownership information is the same between controllers having a metadata synchronization relationship, it can be directly copied to the first metadata area.

[0106] S104: According to the ownership information in the first metadata area, merge the second data blocks in the second storage pool to obtain a merge result, and copy the merge result to the second metadata area.

[0107] In this embodiment, step S104 may be performed after the correction S103 is completed.

[0108] That is to say, for different information in metadata, it has the following characteristics:

[0109] 1. The ownership information is the same for each controller, describing the controller to which a certain space in the array belongs.

[0110] 2. The formatting information of each controller is different. Only the data of the home node and the backup node of the home node are accurate. Whether a certain space in the array is formatted depends on the metadata merging of the home information.

[0111] 3. The reconstruction information of each controller is different. Only the data of the home node and the backup node of the home node are accurate. The description of whether a certain space of the array has been reconstructed depends on the metadata merging of the home information.

[0112] That is to say, when performing metadata merging, all the ownership information has been copied. At this time, the second data block in the second storage pool can be merged directly based on the ownership information in the first metadata area, that is, the target information is merged based on the ownership information to obtain a merged result, which is the local target information excluding the non-ownership information in the metadata that the controller needs to maintain.

[0113] The merged result is directly copied to the second metadata area.

[0114] In a specific implementation of the present application, the second data blocks in the second storage pool are merged according to the ownership information in the first metadata area to obtain a merge result, including:

[0115] Determine the bitmap interval of the belonging node area and the bitmap interval of the non-belonging node area by using the belonging information;

[0116] The second data blocks in the second storage pool are merged according to the merging logic corresponding to the different ownership situations to obtain a merging result.

[0117] For ease of description, the above steps are combined for explanation below.

[0118] Metadata merge added by the controller: The belonging information is located in the header of the metadata, and the belonging information can be obtained first. The mask is set according to the state of the belonging information, and the strip position belonging to the node is set to 0, and the strip position not belonging to the node is set to 1. For the bitmap interval of the belonging node area: the local bitmap is bitwise OR 1 and then bitwise ANDed with the received bitmap as the local bitmap. For the bitmap interval of the non-belonging node area: the receiving bitmap is bitwise OR 1 and then bitwise ANDed with the local bitmap as the local bitmap.

[0119] In a specific implementation of the present application, copying metadata in the first storage pool to the first metadata area includes:

[0120] Copying ownership information from the first storage pool to the first metadata area;

[0121] In the first storage pool, deleting the copied ownership information;

[0122] Correspondingly, according to the ownership information in the first metadata area, the second data block in the second storage pool is merged to obtain a merge result, including:

[0123] Determine whether the first storage pool is empty;

[0124] If yes, then merge the second data blocks in the second storage pool according to the ownership information in the first metadata area to obtain a merge result;

[0125] If no, continue waiting.

[0126] For ease of description, the above steps are combined for explanation below.

[0127] That is, in this embodiment, after the attribution information is copied to the first metadata area, the copied attribution information is deleted in the first storage pool, so that by checking whether the first storage pool is empty, it can be determined whether the attribution information has been copied. In this way, after the attribution information is completely copied, the target information is merged based on the attribution information, which can reduce unnecessary waiting time and speed up data merging efficiency.

[0128] S105: After the data merge is completed, determine that the metadata full disk synchronization is completed.

[0129] After the data merge is completed, a data merge completion notification message may be sent to the target controller so that the target controller can determine that the control has completed the metadata full disk synchronization.

[0130] The method provided in the embodiment of the present application can be applied to a controller for managing a redundant array of independent disks, the controller and a target controller for managing a redundant array of independent disks have a metadata synchronization relationship, a transmission link is provided between the controller and the target controller, and the transmission link includes a first transmission channel and a second transmission channel; the method includes:

[0131] In the case where full disk synchronization of metadata is required, the metadata is transmitted from the target controller to the controller in parallel using the first transmission channel and the second transmission channel; wherein the metadata includes ownership information and target information, the ownership information is transmitted through the first transmission channel, and the target information is transmitted through the second transmission channel; the first data block transmitted through the first transmission channel is stored in the first storage pool, and the second data block transmitted through the second transmission channel is stored in the second storage pool; the metadata in the first storage pool is copied to the first metadata area; according to the ownership information in the first metadata area, the second data block in the second storage pool is merged to obtain a merged result, and the merged result is copied to the second metadata area; after completing the data merge, it is determined that the full disk synchronization of metadata is completed.

[0132] In the present application, when the controller needs to synchronize the metadata of the entire disk, parallel data transmission can be performed based on the transmission link between the controller and the target controller with the metadata synchronization relationship, thereby accelerating the synchronization of the entire disk of metadata. Specifically, for the metadata in the controller, its metadata can be divided into ownership information and target information other than the ownership information. For the controller and the target controller with the metadata synchronization relationship, under normal circumstances, the ownership information stored by each is consistent. For the target information, it is necessary to complete data merging according to the ownership information to obtain the merging result of the local controller. Therefore, in the present application, the synchronization of the entire disk metadata is divided into data transmission and data merging links. For the data transmission link, two transmission channels are used to transmit the ownership information and the target information respectively. Two storage pools are also set up in the controller to store the ownership information and the target information respectively. In the data merging stage, the ownership information is directly copied to the first metadata area; for the target information, the second data block in the second storage pool is merged based on the ownership information in the first metadata area, thereby obtaining the merged result, and the merged result is stored in the second metadata area. After the data merge is completed, it is determined that the synchronization of the entire disk of metadata is completed.

[0133] Technical effect: In the transmission link, since two transmission channels are set for parallel transmission and data merging is performed after data transmission, it can not only accelerate data transmission, but also separate the target controller from the metadata full disk synchronization process after completing data transmission, so that it can handle storage business normally. In addition, in the merging stage, for the first storage pool, it is directly copied and only the second storage pool is merged, which can greatly accelerate the efficiency of the merging process and shorten the overall time of metadata full disk synchronization, so that the controller can join the storage business processing as soon as possible, thereby reducing the impact on the storage business.

[0134] It should be noted that, based on the above embodiments, the embodiments of the present application also provide corresponding improved solutions. In the preferred / improved embodiments, the same steps or corresponding steps as those in the above embodiments can be referenced to each other, and the corresponding beneficial effects can also be referenced to each other, which will not be described one by one in the preferred / improved embodiments of this article.

[0135] In a specific implementation of the present application, when the number of target controllers exceeds one, metadata is transmitted from the target controller to the controller in parallel using a first transmission channel and a second transmission channel, including:

[0136] Transmitting the attribution information of each part from different target controllers to the controller through the first transmission channel; wherein the attribution information in different target controllers is consistent;

[0137] The target information of each target controller is transmitted to the controller through the second transmission channel.

[0138] For ease of description, the above steps are combined for explanation below.

[0139] Since the ownership information between controllers with a metadata synchronization relationship is completely consistent, during the metadata transmission process, multiple target controllers can transmit part of the ownership information to each controller respectively, thereby reducing the data transmission task volume of the target controller and avoiding the target controller from repeatedly transmitting the same ownership information, causing a waste of resources.

[0140] As for the target information, the content is different on different controllers. Therefore, each target controller needs to transmit the target information to the controller.

[0141] In a specific implementation of the present application, the attribution information of each part from different target controllers is transmitted to the controller through the first transmission channel, including:

[0142] transmitting odd-numbered data blocks from a first target controller to the controller via a first transmission channel;

[0143] transmitting the even-numbered data blocks from the second target controller to the controller via the first transmission channel;

[0144] The ownership information in the first target controller and the second target controller are divided and numbered with the same block size to obtain the first data block, and the odd data block and the even data block are determined according to the parity of the sequence number.

[0145] For ease of description, the above steps are combined for explanation below.

[0146] In order to accelerate the transmission, the target controller divides and numbers the belonging information according to the same block size to obtain a number of first data blocks, and determines the odd data blocks and the even data blocks according to the parity of the sequence numbers.

[0147] The first target controller sends the first data block 1 of the attribution information to the controller through channel 1 (first transmission channel), and at the same time, the second target controller sends the first data block 1 of the attribution information to the controller through channel 1.

[0148] The second target controller detects that the first target controller has sent data to the controller, and the second target controller sends the first data block 2 to the controller, and only sends even-numbered data blocks in subsequent transmissions.

[0149] The first target controller detects that the second target controller has sent data to the controller, and the first target controller sends the first data block 3 to the controller, and only sends odd-numbered data blocks in subsequent transmissions.

[0150] In a specific implementation manner of the present application, storing a second data block transmitted through a second transmission channel in a second storage pool includes:

[0151] Determine whether a data block having the same area as the current second data block has been stored in the second storage pool;

[0152] If yes, then hang the current second data block behind the data block in the same area;

[0153] If not, create a new area in the second storage pool and hang the current second data block in the new area;

[0154] Among them, the target information is divided into several areas.

[0155] For ease of description, the above steps are combined for explanation below.

[0156] The metadata is divided into multiple areas, where area 1 is the attribution information, and metadata is merged by copying; area 2 and subsequent areas all need the information of area 1 to merge metadata; there is no dependency between area 2 and subsequent areas, and area 2 and subsequent areas can be merged concurrently.

[0157] Regions 2 and above must all be sent to the controller.

[0158] The first target controller sends data block 1 of metadata area 2 to the controller through channel 2 , and at the same time, the second target controller sends data block 1 of metadata area 2 to the controller through channel 2 .

[0159] The controller receives data from the first target controller and puts it into storage pool 2. The controller receives data from the second target controller and detects whether there are data blocks in the same area. If so, it hangs it behind the data blocks in the same area. Otherwise, it creates a new area and stores it.

[0160] In a specific implementation of the present application, the second data blocks in the second storage pool are merged according to the ownership information in the first metadata area to obtain a merge result, including:

[0161] According to the ownership information in the first metadata area, the second data blocks from different target controllers in the second storage pool are merged to obtain a merge result.

[0162] That is, the controller thread reads a data block that needs to be processed in the second storage pool, detects that all nodes have received the data block, calculates the data block mask according to the ownership information of area 1 corresponding to the data block, calculates the final result of the metadata of this node based on the two data blocks in the second storage pool, writes it to the corresponding area, and deletes the data block in the second storage pool.

[0163] The above steps are executed in a loop until the last data block is written, and the first target controller and the second target controller are notified.

[0164] In a specific implementation of the present application, the second data blocks in the second storage pool are merged according to the ownership information in the first metadata area to obtain a merge result, including:

[0165] Divide the target information into several parallel merging regions;

[0166] A plurality of threads corresponding to the parallel merging regions are used to perform merging processing on the second data blocks corresponding to the parallel merging regions in parallel to obtain a merging result.

[0167] For ease of description, the above steps are combined for explanation below.

[0168] In this embodiment, the target information can be subdivided into several parallel merging areas, such as a formatting information merging area and a reconstruction information merging area. Metadata merging of different areas can be performed in parallel by multiple threads.

[0169] Since all the attribution information is already present, the merging of the data blocks within a parallel merging region does not interfere with each other. Therefore, in actual applications, for a parallel merging region, multiple parallel merging threads can be set to merge different data blocks, thereby improving merging efficiency. That is, the metadata merging of different blocks in the same region can also be done by multiple threads without affecting each other, thereby improving the efficiency of metadata merging.

[0170] Please refer to FIG. 2 . In the present application, another metadata synchronization method is also provided, which is applied to a target controller that manages a redundant array of independent disks. The target controller has a metadata synchronization relationship with a controller that manages a redundant array of independent disks. A transmission link is provided between the controller and the target controller, and the transmission link includes a first transmission channel and a second transmission channel.

[0171] Methods, including:

[0172] Step S201: Pack the local metadata by region to obtain a first data block corresponding to the attribution information and a second data block corresponding to the target information; wherein the metadata includes the attribution information and the target information;

[0173] Step S202: When the controller needs to perform full disk synchronization of metadata, the metadata is transmitted to the controller in parallel using the first transmission channel and the second transmission channel; the first data block is transmitted through the first transmission channel, and the second data block is transmitted through the second transmission channel;

[0174] Step S203: After receiving the notification message of all reception completion fed back by the receiving controller, stop data transmission.

[0175] The first data block is transmitted through the first transmission channel, including:

[0176] Using a preset block size, dividing and numbering the attribution information to obtain a first data block, and sorting the first data block;

[0177] Transmitting a first data block with a sequence number of 1 to the controller through a first transmission channel;

[0178] After receiving the receiving response information fed back by the controller, the first data block with the sequence number of the last transmission sequence number plus 1 is transmitted to the controller through the first transmission channel until all the first data blocks are sent.

[0179] The first data block is transmitted through the first transmission channel, including:

[0180] Using a preset block size, the attribution information is divided and numbered to obtain a first data block, and odd-numbered data blocks and even-numbered data blocks are determined according to the parity of the sequence number;

[0181] Transmitting a first data block with a sequence number of 1 to the controller through a first transmission channel;

[0182] Detect whether there is another controller transmitting a first data block with a sequence number of 1 to the controller through the first transmission channel;

[0183] If yes, the pre-specified odd data blocks or even data blocks are transmitted to the controller in sequence through the first transmission channel.

[0184] It should be noted that the method shown in FIG2 is essentially the same as Figure 1 The methods shown in the figure all correspond to the processes executed by the controllers of different measurement of metadata synchronization. Therefore, the specific implementation steps of the above methods can refer to Figure 1 The description of the method steps shown will not be repeated here one by one.

[0185] also, Figure 1 The method shown and the method shown in FIG. 2 can be implemented simultaneously between different controllers having the same metadata synchronization relationship. To facilitate understanding and implementation by those skilled in the art, the metadata synchronization method is described in detail below with reference to a specific scenario as an example.

[0186] RAID (Redundant Array of Independent Disks) is a data storage technology that improves data reliability, increases storage performance, and provides data redundancy by combining multiple hard disks into a logical unit. In a RAID system, fault-tolerant operations and recovery management of the underlying disks are crucial. These operations are called soft RAID when implemented at the software level. In the process of maintaining disk information consistency, soft RAID will perform a series of background tasks, such as initialization, reconstruction, verification reconstruction, slow disk repair, and inspection. These tasks are used to monitor the disk status, detect and correct abnormal conditions in a timely manner, and ensure data integrity and recoverability.

[0187] When performing these background tasks, the RAID system records the task execution status by managing a piece of memory space (usually a part of RAM). For tasks involving the operation of the entire array stripe, such as initialization and reconstruction, the system usually uses a bitmap to track the processing status of each stripe. The bitmap is an efficient data structure that can quickly determine which stripes have been processed or need to be processed.

[0188] In a multi-controller RAID system, each node (or controller) maintains its own bitmap information and metadata. To ensure data redundancy and reliability, when background tasks are executed, the node will synchronize the relevant metadata information to the peer node (partner node). In this way, even if a node fails, its metadata will not be lost because the peer node has a backup. When a node comes back online after a failure or a controller fails, the memory management module will be responsible for merging all metadata information of the failed node to the surviving node. Similarly, when an offline node returns, it will also receive and merge its metadata information from the surviving node to ensure the integrity and accuracy of the metadata.

[0189] In a four-controller system, the scenarios and directions of metadata merging are particularly complex. Nodes collaborate by forming domains, each of which consists of a master node and a peer node. The master node is responsible for processing write IO operations in the storage area within the domain and managing the metadata of background tasks. In order to avoid lock contention during IO operations, the system adopts a lock-free IO processing mechanism. At the same time, the master node also uses the peer node as a metadata backup to prevent data loss caused by a single point of failure.

[0190] In the metadata merging process, in order to ensure the correctness of the merging results and the efficiency of the merging process, the system needs to properly process the received data information and local data. This usually involves bit operations and logical judgments to ensure that the merged metadata can accurately reflect the current status of the array. In this way, the RAID system can maintain data consistency and reliability in a multi-node environment while providing efficient IO performance and fault tolerance.

[0191] Metadata merge added by the controller: The belonging information is located in the header of the metadata, and the belonging information can be obtained first. The mask is set according to the state of the belonging information, and the strip position belonging to the node is set to 0, and the strip position not belonging to the node is set to 1. For the bitmap interval of the belonging node area: the local bitmap is bitwise OR 1 and then bitwise ANDed with the received bitmap as the local bitmap. For the bitmap interval of the non-belonging node area: the receiving bitmap is bitwise OR 1 and then bitwise ANDed with the local bitmap as the local bitmap.

[0192] Multiple controllers transmit data to a single controller: Record and sort progress: Record the progress information of each node that merges with the regressing node, and sort the progress. First merge: Determine whether the current node is the node with the most progress. If so, perform the first merge. Set the mask and perform a bitmap merge, which is similar to the offline processing, but the local bitmap is the initial value (all 1) during the first merge, and the bitwise AND operation is used for the non-belonging area. Second merge: If it is not the first merge, execute the same bitmap merge logic as offline. Bitmap information interaction: When a node regresses, the node that is about to become its owner and the partner node will send their respective full bitmap information to the regressing node in sequence. There may be different progress and different rates.

[0193] As shown in Figure 3, RAID metadata includes:

[0194] The ownership information is the same for all controllers, describing the controller to which a certain space in the array belongs;

[0195] The formatting information is different for each controller. Only the data of the home node and the backup node of the home node is accurate. The description of whether a certain space in the array is formatted depends on the home information for metadata merging.

[0196] The reconstruction information of each controller is different. Only the data of the home node and the backup node of the home node are accurate. The description of whether a certain space of the array has been reconstructed depends on the metadata merging of the home information.

[0197] As shown in Figure 4, metadata merging between controllers: Controller 1 transmits a 32k data packet to Controller 2, Controller 2 merges the data locally, and Controller 2 notifies Controller 1 to send the next data packet; the above steps are repeated until the data merging is completed; since the attribution information is located at the head of the data transmission, the attribution information is available when formatting and reconstructing the data merge.

[0198] In order to solve the problem of metadata synchronization efficiency, the present application proposes a metadata synchronization method. After applying the metadata synchronization method provided by the present application, its specific implementation includes:

[0199] As shown in Figure 5, metadata is divided into different areas according to metadata dependencies. In addition, the controller divides metadata into data blocks of a certain size:

[0200] Metadata area 1 contains the same ownership information for multiple controllers;

[0201] The data blocks in metadata area 2 are dependent on the data blocks in metadata area 1.

[0202] When data is being transferred:

[0203] 1. Channel 1 (the first transmission channel) transmits the data blocks of metadata area 1 to controller 2 in sequence.

[0204] 2. Channel 2 (the second transmission channel) transmits the data blocks of the remaining metadata area to controller 2 in sequence, for example, block 1 of area 2, block 1 of area 3, block 2 of area 2, and so on.

[0205] Please refer to Figure 6 for the data transmission process:

[0206] Step 1: metadata area 1, metadata area 2, metadata area . . . data are transmitted through two channels. Metadata area 1 is transmitted through channel 1, and data area 2 is transmitted through channel 2.

[0207] Step 2: Controller 1 packages the data blocks and transmits the data to controller 2 through the inter-controller link.

[0208] Step 3: Controller 2 receives data transmitted by the communication link, and applies for two pools in the memory for storing the data. The data transmitted by channel 1 is placed in pool 1 (as shown in FIG. 7 ), and the data transmitted by channel 2 is placed in pool 2 (as shown in FIG. 8 ).

[0209] Step 4: Controller 2 notifies controller 1 that the data has been received.

[0210] Step 5: loop steps 1-4 until data transmission is completed, and notify controller 2 that all data transmission is complete.

[0211] The data merging process includes:

[0212] Step 1: Controller 2 merges (copies) the data. Controller 2 processes the metadata received in the memory through multiple threads, depending on the region:

[0213] Step 2: The metadata area 1 is copied to the metadata area 1 of the controller.

[0214] Step 3: The metadata area 2 merges the metadata by merging, calculates the final result of the metadata according to the corresponding block of the block in the metadata area 1, and copies the result to the controller metadata area 2.

[0215] Step 4: The thread reads the data in pool 1 first, and when the data in pool 1 is empty, it reads the data in pool 2 for processing.

[0216] Step 5: When all data processing is completed and a message is received from controller 1 indicating that all data transmission is complete, controller 2 notifies controller 1 that data merging is complete.

[0217] For a multi-controller cluster of an array, the array space is divided into multiple controllers. The controller that joins the cluster may take over multiple controllers (as a backup for multiple controllers). In this case, there is a scenario where multiple controllers send a single controller message (taking controllers 1 and 2 sending to controller 3 as an example):

[0218] Controller 1 sends data block 1 of metadata area 1 to controller 3 through channel 1, and controller 2 sends data block 1 of metadata area 1 to controller 3 through channel 1.

[0219] Controller 2 detects that controller 1 has sent data to controller 3, and controller 2 sends data block 2 to controller 3, and only sends even-numbered data blocks in subsequent transmissions.

[0220] Controller 1 detects that controller 2 has sent data to controller 3, and controller 1 sends data block 3 to controller 3, and only sends odd-numbered data blocks in subsequent transmissions.

[0221] Area 2 and subsequent areas must all be sent to controller 3, and the data of multiple controllers must be merged and processed, and the final calculation results must be written to the metadata area of ​​controller 3.

[0222] Regarding data transfers from Region 2 and later:

[0223] Controller 1 sends data block 1 of metadata area 2 to controller 3 through channel 1, and controller 2 sends data block 1 of metadata area 2 to controller 3 through channel 1;

[0224] Controller 3 receives data from controller 1 and puts it into pool 2. Controller 3 receives data from controller 2, detects whether there are data blocks in the same area, and hangs it behind the data blocks in the same area.

[0225] Data is merged and written to the metadata area, including:

[0226] Step 1: The thread of controller 3 reads a data block to be processed in pool 2 and detects that all nodes of the data block have received the data block (for the above example, both blocks have been received).

[0227] Step 2: Calculate the data block mask according to the ownership information of area 1 corresponding to the data block, calculate the final result of the metadata of this node in combination with the two data blocks in pool 2, write it to the corresponding area, and delete the data block.

[0228] Step 3: Execute the above steps repeatedly until the last data block is written, and notify controller 1 and controller 2.

[0229] It can be seen that the metadata merging of different regions can be performed in parallel by multiple threads, and the metadata merging of different blocks in the same region can also be performed by multiple threads without affecting each other, thereby improving the efficiency of metadata merging.

[0230] Asynchronous metadata data transmission: metadata data transmission between controllers is performed asynchronously, different blocks are transmitted concurrently, metadata is merged (or copied), and the sending node is notified that data synchronization is successful.

[0231] The data sending areas of multiple controllers are automatically coordinated. The same part of the data area is divided into multiple parts and sent concurrently by multiple controllers. The regional data that needs to be merged is merged, and the final calculation results are written to the corresponding area.

[0232] In other words, this application improves metadata merging efficiency, reduces controller joining time, and ensures business processing continuity.

[0233] Corresponding to the above method embodiment, the embodiment of the present application further provides a metadata synchronization device. The metadata synchronization device described below and the metadata synchronization method described above can refer to each other.

[0234] As shown in FIG. 9 , a controller for managing a redundant array of independent disks has a metadata synchronization relationship with a target controller for managing the redundant array of independent disks, and a transmission link is provided between the controller and the target controller, and the transmission link includes a first transmission channel and a second transmission channel; the device includes:

[0235] The data transmission module 101 is used to transmit metadata from the target controller to the controller in parallel using the first transmission channel and the second transmission channel when full metadata synchronization is required; wherein the metadata includes attribution information and target information, the attribution information is transmitted through the first transmission channel, and the target information is transmitted through the second transmission channel;

[0236] The data storage module 102 is used to store a first data block transmitted through a first transmission channel in a first storage pool, and to store a second data block transmitted through a second transmission channel in a second storage pool;

[0237] The data merging module 103 is used to copy the metadata in the first storage pool to the first metadata area; merge the second data block in the second storage pool according to the ownership information in the first metadata area to obtain a merge result, and copy the merge result to the second metadata area;

[0238] The synchronization completion determination module 104 is used to determine the completion of metadata full disk synchronization after the data merge is completed.

[0239] The device provided in the embodiment of the present application can be applied to a controller for managing a redundant array of independent disks, the controller and a target controller for managing a redundant array of independent disks have a metadata synchronization relationship, a transmission link is provided between the controller and the target controller, and the transmission link includes a first transmission channel and a second transmission channel; the method includes:

[0240] In the case where full disk synchronization of metadata is required, the metadata is transmitted from the target controller to the controller in parallel using the first transmission channel and the second transmission channel; wherein the metadata includes ownership information and target information, the ownership information is transmitted through the first transmission channel, and the target information is transmitted through the second transmission channel; the first data block transmitted through the first transmission channel is stored in the first storage pool, and the second data block transmitted through the second transmission channel is stored in the second storage pool; the metadata in the first storage pool is copied to the first metadata area; according to the ownership information in the first metadata area, the second data block in the second storage pool is merged to obtain a merged result, and the merged result is copied to the second metadata area; after completing the data merge, it is determined that the full disk synchronization of metadata is completed.

[0241] In the present application, when the controller needs to synchronize the metadata of the entire disk, parallel data transmission can be performed based on the transmission link between the controller and the target controller with the metadata synchronization relationship, thereby accelerating the synchronization of the entire disk of metadata. Specifically, for the metadata in the controller, its metadata can be divided into ownership information and target information other than the ownership information. For the controller and the target controller with the metadata synchronization relationship, under normal circumstances, the ownership information stored by each is consistent. For the target information, it is necessary to complete data merging according to the ownership information to obtain the merging result of the local controller. Therefore, in the present application, the synchronization of the entire disk metadata is divided into data transmission and data merging links. For the data transmission link, two transmission channels are used to transmit the ownership information and the target information respectively. Two storage pools are also set up in the controller to store the ownership information and the target information respectively. In the data merging stage, the ownership information is directly copied to the first metadata area; for the target information, the second data block in the second storage pool is merged based on the ownership information in the first metadata area, thereby obtaining the merged result, and the merged result is stored in the second metadata area. After the data merge is completed, it is determined that the synchronization of the entire disk of metadata is completed.

[0242] Technical effect: In the transmission link, since two transmission channels are set for parallel transmission and data merging is performed after data transmission, it can not only accelerate data transmission, but also separate the target controller from the metadata full disk synchronization process after completing data transmission, so that it can handle storage business normally. In addition, in the merging stage, for the first storage pool, it is directly copied and only the second storage pool is merged, which can greatly accelerate the efficiency of the merging process and shorten the overall time of metadata full disk synchronization, so that the controller can join the storage business processing as soon as possible, thereby reducing the impact on the storage business.

[0243] In a specific implementation of the present application, the data merging module is specifically applied to determine the bitmap interval of the belonging node area and the bitmap interval of the non-belonging node area by using the belonging information;

[0244] The second data blocks in the second storage pool are merged according to the merging logic corresponding to the different ownership situations to obtain a merging result.

[0245] In a specific implementation of the present application, the data merging module is specifically used to copy the attribution information first metadata area from the first storage pool;

[0246] In the first storage pool, deleting the copied ownership information;

[0247] Correspondingly, according to the ownership information in the first metadata area, the second data block in the second storage pool is merged to obtain a merge result, including:

[0248] Determine whether the first storage pool is empty;

[0249] If yes, then merge the second data blocks in the second storage pool according to the ownership information in the first metadata area to obtain a merge result;

[0250] If no, continue waiting.

[0251] In a specific implementation of the present application, the data transmission module is also used to determine that full disk synchronization of metadata is required when the controller is in a fault recovery processing stage or the controller is in a new joining stage.

[0252] In a specific implementation of the present application, when the number of target controllers exceeds one, the data transmission module is specifically used to transmit the attribution information of each part from different target controllers to the controller through the first transmission channel; wherein the attribution information in different target controllers is consistent;

[0253] The target information of each target controller is transmitted to the controller through the second transmission channel.

[0254] In a specific implementation of the present application, the data transmission module is specifically used to transmit an odd-numbered data block from a first target controller to the controller through a first transmission channel;

[0255] transmitting the even-numbered data blocks from the second target controller to the controller via the first transmission channel;

[0256] The ownership information in the first target controller and the second target controller are divided and numbered with the same block size to obtain the first data block, and the odd data block and the even data block are determined according to the parity of the sequence number.

[0257] In a specific implementation of the present application, the data storage module is specifically used to determine whether a data block having the same area as the current second data block has been stored in the second storage pool;

[0258] If yes, then hang the current second data block behind the data block in the same area;

[0259] If not, create a new area in the second storage pool and hang the current second data block in the new area;

[0260] Among them, the target information is divided into several areas.

[0261] In a specific implementation of the present application, the data merging module is specifically used to merge the second data blocks from different target controllers in the second storage pool according to the ownership information in the first metadata area to obtain a merge result.

[0262] In a specific implementation of the present application, the data merging module is specifically used to divide the target information into a plurality of parallel merging regions;

[0263] A plurality of threads corresponding to the parallel merging regions are used to perform merging processing on the second data blocks corresponding to the parallel merging regions in parallel to obtain a merging result.

[0264] Corresponding to the above method embodiment, the embodiment of the present application further provides another metadata synchronization device. The metadata synchronization device described below and the metadata synchronization device described above can refer to each other.

[0265] As shown in FIG10 , a target controller for managing a redundant array of independent disks has a metadata synchronization relationship with a controller for managing a redundant array of independent disks, and a transmission link is provided between the controller and the target controller, and the transmission link includes a first transmission channel and a second transmission channel;

[0266] Device, comprising:

[0267] The data packaging module 201 is used to package the local metadata by region to obtain a first data block corresponding to the attribution information and a second data block corresponding to the target information; wherein the metadata includes the attribution information and the target information;

[0268] The data sending module 202 is used to transmit metadata to the controller in parallel using the first transmission channel and the second transmission channel when the controller needs to perform full disk synchronization of metadata; the first data block is transmitted through the first transmission channel, and the second data block is transmitted through the second transmission channel;

[0269] The transmission completion confirmation module 203 is used to stop data transmission after receiving the notification message of all reception completion fed back by the controller.

[0270] In a specific implementation of the present application, the data sending module is specifically used to divide and number the attribution information using a preset block size to obtain a first data block, and sort the first data block;

[0271] Transmitting a first data block with a sequence number of 1 to the controller through a first transmission channel;

[0272] After receiving the receiving response information fed back by the controller, the first data block with the sequence number of the last transmission sequence number plus 1 is transmitted to the controller through the first transmission channel until all the first data blocks are sent.

[0273] In a specific implementation of the present application, the data sending module is specifically used to divide and number the attribution information using a preset block size to obtain a first data block, and determine an odd data block and an even data block according to the parity of the sequence number;

[0274] Transmitting a first data block with a sequence number of 1 to the controller through a first transmission channel;

[0275] Detect whether there is another controller transmitting a first data block with a sequence number of 1 to the controller through the first transmission channel;

[0276] If yes, the pre-specified odd data blocks or even data blocks are transmitted to the controller in sequence through the first transmission channel.

[0277] Corresponding to the above method embodiment, the embodiment of the present application further provides a controller. The controller described below and the metadata synchronization method described above can refer to each other.

[0278] As shown in FIG11 , the controller includes:

[0279] A memory 332, for storing computer programs;

[0280] The processor 322 is configured to implement the steps of the metadata synchronization method of the above method embodiment when executing a computer program.

[0281] Specifically, please refer to FIG. 12, which is a schematic diagram of the specific structure of a controller provided in this embodiment. The controller may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) (for example, one or more processors) and a memory 332, and the memory 332 stores one or more computer programs 342 or data 344. Among them, the memory 332 can be a temporary storage or a permanent storage. The program stored in the memory 332 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the data processing device. Furthermore, the processor 322 can be configured to communicate with the memory 332 and execute a series of instruction operations in the memory 332 on the controller 301.

[0282] The controller 301 may also include one or more power supplies 326 , one or more wired or wireless network interfaces 350 , one or more input and output interfaces 358 , and / or one or more operating systems 341 .

[0283] The steps in the metadata synchronization method described above can be implemented by the structure of the controller.

[0284] Corresponding to the above method embodiment, the embodiment of the present application further provides a readable storage medium. The readable storage medium described below and the metadata synchronization method described above can refer to each other.

[0285] A readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the metadata synchronization method of the above method embodiment are implemented.

[0286] The readable storage medium may specifically be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or other readable storage medium that can store program codes.

[0287] Corresponding to the above method embodiment, the embodiment of the present application further provides a computer program product. The computer program product described below and the metadata synchronization method described above can refer to each other.

[0288] A computer program product includes a computer program / instruction. When the computer program / instruction is executed by a processor, the steps of the above-mentioned metadata synchronization method are implemented.

[0289] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0290] Those skilled in the art may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0291] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0292] Finally, it should be noted that, in this article, relationships such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms include, include or any other variations are intended to cover non-exclusive inclusion, so that a process, method, article or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0293] Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A metadata synchronization method, characterized in that: A controller used for managing a redundant array of independent disks, wherein the controller has a metadata synchronization relationship with a target controller managing the redundant array of independent disks, and a transmission link is provided between the controller and the target controller, wherein the transmission link includes a first transmission channel and a second transmission channel; The method comprises: In the case where full metadata synchronization is required, the metadata is transmitted from the target controller to the controller in parallel using the first transmission channel and the second transmission channel; wherein the metadata includes ownership information and target information, the ownership information is transmitted through the first transmission channel, and the target information is transmitted through the second transmission channel; storing a first data block transmitted through the first transmission channel in a first storage pool, and storing a second data block transmitted through the second transmission channel in a second storage pool; Copying metadata in the first storage pool to a first metadata area; According to the ownership information in the first metadata area, merging the second data blocks in the second storage pool to obtain a merging result, and copying the merging result to the second metadata area; After completing the data merge, confirm that the metadata full disk synchronization is complete; The step of merging the second data blocks in the second storage pool according to the ownership information in the first metadata area to obtain a merging result includes: Determine a bitmap interval of a belonging node area and a bitmap interval of a non-belonging node area by using the belonging information; According to the merging logic corresponding to different ownership situations, the second data blocks in the second storage pool are merged to obtain a merging result; The step of copying the metadata in the first storage pool to the first metadata area includes: copying the ownership information from the first storage pool to the first metadata area; In the first storage pool, deleting the copied ownership information; Correspondingly, according to the ownership information in the first metadata area, the second data blocks in the second storage pool are merged to obtain a merge result, including: Determining whether the first storage pool is empty; If yes, merging the second data blocks in the second storage pool according to the ownership information in the first metadata area to obtain a merging result; If no, continue waiting.

2. The method according to claim 1, characterized in that Determine that full metadata synchronization is required, including: When the controller is in a failure recovery processing stage, or the controller is in a new adding stage, it is determined that full disk synchronization of metadata is required.

3. The method according to claim 1, characterized in that When the number of the target controllers exceeds one, transmitting metadata from the target controller to the controller in parallel by using the first transmission channel and the second transmission channel includes: Transmitting the attribution information of each part from different target controllers to the controller through the first transmission channel; wherein the attribution information in different target controllers is consistent; The target information of each target controller is transmitted to the controller through the second transmission channel.

4. The method according to claim 3, characterized in that Transmitting each portion of the attribution information from different target controllers to the controller through the first transmission channel includes: transmitting odd-numbered data blocks from a first target controller to the controller through the first transmission channel; transmitting the even-numbered data blocks from the second target controller to the controller through the first transmission channel; The ownership information in the first target controller and the second target controller are divided and numbered with the same block size to obtain the first data block, and the odd data block and the even data block are determined according to the parity of the sequence number.

5. The method according to claim 3, characterized in that: Storing the second data block transmitted through the second transmission channel in the second storage pool includes: Determine whether the second storage pool has stored a data block in the same area as the current second data block; If yes, then hang the current second data block behind the data block in the same area; If not, creating a new area in the second storage pool, and hanging the current second data block in the new area; Wherein, the target information is divided into several areas.

6. The method according to claim 3, characterized in that: The method further comprises: performing a merging process on the second data blocks in the second storage pool according to the ownership information in the first metadata area to obtain a merging result, including: According to the ownership information in the first metadata area, second data blocks from different target controllers in the second storage pool are merged to obtain the merge result.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: performing a merging process on the second data blocks in the second storage pool according to the ownership information in the first metadata area to obtain a merging result, including: Dividing the target information into a plurality of parallel merging regions; A plurality of threads corresponding to the parallel merging regions are used to perform merging processing in parallel on the second data blocks corresponding to the parallel merging regions to obtain the merging result.

8. A metadata synchronization method, characterized in that: A target controller for managing a redundant array of independent disks, wherein the target controller has a metadata synchronization relationship with a controller managing the redundant array of independent disks, and a transmission link is provided between the controller and the target controller, wherein the transmission link includes a first transmission channel and a second transmission channel; The method comprises: Packing the local metadata by region to obtain a first data block corresponding to the attribution information and a second data block corresponding to the target information; wherein the metadata includes the attribution information and the target information; In the case where the controller needs to perform full disk synchronization of metadata, the metadata is transmitted to the controller in parallel using the first transmission channel and the second transmission channel; the first data block is transmitted through the first transmission channel, and the second data block is transmitted through the second transmission channel; After receiving the notification message of all reception completion fed back by the controller, stop data transmission; The first data block is transmitted through the first transmission channel, including: Using a preset block size, dividing and numbering the attribution information to obtain the first data blocks, and sorting the first data blocks; Transmitting a first data block with a sequence number of 1 to the controller through the first transmission channel; After receiving the receiving response information fed back by the controller, transmitting the first data block whose sequence number is the last sent sequence number plus 1 to the controller through the first transmission channel until all the first data blocks are sent; Or, the first data block is transmitted through the first transmission channel, including: Using a preset block size, the attribution information is divided and numbered to obtain the first data block, and odd-numbered data blocks and even-numbered data blocks are determined according to the parity of the sequence number; Transmitting a first data block with a sequence number of 1 to the controller through the first transmission channel; Detecting whether another controller transmits a first data block with a sequence number of 1 to the controller through the first transmission channel; If yes, the pre-specified odd data blocks or even data blocks are transmitted to the controller in sequence through the first transmission channel.

9. A metadata synchronization device, characterized in that: include: A controller used for managing a redundant array of independent disks, wherein the controller has a metadata synchronization relationship with a target controller managing the redundant array of independent disks, and a transmission link is provided between the controller and the target controller, wherein the transmission link includes a first transmission channel and a second transmission channel; The device comprises: a data transmission module, configured to transmit metadata from the target controller to the controller in parallel using the first transmission channel and the second transmission channel when full metadata synchronization is required; wherein the metadata includes attribution information and target information, the attribution information is transmitted through the first transmission channel, and the target information is transmitted through the second transmission channel; A data storage module, configured to store a first data block transmitted through the first transmission channel in a first storage pool, and store a second data block transmitted through the second transmission channel in a second storage pool; a data merging module, configured to copy the metadata in the first storage pool to the first metadata area; merge the second data blocks in the second storage pool according to the ownership information in the first metadata area to obtain a merge result, and copy the merge result to the second metadata area; A synchronization completion confirmation module is used to confirm the completion of metadata full disk synchronization after the data merge is completed; The step of merging the second data blocks in the second storage pool according to the ownership information in the first metadata area to obtain a merging result includes: Determine a bitmap interval of a belonging node area and a bitmap interval of a non-belonging node area by using the belonging information; According to the merging logic corresponding to different ownership situations, the second data blocks in the second storage pool are merged to obtain a merging result; The step of copying the metadata in the first storage pool to the first metadata area includes: copying the ownership information from the first storage pool to the first metadata area; In the first storage pool, deleting the copied ownership information; Correspondingly, according to the ownership information in the first metadata area, the second data blocks in the second storage pool are merged to obtain a merge result, including: Determining whether the first storage pool is empty; If yes, merging the second data blocks in the second storage pool according to the ownership information in the first metadata area to obtain a merging result; If no, continue waiting.

10. A metadata synchronization device, characterized in that: A target controller for managing a redundant array of independent disks, wherein the target controller has a metadata synchronization relationship with a controller managing the redundant array of independent disks, and a transmission link is provided between the controller and the target controller, wherein the transmission link includes a first transmission channel and a second transmission channel; The device comprises: A data packaging module, used to package the local metadata by region to obtain a first data block corresponding to the attribution information and a second data block corresponding to the target information; wherein the metadata includes the attribution information and the target information; a data sending module, configured to transmit metadata to the controller in parallel by using the first transmission channel and the second transmission channel when the controller needs to perform full disk synchronization of metadata; the first data block is transmitted through the first transmission channel, and the second data block is transmitted through the second transmission channel; A transmission completion confirmation module is used to stop data transmission after receiving a notification message of all reception completion fed back by the controller; The first data block is transmitted through the first transmission channel, including: Using a preset block size, dividing and numbering the attribution information to obtain the first data blocks, and sorting the first data blocks; Transmitting a first data block with a sequence number of 1 to the controller through the first transmission channel; After receiving the receiving response information fed back by the controller, transmitting the first data block whose sequence number is the last sent sequence number plus 1 to the controller through the first transmission channel until all the first data blocks are sent; Or, the first data block is transmitted through the first transmission channel, including: Using a preset block size, the attribution information is divided and numbered to obtain the first data block, and odd-numbered data blocks and even-numbered data blocks are determined according to the parity of the sequence number; Transmitting a first data block with a sequence number of 1 to the controller through the first transmission channel; Detecting whether another controller transmits a first data block with a sequence number of 1 to the controller through the first transmission channel; If yes, the pre-specified odd data blocks or even data blocks are transmitted to the controller in sequence through the first transmission channel.

11. A controller, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the metadata synchronization method according to any one of claims 1 to 8 when executing the computer program.

12. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the metadata synchronization method according to any one of claims 1 to 8 are implemented.

13. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the metadata synchronization method described in any one of claims 1 to 8 are implemented.

Images