Data synchronization method, device, equipment and medium in remote replication

Abstract

The present invention relates to a method, apparatus, device, and medium for data synchronization in remote replication, and relates to the field of computer technology. The method is applied to a primary storage system, wherein a primary volume storing data is created on the primary storage system. The method comprises: dividing the data to be synchronized in the primary volume into multiple sub-logical units based on the number of mirror pairs in the primary storage system, and dividing the sub-logical units into multiple data units; alternately allocating each data unit to each mirror pair in sequence, so that each data unit is evenly distributed to each mirror pair and adjacent data units are allocated to different mirror pairs; generating a bitmap for recording the data units allocated to the corresponding mirror pairs; and controlling each controller in the primary storage system to synchronize the corresponding data units to the secondary volume of the secondary storage system in sequence according to the corresponding target bitmap, so as to complete the initial synchronization of the remote replication; wherein the target bitmap is the bitmap of the mirror pair to which the controller belongs. The method improves the speed of initial synchronization and the performance of the storage system.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, device, equipment and medium for data synchronization in remote replication. Background Art

[0002] With the continuous development of informatization, data has become a vital enterprise asset, and the reliability and security requirements for critical business data are becoming increasingly stringent. Remote replication disaster recovery technology can enhance the risk mitigation capabilities of critical business operations. In the event of a disaster, it can help enterprises maintain or minimize business interruptions, ensuring continuous availability of business systems. Remote replication is used for data protection or disaster recovery purposes, creating two copies of a single piece of data. Remote replication initial synchronization is the synchronization process before data is copied from primary storage to secondary storage in a remote replication system. This process is crucial because it ensures data integrity and consistency. During initial synchronization, the system creates a replication task and copies data from primary storage to secondary storage. Specifically, all data on the primary volume's LUNs (Logical Unit Numbers) is copied to the secondary volume's LUNs. This process ensures data accuracy and integrity to prevent data loss or corruption.

[0003] The current remote replication initial synchronization strategy cannot evenly distribute LUNs, causing the controller node used for initial synchronization to bear all the work of remote replication initial synchronization. This obviously cannot meet the working requirements of balanced LUNs. That is, only one controller node handles initial synchronization, which means that only this controller node has business pressure, while the other controller nodes only have backups, resulting in a decrease in storage system performance and initial synchronization speed.

[0004] It can be seen that how to improve the speed of initial synchronization and storage system performance is a problem that those skilled in the art need to solve. Summary of the Invention

[0005] The purpose of the embodiments of the present invention is to provide a method, apparatus, device, and medium for data synchronization in remote replication to improve the speed of initial synchronization and storage system performance. The specific solution is as follows:

[0006] In a first aspect, the present application discloses a data synchronization method in remote replication, which is applied to a primary storage system, wherein a primary volume storing data is created on the primary storage system, and the method comprises:

[0007] Dividing the data to be synchronized in the primary volume into a plurality of sub-logical units based on the number of mirror pairs in the primary storage system and dividing the sub-logical units into a plurality of data units;

[0008] Alternately allocating the data units to the mirror pairs in sequence, so that the data units are evenly allocated to the mirror pairs and adjacent data units are allocated to different mirror pairs;

[0009] generating a bitmap for recording the data units allocated to the corresponding mirror pair;

[0010] Control each controller in the primary storage system to synchronize the corresponding data units to the auxiliary volume of the auxiliary storage system in sequence according to the corresponding target bitmap to complete the initial synchronization of remote replication; wherein, the target bitmap is the bitmap of the mirror pair to which the controller belongs.

[0011] Optionally, dividing the data to be synchronized in the primary volume into a plurality of sub-logical units based on the number of mirror pairs in the primary storage system and dividing the sub-logical units into a plurality of data units includes:

[0012] comparing first data stored in the primary volume with second data stored in a secondary volume of the secondary storage system, so as to determine data that is different between the first data and the second data as data to be synchronized;

[0013] The data to be synchronized is divided into a plurality of sub-logical units and the sub-logical units are divided into a plurality of data units based on the number of mirror pairs in the primary storage system; wherein the number of the sub-logical units is the same as the number of mirror pairs in the primary storage system, and the number of the data units corresponding to each sub-logical unit is the same as the number of mirror pairs in the primary storage system.

[0014] Optionally, generating a bitmap for recording the data units allocated to the corresponding mirror pair includes:

[0015] Determining the data units allocated to the current mirror pair, and determining a bitmap size of the current mirror pair based on the number of units of all the data units;

[0016] Requesting a current bitmap of the bitmap size from the primary storage system as a bitmap of the current mirror pair, recording a first mapping relationship between the current mirror pair and the data unit allocated to the mirror pair in the current bitmap, and marking a bitmap space in the current bitmap used for recording the first mapping relationship as a valid bitmap space;

[0017] The second mapping relationship between the other mirror pairs and the data unit allocated to itself is stored in the current bitmap, and the bitmap space in the current bitmap used to record the second mapping relationship is marked as an invalid bitmap space.

[0018] Optionally, controlling each controller in the primary storage system to synchronize the corresponding data units to the secondary volume of the secondary storage system in sequence according to the corresponding target bitmap includes:

[0019] Determine any controller in the primary storage system as a current controller;

[0020] Determine the current mirror pair to which the current controller belongs, and determine the bitmap corresponding to the current mirror pair as a target bitmap;

[0021] Determining the data unit recorded in the valid bitmap space of the target bitmap as the data unit to be synchronized of the current controller;

[0022] The current controller is controlled to synchronize each sub-data unit in the data unit to be synchronized to the auxiliary volume of the auxiliary storage system in sequence.

[0023] Optionally, controlling the current controller to synchronize each sub-data unit in the data unit to be synchronized to the auxiliary volume of the auxiliary storage system in sequence includes:

[0024] Determine the storage location of the sub-data units in the data unit to be synchronized in the primary volume, and control the current controller to read the sub-data units at each storage location in sequence, so as to synchronize the read sub-data units to the auxiliary volume of the auxiliary storage system in sequence.

[0025] Optionally, determining the storage location of the sub-data units in the data unit to be synchronized in the primary volume, and controlling the current controller to sequentially read the sub-data units at each of the storage locations includes:

[0026] Determine whether the sub-data unit currently being synchronized is the last sub-data unit in the corresponding data unit to be synchronized;

[0027] If the currently synchronized sub-data unit is not the last sub-data unit in the corresponding data unit to be synchronized, determining a storage location next to the storage location of the currently synchronized sub-data unit in the master volume as a target storage location of the next synchronized sub-data unit in the master volume, so that the current controller reads the sub-data unit at the target storage location;

[0028] If the currently synchronized sub-data unit is the last sub-data unit in the corresponding data unit to be synchronized, determining whether there is a next data unit to be synchronized that needs to be synchronized by the current controller;

[0029] If so, the target storage location of the first sub-data unit in the next data unit to be synchronized in the primary volume is determined based on the number of mirror pairs in the primary storage system and the amount of data stored in the data unit, so that the current controller can read the sub-data unit at the target storage location.

[0030] Optionally, after the initial synchronization of remote replication is completed, the following steps are also performed:

[0031] When performing periodic asynchronous replication, starting a change volume snapshot operation on the primary volume based on a preset period to obtain a snapshot bitmap;

[0032] Converting the snapshot bitmap according to a difference between the amount of data stored in the data unit corresponding to the snapshot bitmap and the amount of data stored in the data unit during the initial synchronization process to obtain a converted snapshot bitmap;

[0033] The data in the primary volume is periodically and asynchronously copied to the secondary volume based on the post-conversion snapshot bitmap.

[0034] In a second aspect, the present invention discloses a data synchronization device in remote replication, which is applied to a primary storage system, wherein a primary volume storing data is created on the primary storage system, and the device comprises:

[0035] a data partitioning module, configured to partition the data to be synchronized in the primary volume into a plurality of sub-logical units and to partition the sub-logical units into a plurality of data units based on the number of mirror pairs in the primary storage system;

[0036] a data distribution module, configured to distribute the data units to the mirror pairs alternately in sequence, so that the data units are evenly distributed to the mirror pairs and adjacent data units are distributed to different mirror pairs;

[0037] A bitmap generation module, configured to generate a bitmap for recording the data units allocated to the corresponding mirror pair;

[0038] A data synchronization module is used to control each controller in the primary storage system to synchronize the corresponding data units to the auxiliary volumes of the auxiliary storage system in sequence according to the corresponding target bitmap to complete the initial synchronization of remote replication; wherein, the target bitmap is the bitmap of the mirror pair to which the controller belongs.

[0039] It can be seen that the present invention is applied to a primary storage system, on which a primary volume for storing data is created, and the method includes: dividing the data to be synchronized in the primary volume into multiple sub-logical units and dividing the sub-logical units into multiple data units based on the number of mirror pairs in the primary storage system; allocating each of the data units to each of the mirror pairs in turn, so that each of the data units is evenly distributed to each of the mirror pairs and adjacent data units are allocated to different mirror pairs; generating a bitmap for recording the data units allocated to the corresponding mirror pairs; controlling each controller in the primary storage system to synchronize the corresponding data units to the auxiliary volume of the auxiliary storage system in turn according to the corresponding target bitmap, so as to complete the initial synchronization of remote replication; wherein, the target bitmap is the bitmap of the mirror pair to which the controller belongs.

[0040] Beneficial effect: The present invention divides the data to be synchronized into multiple data units, and allocates each data unit to each mirror pair alternately in turn. In this way, each data unit can be evenly allocated to each mirror pair and adjacent data units are allocated to different mirror pairs. That is to say, the data units allocated to each mirror pair are discontinuous, and the data units allocated to each mirror pair will not belong to the same sub-logical unit number. Then, when the subsequent controller performs initial synchronization of the data units according to the bitmap, it will not access the same sub-logical unit number in the primary volume. Then, all controllers need to perform initial synchronization and the amount of data that each controller needs to synchronize is also greatly reduced. This not only meets the load balancing requirements, but also improves the performance of remote replication. At this time, the background synchronization process can be processed in parallel, and each controller processes its own part, thereby improving the initial synchronization speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0042] Figure 1 This is a flow chart of a data synchronization method in remote replication provided by an embodiment of the present invention.

[0043] Figure 2 A specific data unit allocation diagram provided by an embodiment of the present invention;

[0044] Figure 3 A schematic diagram of a specific bitmap space provided by an embodiment of the present invention;

[0045] Figure 4A flowchart of a specific data synchronization method in remote replication provided by an embodiment of the present invention;

[0046] Figure 5 A schematic diagram of the structure of a data synchronization device in remote replication provided by an embodiment of the present invention;

[0047] Figure 6 A structural diagram of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0049] With the continuous development of informatization, data has become a vital asset for enterprises, and the reliability and security requirements for critical business data are becoming increasingly stringent. Remote replication disaster recovery technology can improve the risk prevention capabilities of critical businesses. In the event of a disaster, it can help enterprises maintain or minimize business interruptions, ensuring the continuous availability of business systems. Remote replication is used to achieve the purpose of data protection or disaster recovery, copying one piece of data into two copies. Remote replication initial synchronization refers to the synchronization process before data is copied from primary storage to secondary storage in a remote replication system. This process is very important because it ensures data integrity and consistency. During initial synchronization, the system creates a replication task and copies data from primary storage to secondary storage. Specifically, all data in the primary volume's LUN is copied to the secondary volume's LUN. This process must ensure data accuracy and integrity to avoid data loss or corruption.

[0050] The current remote replication initial synchronization strategy cannot evenly distribute LUNs, causing the controller node used for initial synchronization to bear all the work of remote replication initial synchronization. This obviously cannot meet the working requirements of balanced LUNs. That is, only one controller node handles initial synchronization, which means that only this controller node has business pressure, while the other controller nodes only have backups, resulting in a decrease in storage system performance and initial synchronization speed.

[0051] The terms "including" and "having," as used in the present description and accompanying drawings, and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements and may include steps or elements that are not listed.

[0052] In order to enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0053] Next, a data synchronization solution in remote replication provided by an embodiment of the present invention is described in detail. Figure 1 A data synchronization method in remote replication provided by an embodiment of the present invention is applied to a primary storage system, where a primary volume storing data is created on the primary storage system. The method includes:

[0054] Step S11: dividing the data to be synchronized in the primary volume into a plurality of sub-logical units based on the number of mirror pairs in the primary storage system, and dividing the sub-logical units into a plurality of data units.

[0055] In this embodiment, dividing the data to be synchronized in the primary volume into multiple sub-logical units and dividing the sub-logical units into multiple data units based on the number of mirror pairs in the primary storage system includes: comparing first data stored in the primary volume with second data stored in the secondary volume of the secondary storage system to determine that data that is different between the first data and the second data is data to be synchronized; dividing the data to be synchronized into multiple sub-logical units and dividing the sub-logical units into multiple data units based on the number of mirror pairs in the primary storage system; wherein the number of the sub-logical units is the same as the number of mirror pairs in the primary storage system, and the number of data units corresponding to each sub-logical unit is the same as the number of mirror pairs in the primary storage system.

[0056] At the beginning of remote replication, the first data stored in the primary volume of the primary storage system is not completely consistent with the second data stored in the secondary volume of the secondary storage system. The inconsistent data in the first data is dirty data, or data to be synchronized. The data to be synchronized is divided into multiple sub-logical units (sub-luns), each of which contains multiple data units (segments). In other words, the amount of data stored in a sub-logical unit is greater than the amount of data stored in a data unit. The data to be synchronized is divided into multiple sub-logical units based on the number of mirror pairs in the primary storage system. For example, if the number of mirror pairs (domains) in the primary storage system is 4, the data to be synchronized is divided into 4 sub-logical units, each of which contains 4 data units. Each data unit contains 128 sub-data units (grains). The amount of data stored in a sub-data unit is 256k, so the total amount of data stored in a data unit is 32M.

[0057] Step S12: Alternately distribute the data units to the mirror pairs in sequence, so that the data units are evenly distributed to the mirror pairs and adjacent data units are distributed to different mirror pairs.

[0058] After dividing the data units, each data unit needs to be allocated to each mirror pair. It should be noted that in this embodiment, each data unit is allocated to each mirror pair alternately in sequence. In this way, each data unit can be evenly allocated to each mirror pair, and adjacent data units can be allocated to different mirror pairs. For example, Figure 2 The diagram shows a specific data unit allocation diagram. For example, the data to be synchronized is divided into four sub-logical units, each of which contains four data units. The first sub-logical unit contains data units s1, s2, s3, and s4, the second sub-logical unit contains data units s5, s6, s7, and s8, the third sub-logical unit contains data units s9, s10, s11, and s12, and the fourth sub-logical unit contains data units s13, s14, s15, and s16. Next, s1 is allocated to the mirror pair domain0, s2 is allocated to the mirror pair domain1, s3 is allocated to the mirror pair domain2, s4 is allocated to the mirror pair domain3, and so on. Finally, s16 is allocated to the mirror pair domain3. Therefore, data units belonging to the same sub-logical unit will be allocated to different mirror pairs. Furthermore, because a data unit contains multiple sub-data units, the data units are allocated to the corresponding mirror pairs, that is, the sub-data units contained in the data unit are allocated to the corresponding mirror pairs.

[0059] Step S13: Generate a bitmap for recording the data units allocated to the corresponding mirror pair.

[0060] In this embodiment, the generation of a bitmap for recording the data units allocated to the corresponding mirror pair includes: determining the data units allocated to the current mirror pair, and determining the bitmap size of the current mirror pair based on the number of units of all the data units; requesting a current bitmap of the bitmap size from the primary storage system as the bitmap of the current mirror pair, so as to record in the current bitmap a first mapping relationship between the current mirror pair and the data units allocated to itself, and marking the bitmap space in the current bitmap used to record the first mapping relationship as a valid bitmap space; storing the second mapping relationship between other mirror pairs and the data units allocated to themselves in the current bitmap, and marking the bitmap space in the current bitmap used to record the second mapping relationship as an invalid bitmap space.

[0061] The bitmap management unit is page, 1page = 4KB. Therefore, the bitmap belonging to each mirror pair is an integer multiple of 4KB. The total bitmap size allocated to a volume is an integer multiple of 16KB. For example, if the sub-data unit size is 256KB and the data unit size is 32MB, a 1page bitmap will manage the volume space of 256 segments. The bitmap size required for a volume with n segments is ((n-1) / 4 / 256+1)*4 pages.

[0062] The bitmap management module (memory management) allocates a bitmap of the same size for each mirror pair, and each bitmap records the mapping relationship between all the data units allocated to the mirror itself, such as Figure 3 The following is a schematic diagram of a specific bitmap space, taking a bitmap size of 16 pages as an example. The specific record relationship is as follows:

[0063] 1) Bitmap corresponding to mirror pair domain0: The data units belonging to domain0 are mapped to the 0-1 / 4 space of the entire bitmap in ascending order of LBA. That is, the 0-1 / 4 space records the first mapping relationship and is the valid bitmap space. The remaining space records the second mapping relationship between the other mirror pairs and the data units allocated to them. The remaining space is the invalid bitmap space.

[0064] 2) Bitmap corresponding to mirror pair domain 1: The data units belonging to domain 1 correspond to 1 / 4-1 / 2 of the entire bitmap space in ascending order of LBA. That is, the first mapping relationship is recorded in the 1 / 4-1 / 2 space, which is the valid bitmap space. The remaining space records the second mapping relationship between the other mirror pairs and the data units allocated to them. The remaining space is the invalid bitmap space.

[0065] 3) Bitmap corresponding to mirror pair domain2: The data units belonging to domain2 correspond to the 1 / 2-3 / 4 space of the entire bitmap in ascending order of LBA. That is, the 1 / 2-3 / 4 space records the first mapping relationship and is the valid bitmap space. The remaining space records the second mapping relationship between the other mirror pairs and the data units allocated to them. The remaining space is the invalid bitmap space.

[0066] 4) Bitmap corresponding to mirror pair domain3: The data units belonging to domain3 correspond to the 3 / 4-1 space of the entire bitmap in ascending order of LBA. That is, the 3 / 4-1 space records the first mapping relationship and is the valid bitmap space. The remaining space records the second mapping relationship between the remaining mirror pairs and the data units allocated to them. The remaining space is the invalid bitmap space.

[0067] It can be understood that each mirror pair contains two controllers, which are the primary node and the backup node respectively. For example, there are a total of 4 mirror pairs, among which the controllers of the mirror pair domain0 are node0 (primary node) and node1 (backup node), the controllers of the mirror pair domain1 are node1 (primary node) and node2 (backup node), the controllers of the mirror pair domain2 are node2 (primary node) and node3 (backup node), and the controllers of the mirror pair domain3 are node3 (primary node) and node0 (backup node). The bitmap will be synchronized to each controller, among which the 0-1 / 4 space valid bitmap space will be synchronized between node0 and node1, the 1 / 4-1 / 2 space valid bitmap space will be synchronized between node1 and node2, the 1 / 2-3 / 4 space valid bitmap space will be synchronized between node2 and node3, and the 3 / 4-1 space valid bitmap space will be synchronized between node3 and node1.

[0068] Step S14: Control each controller in the primary storage system to synchronize the corresponding data units to the secondary volume of the secondary storage system in sequence according to the corresponding target bitmap to complete the initial synchronization of remote replication; wherein the target bitmap is the bitmap of the mirror pair to which the controller belongs.

[0069] In this embodiment, the control of each controller in the primary storage system to synchronize the corresponding data units to the auxiliary volume of the auxiliary storage system in sequence according to the corresponding target bitmap includes: determining any controller in the primary storage system as the current controller; determining the current mirror pair to which the current controller belongs, and determining the bitmap corresponding to the current mirror pair as the target bitmap; determining the data unit recorded in the valid bitmap space of the target bitmap as the data unit to be synchronized of the current controller; and controlling the current controller to synchronize each sub-data unit in the data unit to be synchronized to the auxiliary volume of the auxiliary storage system in sequence.

[0070] Any controller in the primary storage system is determined as the current controller. Specifically, for example, controller node1 is determined as the current controller. Controller node1 is the master node of the mirror pair domain1, that is, the mirror pair to which controller node1 belongs is domain1, that is, domain1 is the current mirror pair. The bitmap corresponding to the current mirror pair is determined as the target bitmap. It can be understood that the effective bitmap space of the target bitmap is 1 / 4-1 / 2 space. Therefore, the data unit recorded in the 1 / 4-1 / 2 space is determined as the data unit to be synchronized of the current controller, and the current controller is controlled to synchronize each sub-data unit in the data unit to be synchronized to the auxiliary volume of the auxiliary storage system in sequence. It can be seen that each controller participates in the initial synchronization at the same time, and each controller only needs to synchronize the data recorded in the target bitmap. There is no need for each controller to synchronize all the data to be synchronized, which can effectively improve the synchronization efficiency. Furthermore, when the current controller synchronizes each sub-data unit in the data unit to be synchronized to the auxiliary volume of the auxiliary storage system, since the data units to be synchronized corresponding to the current controller come from different sub-logical units, the current controller accesses the current sub-logical unit to synchronize each sub-data unit in the current data unit to be synchronized to the auxiliary volume, and then the current controller accesses the next sub-logical unit to synchronize each sub-data unit in the next data unit to be synchronized to the auxiliary volume. In other words, the current controller will not access the same sub-logical unit too frequently, thereby achieving load balancing.

[0071] In this embodiment, the control of the current controller to synchronize each sub-data unit in the data unit to be synchronized to the auxiliary volume of the auxiliary storage system in sequence includes: determining the storage position of the sub-data unit in the data unit to be synchronized in the primary volume, and controlling the current controller to read the sub-data units at each of the storage positions in sequence, so as to synchronize each of the read sub-data units to the auxiliary volume of the auxiliary storage system in sequence. Different sub-data units have different storage positions in the primary volume, and the storage position of the sub-data unit in the primary volume is determined, and the current controller is controlled to read the sub-data unit at the storage position, so as to synchronize each of the read sub-data units to the auxiliary volume of the auxiliary storage system. It can be understood that when the current controller synchronizes each sub-data unit to the auxiliary volume of the auxiliary storage system, it specifically synchronizes the sub-data unit to the master node in the mirror pair corresponding to the auxiliary volume.

[0072] In this embodiment, determining the storage position of the sub-data unit in the data unit to be synchronized in the primary volume and controlling the current controller to read the sub-data units at each storage position in sequence includes: judging whether the sub-data unit to be synchronized is the last sub-data unit in the corresponding data unit to be synchronized; if the sub-data unit to be synchronized is not the last sub-data unit in the corresponding data unit to be synchronized, determining the next storage position of the storage position of the sub-data unit to be synchronized in the primary volume as the target storage position of the next sub-data unit to be synchronized in the primary volume, so that the current controller reads the sub-data unit at the target storage position; if the sub-data unit to be synchronized is the last sub-data unit in the corresponding data unit to be synchronized, judging whether there is a next data unit to be synchronized that needs to be synchronized by the current controller; if so, determining the target storage position of the first sub-data unit in the next data unit to be synchronized in the primary volume based on the number of mirror pairs in the primary storage system and the amount of data stored in the data unit, so that the current controller reads the sub-data unit at the target storage position.

[0073] The data units to be synchronized by the current controller belong to different sub-logical units. In other words, the number of data units between the current data unit to be synchronized and the next data unit to be synchronized is the number of mirror pairs minus one. Therefore, based on the alternating allocation method of data units, the storage location of the next sub-data unit to be synchronized needs to be determined based on the specific situation of the currently synchronized sub-data unit, as follows:

[0074] 1) If the currently synchronized sub-data unit grain N n It is not the last sub-data unit of the data unit N to be synchronized, that is, there is a grain in the data unit N to be synchronized N n+1 , then the sub-data unit grain N n The next storage location after the storage location is the grain N n+1 The current controller reads the sub-data unit grain at the target storage location. N n+1 ;

[0075] 2) If the currently synchronized sub-data unit grain N n The last sub-data unit of the data unit N to be synchronized, that is, there is no grain in the data unit N to be synchronized N n+1, and there is a next data unit to be synchronized N+(M-1) that needs to be synchronized by the current controller, then the first sub-data unit grain in the next data unit to be synchronized N+(M-1) N+（M-1） The target storage location of 1 is determined based on the number of mirror pairs in the primary storage system and the amount of data stored in the data unit. The specific determination process is as follows:

[0076] 2.1) Determine the number of mirror pairs in the primary storage system. Subtract one from the number of mirror pairs and multiply it by the amount of data stored in the data unit.

[0077] 2.2) Add the obtained product to the currently synchronized sub-data unit grain N n Add the storage locations of

[0078] 2.3) Add one to the sum to get the first sub-data unit grain in the next data unit to be synchronized N+(M-1) N +（M-1） 1's target storage location;

[0079] For example, if the number of mirror pairs is 4 and the amount of data stored in the data unit is 128 grains, then the grain N+3 1=128*3+grain N n +1;

[0080] 3) If the currently synchronized sub-data unit grain N n The last sub-data unit of the data unit N to be synchronized, that is, there is no grain in the data unit N to be synchronized N n+1 , and there is no next data unit to be synchronized that needs to be synchronized by the current controller, then there is no need to determine the storage location of the first sub-data unit of the next data unit to be synchronized.

[0081] It can be seen that the present invention is applied to a primary storage system, on which a primary volume for storing data is created, and the method includes: dividing the data to be synchronized in the primary volume into multiple sub-logical units and dividing the sub-logical units into multiple data units based on the number of mirror pairs in the primary storage system; allocating each of the data units to each of the mirror pairs in turn, so that each of the data units is evenly distributed to each of the mirror pairs and adjacent data units are allocated to different mirror pairs; generating a bitmap for recording the data units allocated to the corresponding mirror pairs; controlling each controller in the primary storage system to synchronize the corresponding data units to the auxiliary volume of the auxiliary storage system in turn according to the corresponding target bitmap, so as to complete the initial synchronization of remote replication; wherein, the target bitmap is the bitmap of the mirror pair to which the controller belongs.

[0082] Beneficial effect: The present invention divides the data to be synchronized into multiple data units, and allocates each data unit to each mirror pair alternately in turn. In this way, each data unit can be evenly allocated to each mirror pair and adjacent data units are allocated to different mirror pairs. That is to say, the data units allocated to each mirror pair are discontinuous, and the data units allocated to each mirror pair will not belong to the same sub-logical unit number. Then, when the subsequent controller performs initial synchronization of the data units according to the bitmap, it will not access the same sub-logical unit number in the primary volume. Then, all controllers need to perform initial synchronization and the amount of data that each controller needs to synchronize is also greatly reduced. This not only meets the load balancing requirements, but also improves the performance of remote replication. At this time, the background synchronization process can be processed in parallel, and each controller processes its own part, thereby improving the initial synchronization speed.

[0083] See also Figure 4 This embodiment of the present invention discloses a specific method for data synchronization in remote replication. Compared to the previous embodiment, this embodiment further illustrates and optimizes the technical solution. This embodiment is applied to a primary storage system, on which a primary volume storing data is created. The method includes:

[0084] Step S21: dividing the data to be synchronized in the primary volume into a plurality of sub-logical units based on the number of mirror pairs in the primary storage system, and dividing the sub-logical units into a plurality of data units.

[0085] Step S22 : Alternately distribute the data units to the mirror pairs in sequence, so that the data units are evenly distributed to the mirror pairs and adjacent data units are distributed to different mirror pairs.

[0086] Step S23: Generate a bitmap for recording the data units allocated to the corresponding mirror pair.

[0087] Step S24: Control each controller in the primary storage system to synchronize the corresponding data units to the secondary volume of the secondary storage system in sequence according to the corresponding target bitmap to complete the initial synchronization of remote replication; wherein the target bitmap is the bitmap of the mirror pair to which the controller belongs.

[0088] After each controller synchronizes all corresponding data units in the target bitmap to the secondary volume of the secondary storage system, the data of the remote replication primary and secondary volumes are consistent. The background synchronization progress of the remote replication is updated on the master (owner) node of each mirror pair. After that, the synchronization progress of the four mirror pairs is integrated. The data of the remote replication primary and secondary volumes are consistent, that is, the synchronization progress reaches 100%, and the initial synchronization process is considered complete.

[0089] The data synchronization process is to realize data transmission from the primary storage system to the secondary storage system. In this embodiment, during data transmission, the data in each sub-data unit can be encrypted and transmitted, where the encryption algorithm is, for example, a symmetric encryption algorithm, an asymmetric encryption algorithm, a hash algorithm, etc. The encryption algorithm for data transmission can ensure that the data maintains confidentiality and integrity during the transmission process. These algorithms convert plaintext data into ciphertext data through specific mathematical rules and keys to prevent unauthorized personnel from reading or tampering with the data, providing strong protection for data synchronization in remote replication.

[0090] Step S25: When performing periodic asynchronous replication, a change volume snapshot operation is initiated for the primary volume based on a preset period to obtain a snapshot bitmap; the snapshot bitmap is converted according to the difference between the amount of data stored in the data unit corresponding to the snapshot bitmap and the amount of data stored in the data unit during the initial synchronization process to obtain a converted snapshot bitmap; and the data in the primary volume is periodically asynchronously replicated to the auxiliary volume based on the converted snapshot bitmap.

[0091] After completing the initial synchronization, if periodic asynchronous replication in remote replication is required, data synchronization is implemented based on a preset period. Specifically, a change volume snapshot operation is initiated on the primary volume based on the preset period to obtain a snapshot bitmap. It should be noted that the amount of data stored in the data unit corresponding to the snapshot bitmap is inconsistent with the amount of data stored in the data unit during the initial synchronization process. For example, the data unit during the initial synchronization process is 128K, while the data unit corresponding to the snapshot bitmap is usually not 128K. Therefore, it is necessary to convert the snapshot bitmap based on the difference between the amount of data stored in the data unit corresponding to the snapshot bitmap and the amount of data stored in the data unit during the initial synchronization process, so that the converted snapshot bitmap can be merged with the remote replication bitmap, and then the data in the primary volume is periodically asynchronously copied to the auxiliary volume based on the merged bitmap. It can be understood that the logic of obtaining the snapshot bitmap is the same as the logic of obtaining the bitmap during the initial synchronization process, that is, the logic of alternately allocating data units to corresponding mirror pairs and determining the storage location of the sub-data unit is the same as the initial synchronization process.

[0092] It can be seen that the present invention allocates each data unit to each mirror pair alternately in turn, and each data unit can be evenly allocated to each mirror pair and adjacent data units are allocated to different mirror pairs. That is to say, the data units allocated to each mirror pair will not belong to the same sub-logical unit number, which meets the load balancing requirements and can further improve the performance of remote replication. At this time, the background synchronization process can be processed in parallel, and each controller processes its own part, thereby improving the initial synchronization speed; further, because the logic of obtaining the snapshot bitmap in periodic asynchronous replication is consistent with the logic of obtaining the bitmap in initial synchronization, the present invention can also achieve load balancing and more efficient periodic asynchronous replication. Because the data volume of the sub-data unit in the snapshot bitmap is inconsistent with the data volume of the sub-data unit in the initial synchronization process, the snapshot bitmap needs to be converted so that periodic asynchronous replication can be achieved.

[0093] Figure 5 A schematic diagram of a data synchronization device in remote replication provided by an embodiment of the present invention is applied to a primary storage system. A primary volume storing data is created on the primary storage system. The device includes:

[0094] A data partitioning module 11 is configured to partition the data to be synchronized in the primary volume into a plurality of sub-logical units and to partition the sub-logical units into a plurality of data units based on the number of mirror pairs in the primary storage system;

[0095] A data distribution module 12 is configured to distribute the data units to the mirror pairs alternately in sequence, so that the data units are evenly distributed to the mirror pairs and adjacent data units are distributed to different mirror pairs;

[0096] A bitmap generation module 13, configured to generate a bitmap for recording the data units allocated to the corresponding mirror pair;

[0097] The data synchronization module 14 is used to control each controller in the primary storage system to synchronize the corresponding data units to the auxiliary volumes of the auxiliary storage system in sequence according to the corresponding target bitmap to complete the initial synchronization of remote replication; wherein, the target bitmap is the bitmap of the mirror pair to which the controller belongs.

[0098] It can be seen that the present invention is applied to a primary storage system, on which a primary volume for storing data is created, and the method includes: dividing the data to be synchronized in the primary volume into multiple sub-logical units and dividing the sub-logical units into multiple data units based on the number of mirror pairs in the primary storage system; allocating each of the data units to each of the mirror pairs in turn, so that each of the data units is evenly distributed to each of the mirror pairs and adjacent data units are allocated to different mirror pairs; generating a bitmap for recording the data units allocated to the corresponding mirror pairs; controlling each controller in the primary storage system to synchronize the corresponding data units to the auxiliary volume of the auxiliary storage system in turn according to the corresponding target bitmap, so as to complete the initial synchronization of remote replication; wherein, the target bitmap is the bitmap of the mirror pair to which the controller belongs.

[0099] Beneficial effect: The present invention divides the data to be synchronized into multiple data units, and allocates each data unit to each mirror pair alternately in turn. In this way, each data unit can be evenly allocated to each mirror pair and adjacent data units are allocated to different mirror pairs. That is to say, the data units allocated to each mirror pair are discontinuous, and the data units allocated to each mirror pair will not belong to the same sub-logical unit number. Then, when the subsequent controller performs initial synchronization of the data units according to the bitmap, it will not access the same sub-logical unit number in the primary volume. Then, all controllers need to perform initial synchronization and the amount of data that each controller needs to synchronize is also greatly reduced. This not only meets the load balancing requirements, but also improves the performance of remote replication. At this time, the background synchronization process can be processed in parallel, and each controller processes its own part, thereby improving the initial synchronization speed.

[0100] In a specific embodiment, the data partitioning module 11 includes:

[0101] a synchronization data determining unit, configured to compare the first data stored in the primary volume with the second data stored in the secondary volume of the secondary storage system, so as to determine data that is different between the first data and the second data as data to be synchronized;

[0102] a logical unit division unit, configured to divide the data to be synchronized into a plurality of sub-logical units and to divide the sub-logical units into a plurality of data units based on the number of mirror pairs in the primary storage system; wherein the number of the sub-logical units is the same as the number of mirror pairs in the primary storage system, and the number of the data units corresponding to each sub-logical unit is the same as the number of mirror pairs in the primary storage system.

[0103] In a specific embodiment, the bitmap generation module 13 includes:

[0104] a bitmap size determining unit, configured to determine the data units allocated to the current mirror pair, and determine a bitmap size of the current mirror pair based on the number of units of all the data units;

[0105] a first marking unit, configured to request a current bitmap of the bitmap size from the primary storage system as a bitmap of the current mirror pair, record a first mapping relationship between the current mirror pair and the data unit allocated to the first bitmap, and mark a bitmap space in the current bitmap used for recording the first mapping relationship as a valid bitmap space;

[0106] The second marking unit is configured to store the second mapping relationship between the other mirror pairs and the data unit allocated to itself into the current bitmap, and mark the bitmap space in the current bitmap used to record the second mapping relationship as an invalid bitmap space.

[0107] In a specific embodiment, the data synchronization module 14 includes:

[0108] a controller determining unit, configured to determine any controller in the primary storage system as a current controller;

[0109] a target bitmap determining unit, configured to determine the current mirror pair to which the current controller belongs, and determine the bitmap corresponding to the current mirror pair as a target bitmap;

[0110] a to-be-synchronized data unit determining unit, configured to determine the data unit recorded in the valid bitmap space of the target bitmap as the to-be-synchronized data unit of the current controller;

[0111] The sub-data unit synchronization sub-module is used to control the current controller to synchronize each sub-data unit in the data unit to be synchronized to the auxiliary volume of the auxiliary storage system in sequence.

[0112] In a specific embodiment, the sub-data unit synchronization submodule includes:

[0113] The sub-data synchronization unit is used to determine the storage location of the sub-data unit in the data unit to be synchronized in the primary volume, and control the current controller to read the sub-data units at each of the storage locations in sequence, so as to synchronize the read sub-data units to the auxiliary volume of the auxiliary storage system in sequence.

[0114] In a specific embodiment, the sub-data synchronization unit includes:

[0115] A judging subunit, configured to judge whether the sub-data unit currently being synchronized is the last sub-data unit in the corresponding data unit to be synchronized;

[0116] a first execution sub-unit, configured to, if the currently synchronized sub-data unit is not the last sub-data unit in the corresponding data unit to be synchronized, determine a storage location next to the storage location of the currently synchronized sub-data unit in the primary volume as a target storage location of a next synchronized sub-data unit in the primary volume, so that the current controller reads the sub-data unit at the target storage location;

[0117] a second execution sub-unit, configured to determine whether there is a next data unit to be synchronized that needs to be synchronized by the current controller if the currently synchronized sub-data unit is the last sub-data unit in the corresponding data unit to be synchronized;

[0118] The third execution sub-unit is used to determine the target storage location of the first sub-data unit in the next data unit to be synchronized in the primary volume based on the number of mirror pairs in the primary storage system and the amount of data stored in the data unit, if any, so that the current controller can read the sub-data unit at the target storage location.

[0119] In a specific embodiment, the data synchronization device in remote replication includes:

[0120] a snapshot bitmap acquisition unit, configured to initiate a change volume snapshot operation on the primary volume based on a preset period when performing periodic asynchronous replication, so as to obtain a snapshot bitmap;

[0121] a bitmap conversion unit, configured to convert the snapshot bitmap according to a difference between the amount of data stored in the data unit corresponding to the snapshot bitmap and the amount of data stored in the data unit during the initial synchronization process, so as to obtain a converted snapshot bitmap;

[0122] A periodic asynchronous replication unit is configured to periodically asynchronously replicate the data in the primary volume to the secondary volume based on the post-conversion snapshot bitmap.

[0123] Furthermore, the embodiment of the present application also discloses an electronic device, Figure 6This is a structural diagram of an electronic device according to an exemplary embodiment. The content in the diagram should not be considered as any limitation on the scope of use of this application. The electronic device may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the data synchronization method in remote replication disclosed in any of the aforementioned embodiments. In addition, the electronic device in this embodiment may specifically be an electronic computer.

[0124] In this embodiment, the power supply 23 is used to provide operating voltage for various hardware devices on the electronic device; the communication interface 24 can create a data transmission channel between the electronic device and external devices. The communication protocol it follows is any communication protocol that can be applied to the technical solution of this application and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world. Its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0125] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or CD, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0126] The operating system 221 is used to manage and control the hardware devices on the electronic device, as well as the computer program 222, which can be Windows Server, NetWare, Unix, Linux, etc. In addition to including a computer program capable of implementing the data synchronization method in remote replication performed by the electronic device disclosed in any of the aforementioned embodiments, the computer program 222 can further include computer programs capable of performing other specific tasks.

[0127] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when executed by a processor, the computer program implements the aforementioned method for synchronizing data in remote replication. The specific steps of this method can be referred to the corresponding contents disclosed in the aforementioned embodiments and will not be repeated here.

[0128] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. Reference can be made to the descriptions of the identical or similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions of the methods.

[0129] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0130] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0131] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0132] The above is a detailed introduction to the technical solution provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the ideas of the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A data synchronization method in remote replication, characterized in that: Applied to a primary storage system, where a primary volume storing data is created on the primary storage system, the method includes: Dividing the data to be synchronized in the primary volume into a plurality of sub-logical units based on the number of mirror pairs in the primary storage system and dividing the sub-logical units into a plurality of data units; Alternately allocating the data units to the mirror pairs in sequence, so that the data units are evenly allocated to the mirror pairs and adjacent data units are allocated to different mirror pairs; generating a bitmap for recording the data units allocated to the corresponding mirror pair; Controlling each controller in the primary storage system to synchronize the corresponding data units to the secondary volume of the secondary storage system in sequence according to the corresponding target bitmap, so as to complete the initial synchronization of the remote replication; wherein the target bitmap is the bitmap of the mirror pair to which the controller belongs; Partitioning the data to be synchronized in the primary volume into a plurality of sub-logical units based on the number of mirror pairs in the primary storage system and dividing the sub-logical units into a plurality of data units includes: Comparing the first data stored in the primary volume with the second data stored in the secondary volume of the secondary storage system to determine the data that is different between the first data and the second data as data to be synchronized; dividing the data to be synchronized into a plurality of sub-logical units based on the number of mirror pairs in the primary storage system, and dividing the sub-logical units into a plurality of data units; wherein the number of the sub-logical units is the same as the number of mirror pairs in the primary storage system, the number of the data units corresponding to each sub-logical unit is the same as the number of mirror pairs in the primary storage system, and data units belonging to the same sub-logical unit will be allocated to different mirror pairs.

2. The data synchronization method in remote replication according to claim 1, characterized in that: The generating of a bitmap for recording the data units allocated to the corresponding mirror pair includes: Determining the data units allocated to the current mirror pair, and determining a bitmap size of the current mirror pair based on the number of units of all the data units; Requesting a current bitmap of the bitmap size from the primary storage system as a bitmap of the current mirror pair, recording a first mapping relationship between the current mirror pair and the data unit allocated to the mirror pair in the current bitmap, and marking a bitmap space in the current bitmap used for recording the first mapping relationship as a valid bitmap space; The second mapping relationship between the other mirror pairs and the data unit allocated to itself is stored in the current bitmap, and the bitmap space in the current bitmap used to record the second mapping relationship is marked as an invalid bitmap space.

3. The data synchronization method in remote replication according to claim 2, characterized in that: The controlling each controller in the primary storage system to sequentially synchronize the corresponding data units to the secondary volumes of the secondary storage system according to the corresponding target bitmap includes: Determine any controller in the primary storage system as a current controller; Determine the current mirror pair to which the current controller belongs, and determine the bitmap corresponding to the current mirror pair as a target bitmap; Determining the data unit recorded in the valid bitmap space of the target bitmap as the data unit to be synchronized of the current controller; The current controller is controlled to synchronize each sub-data unit in the data unit to be synchronized to the auxiliary volume of the auxiliary storage system in sequence.

4. The data synchronization method in remote replication according to claim 3, characterized in that: The controlling the current controller to sequentially synchronize each sub-data unit in the data unit to be synchronized to the auxiliary volume of the auxiliary storage system includes: Determine the storage location of the sub-data units in the data unit to be synchronized in the primary volume, and control the current controller to read the sub-data units at each storage location in sequence, so as to synchronize the read sub-data units to the auxiliary volume of the auxiliary storage system in sequence.

5. The data synchronization method in remote replication according to claim 4, characterized in that: The determining of the storage location of the sub-data units in the data unit to be synchronized in the main volume, and controlling the current controller to sequentially read the sub-data units at each of the storage locations, includes: Determine whether the sub-data unit currently being synchronized is the last sub-data unit in the corresponding data unit to be synchronized; If the currently synchronized sub-data unit is not the last sub-data unit in the corresponding data unit to be synchronized, determining a storage location next to the storage location of the currently synchronized sub-data unit in the master volume as a target storage location of the next synchronized sub-data unit in the master volume, so that the current controller reads the sub-data unit at the target storage location; If the currently synchronized sub-data unit is the last sub-data unit in the corresponding data unit to be synchronized, determining whether there is a next data unit to be synchronized that needs to be synchronized by the current controller; If so, the target storage location of the first sub-data unit in the next data unit to be synchronized in the primary volume is determined based on the number of mirror pairs in the primary storage system and the amount of data stored in the data unit, so that the current controller can read the sub-data unit at the target storage location.

6. The data synchronization method in remote replication according to any one of claims 1 to 5, characterized in that: After the initial synchronization of remote replication is completed, it also includes: When performing periodic asynchronous replication, starting a change volume snapshot operation on the primary volume based on a preset period to obtain a snapshot bitmap; Converting the snapshot bitmap according to a difference between the amount of data stored in the data unit corresponding to the snapshot bitmap and the amount of data stored in the data unit during the initial synchronization process to obtain a converted snapshot bitmap; The data in the primary volume is periodically and asynchronously copied to the secondary volume based on the post-conversion snapshot bitmap.

7. A data synchronization device in remote replication, characterized in that: Applied to a primary storage system, where a primary volume storing data is created, the device comprises: a data partitioning module, configured to partition the data to be synchronized in the primary volume into a plurality of sub-logical units and to partition the sub-logical units into a plurality of data units based on the number of mirror pairs in the primary storage system; a data distribution module, configured to distribute the data units to the mirror pairs alternately in sequence, so that the data units are evenly distributed to the mirror pairs and adjacent data units are distributed to different mirror pairs; A bitmap generation module, configured to generate a bitmap for recording the data units allocated to the corresponding mirror pair; a data synchronization module, configured to control each controller in the primary storage system to sequentially synchronize the corresponding data units to the secondary volumes of the secondary storage system according to the corresponding target bitmap, so as to complete the initial synchronization of the remote replication; wherein the target bitmap is the bitmap of the mirror pair to which the controller belongs; The data synchronization device in remote replication is specifically used for: Comparing the first data stored in the primary volume with the second data stored in the secondary volume of the secondary storage system to determine the data that is different between the first data and the second data as data to be synchronized; dividing the data to be synchronized into a plurality of sub-logical units based on the number of mirror pairs in the primary storage system, and dividing the sub-logical units into a plurality of data units; wherein the number of the sub-logical units is the same as the number of mirror pairs in the primary storage system, the number of the data units corresponding to each sub-logical unit is the same as the number of mirror pairs in the primary storage system, and data units belonging to the same sub-logical unit will be allocated to different mirror pairs.

8. An electronic device, characterized in that: include: memory for storing computer programs; A processor is configured to execute the computer program to implement the steps of the data synchronization method in remote replication according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the data synchronization method in remote replication according to any one of claims 1 to 6 are implemented.

Images