Method for updating synchronization log, computer equipment, storage medium and program product

By introducing synchronization log files in the field of storage technology to mark differential data, the problem of low request processing efficiency in the prior art is solved, and more efficient data synchronization operations are achieved.

CN119396851AActive Publication Date: 2025-02-07INSPUR SUZHOU INTELLIGENT TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411980960.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-12-30
Publication Date
2025-02-07
Estimated Expiration
2044-12-30

AI Technical Summary

Technical Problem

In the field of storage technology, the prior art is less efficient when processing consecutive multiple requests, especially when these requests are modified for data at different locations of the same storage particle, the synchronization operation and the difference bit of the preamble request are first completed. The update of the graph results in a reduced processing efficiency.

Method used

By introducing a synchronization log file into the data synchronization system, the log of each request is recorded in place of the difference bitmap to mark the difference data. Each time a request is received, the request information is parsed, the target log is generated, and it is added to the synchronous log file. At the same time, synchronization instructions are sent to the auxiliary storage system to perform data synchronization operations.

Benefits of technology

This method can mark differential data in a smaller dimension than the storage particles, allowing multiple requests to be executed in parallel, significantly improving the request processing efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119396851A_ABST
    Figure CN119396851A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of storage, and discloses a method for updating a synchronization log, computer equipment, a storage medium and a program product.The method comprises the steps that when a target request is obtained, the target request is analyzed, and data operation information corresponding to the target request is obtained; generating a target log corresponding to the target request according to a pre-constructed data synchronization mechanism and the data operation information; obtaining identification information of the current node; determining a synchronous log file corresponding to the identification information of the current node according to the identification information of the current node; and adding the target log to the synchronization log file, and sending synchronization indication information to an auxiliary node corresponding to the current node in the auxiliary storage system so as to execute data synchronization operation. The request processing efficiency can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of storage technology, and in particular to a method for updating a synchronization log, a computer device, a storage medium and a program product. Background Art

[0002] In the field of storage technology, a remote copy (RC) relationship is defined as the relationship between a primary storage system and its volumes. Based on the remote copy relationship, the primary storage system synchronizes the data on each primary volume to the corresponding secondary volume in the primary storage system.

[0003] When creating a remote replication relationship, a difference bitmap can be created on the primary storage system. Each difference bit included in the difference bitmap is used to indicate whether the latest data written on the corresponding storage grain on the primary volume has been synchronized to the secondary volume. After receiving a request, any node in the primary storage system will modify the difference bit corresponding to the storage grain where the requested operation data is located in the difference bitmap it manages to "dirty", and then synchronize the operation data corresponding to the request to the corresponding storage location on the secondary storage system. After completing the write operation of the primary storage system and the synchronization operation of the secondary storage system, the current node will modify the difference bit corresponding to the storage grain to "clean".

[0004] Since the difference bitmap is constructed based on the granularity of storage particles, if the consecutive requests received are requests to modify data at different locations in the same storage particle, the subsequent requests can only be processed after the previous request completes the synchronization operation and updates the difference bitmap, which is inefficient. For example, when the three requests received are requests to modify different locations of storage particle 1, the data corresponding to request 1 must first be written to storage particle 1, the difference bit of storage particle 1 in the difference bitmap must be modified to "dirty", and the data corresponding to request 1 must be synchronized to the auxiliary storage system. After completing the synchronization operation corresponding to request 1, the difference bit of storage particle 1 must be modified to "clean" before the related operations of request 2 can be performed. Summary of the invention

[0005] In view of this, the present invention provides a method, apparatus, computer equipment, storage medium and program product for updating a synchronization log to solve the problem of low request processing efficiency and data synchronization efficiency.

[0006] In a first aspect, the present invention provides a method for updating a synchronization log, the method being applied to a data synchronization system, the data synchronization system comprising a primary storage system and an auxiliary storage system, the primary storage system and the auxiliary storage system respectively comprising a plurality of nodes of the same number, the method being executed by a current node, the current node being any one of the plurality of nodes included in the primary storage system, the method comprising: When a target request is obtained, the target request is parsed to obtain data operation information corresponding to the target request; Generate target logs corresponding to target requests based on pre-built data synchronization mechanisms and data operation information; Get the identification information of the current node; According to the identification information of the current node, determine the synchronization log file corresponding to the identification information of the current node; Add the target log to the sync log file; Send synchronization indication information to the secondary node corresponding to the current node in the secondary storage system, wherein the synchronization indication information includes the data to be synchronized corresponding to the target log and the location information corresponding to the data to be synchronized, so that the secondary node performs a data synchronization operation.

[0007] The method for updating synchronization logs provided by the present invention has the following advantages: Each time this solution obtains a request, it can parse the request to obtain data operation information. Moreover, a log can be generated according to the data synchronization mechanism of the current node and the parsed data operation information. In addition, this solution determines the corresponding synchronization log file according to the identification information of the current node. Further, the current node can add the generated log to the synchronization log file to mark the data corresponding to the target log as the difference data between the primary storage system and the auxiliary storage system, so as to replace the role of using a difference bitmap to mark the difference data in the related technology. In this way, for multiple consecutive requests for different storage locations of the same storage particle, this solution can record a log for each request in the synchronization log file, that is, this solution can mark the difference data in a dimension smaller than the storage particle, and execute the synchronization operations corresponding to multiple requests for different storage locations of the same storage particle in parallel, which can improve the request processing efficiency.

[0008] In some optional implementations, after sending the synchronization indication information to the secondary node corresponding to the current node in the secondary storage system, the method further includes: receiving target response information fed back by the secondary node, wherein the target response information is response information generated by the secondary node after completing the data synchronization operation according to the synchronization indication information; Delete the target log from the synchronization log file according to the target response information.

[0009] Specifically, when the target response information fed back by the auxiliary node is received, the data synchronization operation can be determined, and at this time, the corresponding target log is deleted in time to indicate that the difference data corresponding to the target log has been synchronized. In this way, storage resources can be vacated in time.

[0010] In an optional implementation, the data operation information includes a target logical start address and a target data length.

[0011] Specifically, the storage location of the target request operation data can be indicated by the target logical start address and the target data length. When a target log is generated based on this information, the specific storage location can be indicated by the target log, which facilitates accurate data synchronization operations.

[0012] In an optional implementation, a target log corresponding to a target request is generated according to a pre-built data synchronization mechanism and data operation information, including: When the data synchronization mechanism is a real-time type data synchronization mechanism, a target log is generated according to the target logical start address and the target data length.

[0013] Specifically, since the real-time data synchronization mechanism means that a data synchronization operation is performed after each request is completed in the write operation of the main storage system, for the real-time data synchronization mechanism, this solution can generate a target log based on the target logical starting address and target data length to mark the location of the currently stored difference data.

[0014] In an optional implementation, a target log corresponding to a target request is generated according to a pre-built data synchronization mechanism and data operation information, including: When the data synchronization mechanism is a periodic data synchronization mechanism, determining whether the current node is in a data synchronization state, wherein the data synchronization state is used to indicate that the current node is performing a data synchronization operation of the current cycle; When it is determined that the current node is not in a data synchronization state, determining identification information of a target storage particle corresponding to the target request according to a target logical start address and a preset data length, wherein the preset data length is a data length of the storage particle; Generate a target log according to the identification information of the target storage granule.

[0015] Specifically, for a periodic data synchronization mechanism, multiple requests need to be synchronized in each cycle. There are multiple requests for writing data in different storage locations of the same storage particle in the synchronized request, and each synchronization requires network resources and computing resources. Therefore, under this mechanism and when the node is not in a data synchronization state, the identification information of the target storage particle corresponding to the data operated by the target request is calculated according to the target logical starting address and the preset data length, and recorded as a target log. In this way, when performing data synchronization operations, data synchronization operations can be performed in units of storage particles, saving network resources and computing resources.

[0016] In an optional embodiment, the method further includes: When it is determined that the current node is in a data synchronization state, a target log is generated according to the target logical start address and the target data length.

[0017] Specifically, when the node is in the data synchronization state, a target log is generated according to the target logical start address and the target data length. In this way, the data that needs to be synchronized in the current cycle can be distinguished, making the data synchronization operation more standardized and clear.

[0018] In an optional implementation, after adding the target log to the synchronization log file, the method further includes: When it is determined that the data synchronization operation of the current cycle is completed, the identification information of the target storage particle corresponding to the target request is determined according to the target logical start address and the preset data length; The target logical start address and the target data length included in the target log are converted into identification information of the target storage particle, so as to perform a data synchronization operation of the next cycle corresponding to the current cycle.

[0019] Specifically, under the periodic data synchronization mechanism, when the data synchronization operation of the current cycle is completed, the target log can be converted from the target logical start address and the target data length to the identification information of the target storage particle, so as to facilitate the data synchronization operation of the next cycle.

[0020] In an optional embodiment, the method further includes: When it is determined that the data synchronization operation corresponding to the target request fails, determining identification information of the target storage particle corresponding to the target request according to the target logical start address and the preset data length; The identification information of the target storage granule is added to the target log to indicate to re-execute the data synchronization operation corresponding to the target request.

[0021] Specifically, in a real-time data synchronization mechanism, if the data synchronization operation fails, the target log can be converted from the target logical start address and the target data length to the identification information of the target storage granule. In this way, it is convenient to carry out comprehensive processing of subsequent requests for failed data synchronization operations in units of storage granules, saving network resources and computing resources while ensuring that the failure is handled.

[0022] In an optional implementation, after adding the identification information of the target storage granule to the target log, the method further includes: Start timing; When the timing duration reaches the preset timing duration, each log included in the synchronization log file is traversed; Each time a log is traversed, determine whether the traversed log contains identification information of the storage granule; When it is determined that the traversed log includes identification information of the storage granule, the traversed log is determined as a candidate log; When at least one candidate log is determined, each candidate log is classified according to identification information of the storage granules included in each candidate log to obtain at least one log group, wherein identification information of the storage granules respectively included in all candidate logs belonging to the same log group is the same; According to identification information of the storage granule corresponding to the first log group, a data synchronization operation of the storage granule corresponding to the first log group is performed, wherein the first log group is any one of the at least one log group.

[0023] Specifically, if the data synchronization operation fails, it may indicate that the primary storage system or the secondary storage system has a fault. The nodes of the primary storage system can perform timing, and during the timing process, the fault may be repaired. Therefore, when the timing duration reaches the preset timing duration, the log is traversed again, and the data synchronization operation is performed on the failed request to ensure data consistency between the primary storage system and the secondary storage system.

[0024] In an optional embodiment, the method further includes: When it is determined that the data synchronization operation on the storage granule corresponding to the first log group is completed, each candidate log included in the first log group is deleted from the synchronization log file.

[0025] Specifically, when it is determined that the data synchronization operation on the storage granules corresponding to the first log group is completed, each candidate log included in the first log group is deleted from the synchronization log file, so that storage resources can be freed up in time.

[0026] In an optional implementation, when the current node is a master node in the master storage system, the method further includes: When it is detected that any node other than the master node in the primary storage system fails, identification information of the failed node is obtained; Determine a first candidate node corresponding to the faulty node according to the identification information of the faulty node and the mapping relationship between the identification information of the pre-built management node and the identification information of the candidate node, wherein the first candidate node is any node in the main storage system except the faulty node; The management right of the synchronization log file corresponding to the failed node is transferred to the first candidate node.

[0027] Specifically, after a node fails, the master node can promptly determine the first candidate node corresponding to the failed node, and the first candidate node promptly takes over the failed node as the log file managed by the management node, and can accurately mark the difference data during the failure of the failed node. At present, in order to ensure that the difference data can be correctly marked even in the case of a failure, after any bit in the difference bitmap of the management node changes, the changed difference bitmap needs to be synchronized to the corresponding candidate node (that is, the candidate node needs to back up the difference bitmap and update it in real time). In this way, two interactions are required for each request to be processed, resulting in more network resources being occupied. However, after a management node fails, this solution only needs to transfer the management rights of the corresponding synchronization log file, without the need for multiple interactions, which is more convenient and can save network resources.

[0028] In an optional embodiment, the method further includes: Determine the target management node corresponding to the faulty node according to the identification information and mapping relationship of the faulty node; According to the pre-built fault handling mechanism, a second candidate node is selected from other nodes in the primary storage system except the target management node and the faulty node; The second candidate node is determined as the candidate node corresponding to the target management node.

[0029] Specifically, for log files with a failed node as a candidate node, a new candidate node is timely reselected for the log file to avoid the problem of being unable to perform data synchronization operations due to no node managing the log file when the management node corresponding to the log file fails.

[0030] In an optional embodiment, the method further includes: When performing data synchronization operation on the primary storage system and the secondary storage system for the first time, traverse each storage granule managed by the current node; Each time a storage particle is traversed, the data stored in the traversed storage particle is synchronized to the standby storage particle corresponding to the traversed storage particle in the auxiliary storage system; When the data synchronization operation of each storage particle is completed, the value of the pre-built synchronization progress variable is updated according to the preset data length; When it is determined that the value of the synchronization progress variable is equal to a preset threshold, the data synchronization operation is stopped, wherein the preset threshold is the total amount of data stored in all storage particles managed by the current node.

[0031] Specifically, when the data synchronization operation is performed for the first time, there is no need to use a difference bitmap, and the data synchronization progress can be understood by updating the synchronization progress variable, which is more convenient.

[0032] In a second aspect, the present invention provides a device for updating a synchronization log, the device comprising: The acquisition module is used to parse the target request when the target request is acquired, and acquire the data operation information corresponding to the target request; A generation module, used to generate a target log corresponding to a target request according to a pre-built data synchronization mechanism and data operation information; The acquisition module is also used to obtain the identification information of the current node; A determination module, used to determine the synchronization log file corresponding to the identification information of the current node according to the identification information of the current node; Add module for adding target log to sync log file; The sending module is used to send synchronization indication information to the auxiliary node corresponding to the current node in the auxiliary storage system, wherein the synchronization indication information includes the data to be synchronized corresponding to the target log and the location information corresponding to the data to be synchronized, so that the auxiliary node performs data synchronization operation.

[0033] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the method for updating synchronization logs of the above-mentioned first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0034] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method for updating a synchronization log according to the first aspect or any corresponding embodiment thereof.

[0035] In a fifth aspect, the present invention provides a computer program product, including computer instructions, where the computer instructions are used to enable a computer to execute the method for updating a synchronization log in the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the related technologies, the drawings required for use in the specific embodiments or the related technical descriptions will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 is a schematic diagram of the architecture of a primary storage system according to an embodiment of the present invention; Figure 2 is a flowchart of a method for updating a synchronization log according to an embodiment of the present invention; Figure 3 is a schematic diagram of a synchronization log file under a real-time data synchronization mechanism according to an embodiment of the present invention; Figure 4 is a schematic diagram of a synchronization log file under a periodic data synchronization mechanism according to an embodiment of the present invention; Figure 5 is a schematic diagram of a synchronization log file under another periodic data synchronization mechanism according to an embodiment of the present invention; Figure 6 is a schematic diagram of a synchronization log file under another real-time data synchronization mechanism according to an embodiment of the present invention; Figure 7 is a flowchart of a method for determining a candidate node according to an embodiment of the present invention; Figure 8 is a schematic diagram of synchronization log files of a faulty node according to an embodiment of the present invention; Fig. 9 is a schematic diagram of a mapping relationship between identification information of a management node and identification information of a candidate node according to an embodiment of the present invention; Fig.10 is a flow chart of an initial data synchronization operation method according to an embodiment of the present invention; Fig.11 is a synchronization progress diagram of an initial data synchronization operation according to an embodiment of the present invention; Fig.12 is a structural block diagram of a device for updating a synchronization log according to an embodiment of the present invention; Fig.13 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0039] The embodiment of the present invention can be applied to a data synchronization system, which may include a primary storage system and an auxiliary storage system. Figure 1As shown, the primary storage system and the secondary storage system may include nodes and storage devices of the same data. Among the multiple nodes in the primary storage system, one node serves as a master node to manage the other nodes. The primary storage system may be a multi-controller storage system (MCS).

[0040] When the primary storage system is initialized, a log volume may be created in the storage device included in the primary storage system. The log volume may be provided with multiple log files. The number of log files may be the same as the number of nodes included in the primary storage system. Each node in the primary storage system may manage a log file. The identification information of the log file may include the identification information of the corresponding management node. Figure 1 As shown, node 1 in the primary storage system can manage log file 1. And the primary storage system can access all log files in the log volume. In addition, the storage device in the primary storage system also includes a storage volume (which can be called a primary volume) for storing data, and the storage volume can include multiple storage grains. Each node in the primary storage system is used to manage the data of some storage grains on the storage device of the primary storage system. The auxiliary storage system also includes a storage volume (which can be called an auxiliary volume), and each node in the auxiliary storage system is used to manage the data of some storage grains on the storage device of the auxiliary storage system.

[0041] For example, the identification information of a log volume may be "9999", the size may be "400GB", and it may be hidden from the user, that is, the user cannot use the log volume. The identification information of a log file may be "NODE1.LOG", indicating that the log file is a log file managed by node 1.

[0042] According to an embodiment of the present invention, a method embodiment for updating a synchronization log is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0043] In this embodiment, a method for updating a synchronization log is provided, which can be executed by any node (hereinafter referred to as the current node) included in the primary storage system. Figure 2 is a flowchart of a method for updating a synchronization log according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps: Step S201, when a target request is obtained, the target request is parsed to obtain data operation information corresponding to the target request.

[0044] Among them, the target request is any request obtained by the current node. The data synchronization mechanism can be a real-time data synchronization mechanism or a periodic data synchronization mechanism. The real-time data synchronization mechanism refers to performing data synchronization operations after completing a write operation on the main storage system for each request. The periodic data synchronization mechanism refers to the node periodically performing data synchronization operations according to a preset cycle length. The data operation information may include the target logical start address, the target data length (which may be called Size), and the operation data. The target logical start address may be a logical block address (Logical Block Address, LBA).

[0045] Specifically, the current node can receive a request sent from the client (this case will be used as an example later), or can generate a request according to its own business processing mechanism, so that the current node can obtain the request. Taking the target request as an example, after obtaining the target request, the current node can parse the target request and obtain data operation information.

[0046] After parsing and obtaining the data operation information, the current node can perform a write operation on the corresponding target storage location on the storage device included in the primary storage system according to the target logical start address, the target data length and the operation data. In addition, the current node can synchronize the newly written data in the target storage location to the corresponding storage location in the secondary storage system based on the remote copy (RC) relationship between the primary storage system and the secondary storage system. The remote copy relationship can include a synchronous remote copy relationship, an asynchronous remote copy relationship and a periodic remote copy relationship. In a synchronous remote copy relationship, after the current node completes the write operation corresponding to the request and synchronizes the data to the secondary storage system, it sends a request completion notification to the client. In an asynchronous remote copy relationship, after the current node completes the write operation corresponding to the request, it sends a request completion notification to the client and then performs the data synchronization operation. In a periodic remote copy relationship, after the current node completes the write operation corresponding to the request, it sends a request completion notification to the client and performs the data synchronization operation according to the set period duration.

[0047] In the process of executing the write operation on the target request, the write to the cache of the current node may fail. In this case, the current node can return a write failure notification to the client. Alternatively, in the process of data synchronization, the write to the auxiliary storage system may fail. In this case, as long as the write operation of the current node is successful, a write success notification can be sent to the client.

[0048] Step S202: Generate a target log corresponding to the target request according to the pre-built data synchronization mechanism and data operation information.

[0049] Specifically, based on different data synchronization mechanisms and data operation information, the following three situations can be handled: In the first case, when the data synchronization mechanism is a real-time data synchronization mechanism, the current node can generate a target log according to the target logical start address and the target data length. For example, Figure 3 All logs in the synchronization log file shown in include a logical start address and a data length. The data synchronization operation under this data synchronization mechanism can be performed based on the above-mentioned synchronous remote replication relationship or asynchronous remote replication relationship.

[0050] In the second case, when the data synchronization mechanism is a periodic data synchronization mechanism, it can be determined whether the current node is in a data synchronization state. When it is determined that the current node is not in a data synchronization state, the identification information (Grain Index) of the target storage grain corresponding to the target request is determined according to the target logical starting address and the preset data length. The target log is generated according to the identification information of the target storage grain. For example, Figure 4 All logs in the synchronization log file shown include identification information of storage particles. The data synchronization operation under this data synchronization mechanism can be performed based on the above-mentioned periodic remote replication relationship.

[0051] The data synchronization state may be used to indicate that the current node is performing a data synchronization operation in the current cycle. The identification information of the target storage particle may be obtained by rounding down the ratio of the target logical start address and the preset data length. The preset data length is the data length of the storage particle.

[0052] In the third case, when the data synchronization mechanism is a periodic data synchronization mechanism, it is determined whether the current node is in a data synchronization state. When it is determined that the current node is in a data synchronization state, a target log is generated according to the target logical start address and the target data length. Figure 5 As shown, some logs in the synchronization log file only include identification information of storage particles, and some logs only include logical start addresses and data lengths, the latter of which are logs corresponding to requests received during the execution of data synchronization operations in the current cycle.

[0053] Step S203, obtaining identification information of the current node.

[0054] Step S204: determining the synchronization log file corresponding to the identification information of the current node according to the identification information of the current node.

[0055] Specifically, the current node may determine the synchronization log file of the current node in the log volume of the storage device included in the primary storage system according to its own identification information.

[0056] Step S205: Add the target log to the synchronization log file.

[0057] Specifically, the current node may obtain the number of the latest log in the synchronization log file, and add one to the number to obtain the number of the target log, and then, according to the number of the target log, store the target log at a position corresponding to the number of the target log.

[0058] In some optional implementations, since the performance of directly writing logs to a storage device is not high, in order to speed up the processing, the current node may first record the target log in the cache of the current node. Then, according to a preset cache algorithm, when determining to execute the write operation, the target log in the cache is written to the synchronization log file of the storage device of the primary storage system. In this way, the efficiency of updating the synchronization log can be improved.

[0059] Step S206: Send synchronization indication information to the secondary node corresponding to the current node in the secondary storage system.

[0060] The synchronization indication information may include the data to be synchronized corresponding to the target log, and the location information corresponding to the data to be synchronized.

[0061] Specifically, based on different data synchronization mechanisms, sending synchronization indication information to the secondary node corresponding to the current node in the secondary storage system may include the following two situations: Case 1: When the data synchronization mechanism is a real-time data synchronization mechanism, the current node can generate synchronization indication information based on the parsed target logical starting address, target data length, and operation data (or, can only be based on the target logical starting address and operation data) and send it to the auxiliary node corresponding to the current node (for example, node 1 in the primary storage system sends synchronization indication information to node 1 in the auxiliary storage system). The operation data is the above-mentioned data to be synchronized, and the target logical starting address and target data length are the location information corresponding to the data to be synchronized. The auxiliary node determines the corresponding storage location on the storage device included in the auxiliary storage system based on the target logical starting address and target data length included in the synchronization indication information (or, only based on the target logical starting address), and writes the operation data, thereby completing the data synchronization operation corresponding to the target request.

[0062] Case 2: When the data synchronization mechanism is a periodic data synchronization mechanism, the current node can start traversing the synchronization log file at the periodic triggering moment. Each time a log is traversed, it is determined whether the traversed log includes the identification information of the storage granule. When it is determined that the traversed log includes the identification information of the storage granule, the traversed log is determined as a candidate log. When at least one candidate log is determined, each candidate log is classified according to the identification information of the storage granule included in each candidate log to obtain at least one log group. According to the identification information of the storage granule corresponding to the target log group, data is extracted from the storage granule corresponding to the target log group, and according to the identification information of the storage granule and the data extracted from the storage granule corresponding to the target log group, synchronization indication information is generated and sent to the auxiliary node corresponding to the current node. Among them, the target log group is any one of the log groups in at least one log group. When the target log group includes the target log, the data extracted from the storage granule corresponding to the target log group includes the data to be synchronized corresponding to the target log, and the location information corresponding to the data to be synchronized is the identification information of the storage granule corresponding to the target log group.

[0063] It should be noted that when the data synchronization mechanism is a periodic data synchronization mechanism, since all logs included in the synchronization log file at the triggering moment of the current cycle include the identification information of the storage particles, the data synchronization operation also takes a certain amount of time, and new requests may be received during this data synchronization process. In order to distinguish the data corresponding to the newly received request from the periodically synchronized data, these logs will be recorded including the data length and the starting address to mark that the data corresponding to these logs do not participate in the current data synchronization operation, but participate in the data synchronization operation of the next cycle. Therefore, during the traversal process, only the logs including the storage particle identification information are traversed during the data synchronization process of the current cycle.

[0064] In some optional implementations, after sending the synchronization indication information to the secondary node corresponding to the current node in the secondary storage system, the current node may perform the following operations: receiving target response information fed back by the secondary node, wherein the target response information is response information generated by the secondary node after completing the data synchronization operation according to the synchronization indication information; Delete the target log from the synchronization log file according to the target response information.

[0065] Specifically, when the data synchronization mechanism is a real-time type of synchronization log, when the target response information fed back by the auxiliary node is received, it can be determined that the data synchronization operation corresponding to the target request is successful, and at this time, the target log can be deleted from the synchronization log file. Alternatively, when the data synchronization mechanism is a real-time type of synchronization log, after receiving the response information fed back by the auxiliary node, it is determined that the data synchronization operation of the storage granule corresponding to the target log group is completed, and all logs belonging to the target log group in the synchronization log file can be deleted.

[0066] In some optional implementations, under a real-time synchronization mechanism, when the data synchronization operation corresponding to the target request fails (a case where writing to the cache of the current node fails, or writing to the auxiliary storage system fails, for example, no response information is received within a preset time period), the following processing may be performed: Step 1: Determine identification information of a target storage particle corresponding to the target request according to a target logical start address and a preset data length.

[0067] Step 2: Add the identification information of the target storage granule to the target log to indicate to re-execute the data synchronization operation corresponding to the target request.

[0068] For example, in Figure 6 In the log, the first and second logs are the logs corresponding to the requests that failed the data synchronization operation.

[0069] In some optional implementations, after the identification information of the target storage granule is added to the target log, the current node may set the current remote replication relationship to a resynchronization state. At this time, all nodes in the primary storage system enter the resynchronization state and perform the following specific operations (that is, the following operations may be performed after the above step 3): Step 1: Start timing.

[0070] Step 2: When the timing duration reaches a preset timing duration, each log included in the synchronization log file is traversed.

[0071] For example, the preset timing duration may be 7.5 minutes.

[0072] Step 3: Each time a log is traversed, determine whether the traversed log includes identification information of the storage granule.

[0073] Step 4: When it is determined that the traversed log includes identification information of the storage granule, the traversed log is determined as a candidate log.

[0074] Step 5: When at least one candidate log is determined, each candidate log is classified according to the identification information of the storage granule included in each candidate log to obtain at least one log group.

[0075] Among them, all candidate logs belonging to the same log group respectively include the same identification information of storage particles.

[0076] Step six: perform a data synchronization operation on the storage granule corresponding to the first log group according to the identification information of the storage granule corresponding to the first log group.

[0077] The first log group may be any log group of the at least one log group.

[0078] In some optional implementations, in order to save resources, when it is determined that the data synchronization operation on the storage granule corresponding to the first log group is completed, the current node may delete each candidate log included in the first log group from the synchronization log file.

[0079] For the second and third cases in step S202 above, the current node may perform the following specific steps: Step 1: when it is determined that the data synchronization operation of the current cycle is completed, the identification information of the target storage particle corresponding to the target request is determined according to the target logical start address and the preset data length.

[0080] Step 2: Convert the target logical start address and target data length in the target log into identification information of the target storage particle and add it to the target log to perform the data synchronization operation of the next cycle corresponding to the current cycle.

[0081] The method for updating the synchronization log provided by this embodiment can parse the request and obtain data operation information every time a request is obtained. In addition, a log can be generated according to the data synchronization mechanism of the current node and the parsed data operation information. In addition, the scheme determines the corresponding synchronization log file according to the identification information of the current node. Further, the current node can add the generated log to the synchronization log file to mark the data corresponding to the target log as the difference data between the primary storage system and the auxiliary storage system, so as to replace the role of using the difference bitmap to mark the difference data in the related technology. In this way, for multiple consecutive requests for different storage locations of the same storage particle, the scheme can record a log for each request in the synchronization log file, that is, the scheme can mark the difference data in a dimension smaller than the storage particle, and execute the synchronization operations corresponding to multiple requests for different storage locations of the same storage particle in parallel, which can improve the request processing efficiency.

[0082] For the master node, the master node can monitor whether each node included in the main storage system fails, and transfer the log file managed by the failed node to the corresponding candidate node. Accordingly, in this embodiment, a method for determining the candidate node is provided, which can be executed by the master node included in the main storage system (the above-mentioned current node can be the master node). Figure 7 is a flowchart of a method for determining candidate nodes according to an embodiment of the present invention. Figure 7 As shown, the process includes the following steps: Step S701: when it is detected that any node other than the master node in the primary storage system fails, identification information of the failed node is obtained.

[0083] Specifically, when the master node detects that any node fails, it can obtain identification information of the failed node.

[0084] Step S702: Determine a first candidate node corresponding to the faulty node according to the identification information of the faulty node and the pre-built mapping relationship between the identification information of the management node and the identification information of the candidate node.

[0085] The first candidate node is any node in the main storage system except the failed node.

[0086] Specifically, the above mapping relationship is as follows: Figure 8 As shown, when the identification information of the faulty node is node 1, it can be determined according to the mapping relationship that the identification information of the candidate node corresponding to node 1 is node 2, and the current node can determine node 2 as the first candidate node.

[0087] Step S703: transferring the management right of the synchronization log file corresponding to the failed node to the first candidate node.

[0088] Specifically, the current node can generate a management right transfer notification based on the identification information of the faulty node and send it to the first candidate node. In this way, after receiving the management right transfer notification, the first candidate node can parse the management right transfer notification and obtain the identification information of the faulty node. The first candidate node can perform management operations on the synchronization log file corresponding to the identification information of the faulty node in the storage device included in the primary storage system based on the identification information of the faulty node.

[0089] In some optional implementations, when the data synchronization mechanism is a real-time data synchronization mechanism and the first candidate node is the current node, the current node may perform the following operations: According to the identification information of the faulty node, determine the synchronization log file corresponding to the identification information of the faulty node. According to the first logical starting address and the first data length included in the first log, calculate the identification information of the first storage particle (similar to the process of calculating the identification information of the target storage particle, which will not be repeated here), and add the identification information of the first storage particle to the first log. The first log is any log in the synchronization log file corresponding to the identification information of the faulty node that does not include the identification information of the storage particle. The first candidate node can immediately traverse all the logs in the synchronization log file corresponding to the identification information of the faulty node, and perform grouping operations according to the identification information of the storage particles in each log. The current node can perform data synchronization operations on the storage particles corresponding to each group in units of storage particles. For example, Figure 8 It shows that after node 1 fails, all logs in the synchronization log file will include the logical start address, data length, and storage granule identification information.

[0090] In this way, after the management right is transferred to the first candidate node, the first candidate node can complete the unfinished data synchronization operation in time to ensure the data consistency between the primary storage system and the auxiliary storage system.

[0091] In some optional implementations, the failed node may serve as a candidate node for other nodes in addition to being a management node. Therefore, the current node may reselect a candidate node for the failed node as a management node corresponding to the candidate node through the following specific steps: Step 1: Determine the target management node corresponding to the faulty node according to the identification information and mapping relationship of the faulty node.

[0092] Step 2: According to the pre-built fault handling mechanism, a second candidate node is selected from other nodes in the primary storage system except the target management node and the faulty node.

[0093] Step three: determine the second candidate node as the candidate node corresponding to the target management node.

[0094] Specifically, the current node can, based on the identification information of the faulty node, Fig. 9 In the mapping relationship shown, the identification information of the corresponding target management node is determined. For example, according to node 1, the corresponding target management node is determined to be node 3.

[0095] When the current node selects a candidate node for the target management node, it can first determine the number of log files managed by each candidate node (other nodes except the target management node and the faulty node), and select the candidate node with the least number of log files managed as the candidate node corresponding to the target management node.

[0096] The method for determining the candidate node provided in this embodiment is that after a node fails, the main node can promptly determine the first candidate node corresponding to the failed node, and the first candidate node promptly takes over the failed node as the log file managed by the management node, and accurately marks the difference data during the failure of the failed node. At present, in order to ensure that the difference data can also be marked in the case of a failure, after any bit in the difference bitmap of the management node changes, the changed difference bitmap needs to be synchronized to the corresponding candidate node (that is, the candidate node needs to back up the difference bitmap and update it in real time). In this way, two interactions are required for each request to be processed, resulting in more network resources being occupied. However, after a management node fails, this solution only needs to transfer the management rights of the corresponding synchronization log file, without the need for multiple interactions, which is more convenient and can save network resources.

[0097] In the case of using an update synchronization log to perform a data synchronization operation in a write request process, an initial data synchronization operation method is provided in this embodiment, which can be performed by any node included in the primary storage system. Fig.10 is a flow chart of a data synchronization operation method according to an embodiment of the present invention. Fig.10 As shown, the process includes the following steps: Step S1001, when performing a data synchronization operation on a primary storage system and a secondary storage system for the first time, traverse each storage particle managed by the current node.

[0098] The first data synchronization operation is also generally referred to as an initial synchronization operation.

[0099] Step S1002: every time a storage particle is traversed, the data stored in the traversed storage particle is synchronized to the standby storage particle corresponding to the traversed storage particle in the auxiliary storage system.

[0100] Step S1003: when the data synchronization operation of each storage particle is completed, the value of the pre-built synchronization progress variable is updated according to the preset data length.

[0101] Among them, the synchronization progress variable can be called a cursor.

[0102] Specifically, a synchronization progress variable may be set in each node, and the synchronization progress variable may be set with an initial value and a preset threshold value, which is used to indicate the storage location range of the storage particles managed by the node in the storage device included in the main storage system. For example, node 1: 0-128K, node 2: 128K-256K, node 3: 256K-384K, node 4: 384K-512K.

[0103] After each data synchronization operation of a storage particle is completed, the current node can add the preset data length and the value of the synchronization progress variable to obtain the updated value of the synchronization progress variable. Fig.11 As shown, it shows the synchronization progress in node 1 at 50%, where the black filled box represents the part that has completed the initial data synchronization operation, the white filled box represents the part that has not completed the initial data synchronization operation, and the arrow represents the cursor.

[0104] Step S1004: when it is determined that the value of the synchronization progress variable is equal to a preset threshold, the data synchronization operation is stopped.

[0105] The preset threshold is the total amount of data stored in all storage particles managed by the current node.

[0106] The data synchronization operation method provided in this embodiment does not need to use a difference bitmap when the data synchronization operation is performed for the first time. The data synchronization progress can be understood by updating a synchronization progress variable, which is more convenient.

[0107] In this embodiment, a device for updating the synchronization log is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0108] This embodiment provides a device for updating a synchronization log, such as Fig.12 As shown, including: The acquisition module 1201 is used to parse the target request when the target request is acquired, and acquire data operation information corresponding to the target request; A generating module 1202, configured to generate a target log corresponding to a target request according to a pre-built data synchronization mechanism and data operation information; The acquisition module 1201 is also used to obtain the identification information of the current node; The determination module 1203 is used to determine the synchronization log file corresponding to the identification information of the current node according to the identification information of the current node; An adding module 1204 is used to add a target log to a synchronization log file; The sending module 1205 is used to send synchronization indication information to the secondary node corresponding to the current node in the secondary storage system, wherein the synchronization indication information includes the data to be synchronized corresponding to the target log and the location information corresponding to the data to be synchronized, so that the secondary node performs data synchronization operation.

[0109] In some optional implementations, the device further includes a deletion module 1206, configured to: receiving target response information fed back by the secondary node, wherein the target response information is response information generated by the secondary node after completing the data synchronization operation according to the synchronization indication information; Delete the target log from the synchronization log file according to the target response information.

[0110] In some optional implementations, the data operation information includes a target logical start address and a target data length.

[0111] In some optional implementations, the generating module 1202 is specifically configured to: When the data synchronization mechanism is a real-time type data synchronization mechanism, a target log is generated according to the target logical start address and the target data length.

[0112] In some optional implementations, the generating module 1202 is specifically configured to: When the data synchronization mechanism is a periodic data synchronization mechanism, determining whether the current node is in a data synchronization state, wherein the data synchronization state is used to indicate that the current node is performing a data synchronization operation of the current cycle; When it is determined that the current node is not in a data synchronization state, determining identification information of a target storage particle corresponding to the target request according to a target logical start address and a preset data length, wherein the preset data length is a data length of the storage particle; Generate a target log according to the identification information of the target storage granule.

[0113] In some optional implementations, the generating module 1202 is specifically configured to: When it is determined that the current node is in a data synchronization state, a target log is generated according to the target logical start address and the target data length.

[0114] In some optional embodiments, the device further includes a monitoring module 1207, which is used to: When it is determined that the data synchronization operation of the current cycle is completed, the identification information of the target storage particle corresponding to the target request is determined according to the target logical start address and the preset data length; The target logical start address and the target data length included in the target log are converted into identification information of the target storage particle, so as to perform a data synchronization operation of the next cycle corresponding to the current cycle.

[0115] In some optional implementations, the determination module 1203 is further configured to determine identification information of a target storage particle corresponding to the target request according to a target logical start address and a preset data length when it is determined that the data synchronization operation corresponding to the target request fails; The adding module 1204 is further configured to add the identification information of the target storage granule to the target log, so as to indicate to re-execute the data synchronization operation corresponding to the target request.

[0116] In some optional implementations, the device further includes a synchronization module 1208, configured to: Start timing; When the timing duration reaches the preset timing duration, each log included in the synchronization log file is traversed; Each time a log is traversed, determine whether the traversed log contains identification information of the storage granule; When it is determined that the traversed log includes identification information of the storage granule, the traversed log is determined as a candidate log; When at least one candidate log is determined, each candidate log is classified according to identification information of the storage granules included in each candidate log to obtain at least one log group, wherein identification information of the storage granules respectively included in all candidate logs belonging to the same log group is the same; According to identification information of the storage granule corresponding to the first log group, a data synchronization operation of the storage granule corresponding to the first log group is performed, wherein the first log group is any one of the at least one log group.

[0117] In some optional implementations, when the current node is a master node in the master storage system, the method further includes: When it is detected that any node other than the master node in the primary storage system fails, identification information of the failed node is obtained; Determine a first candidate node corresponding to the faulty node according to the identification information of the faulty node and the mapping relationship between the identification information of the pre-built management node and the identification information of the candidate node, wherein the first candidate node is any node in the main storage system except the faulty node; The management right of the synchronization log file corresponding to the failed node is transferred to the first candidate node.

[0118] In some optional implementations, the fault processing module 1209 is further configured to: Determine the target management node corresponding to the faulty node according to the identification information and mapping relationship of the faulty node; According to the pre-built fault handling mechanism, a second candidate node is selected from other nodes in the primary storage system except the target management node and the faulty node; The second candidate node is determined as the candidate node corresponding to the target management node.

[0119] In some optional implementations, the synchronization module 1208 is further configured to: When performing data synchronization operation on the primary storage system and the secondary storage system for the first time, traverse each storage granule managed by the current node; Each time a storage particle is traversed, the data stored in the traversed storage particle is synchronized to the standby storage particle corresponding to the traversed storage particle in the auxiliary storage system; When the data synchronization operation of each storage particle is completed, the value of the pre-built synchronization progress variable is updated according to the preset data length; When it is determined that the value of the synchronization progress variable is equal to a preset threshold, the data synchronization operation is stopped, wherein the preset threshold is the total amount of data stored in all storage particles managed by the current node.

[0120] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0121] The device for updating the synchronization log in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0122] The embodiment of the present invention also provides a computer device having the above Fig.12 The device shown updates the synchronization log.

[0123] See also Fig.13 , Fig.13 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Fig.13 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig.13 A processor 10 is taken as an example.

[0124] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware integrated circuit. The hardware integrated circuit may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable logic gate array, a general purpose array logic or any combination thereof.

[0125] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0126] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0127] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0128] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0129] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0130] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0131] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for updating a synchronization log, characterized in that: The method is applied to a data synchronization system, the data synchronization system includes a primary storage system and an auxiliary storage system, the primary storage system and the auxiliary storage system respectively include a plurality of nodes of the same number, the method is executed by a current node, the current node is any one of the plurality of nodes included in the primary storage system, and the method includes: When a target request is obtained, the target request is parsed to obtain data operation information corresponding to the target request; Generate a target log corresponding to the target request according to the pre-built data synchronization mechanism and the data operation information; Get the identification information of the current node; Determine, according to the identification information of the current node, a synchronization log file corresponding to the identification information of the current node; Adding the target log to the synchronization log file; Send synchronization indication information to the secondary node corresponding to the current node in the secondary storage system, wherein the synchronization indication information includes the data to be synchronized corresponding to the target log and the location information corresponding to the data to be synchronized, so that the secondary node performs a data synchronization operation.

2. The method according to claim 1, characterized in that After sending the synchronization indication information to the secondary node corresponding to the current node in the secondary storage system, the method further includes: receiving target response information fed back by the secondary node, wherein the target response information is response information generated by the secondary node after completing the data synchronization operation according to the synchronization indication information; The target log is deleted from the synchronization log file according to the target response information.

3. The method according to claim 1, characterized in that The data operation information includes a target logical start address and a target data length.

4. The method according to claim 3, characterized in that: The generating a target log corresponding to the target request according to the pre-built data synchronization mechanism and the data operation information includes: When the data synchronization mechanism is a real-time data synchronization mechanism, the target log is generated according to the target logical start address and the target data length.

5. The method according to claim 3, characterized in that: The generating a target log corresponding to the target request according to the pre-built data synchronization mechanism and the data operation information includes: When the data synchronization mechanism is a periodic data synchronization mechanism, determining whether the current node is in a data synchronization state, wherein the data synchronization state is used to indicate that the current node is performing a data synchronization operation of a current cycle; When it is determined that the current node is not in the data synchronization state, determining identification information of a target storage particle corresponding to the target request according to the target logical start address and a preset data length, wherein the preset data length is a data length of a storage particle; The target log is generated according to the identification information of the target storage particle.

6. The method according to claim 5, characterized in that The method further comprises: When it is determined that the current node is in the data synchronization state, the target log is generated according to the target logical start address and the target data length.

7. The method according to claim 6, characterized in that After adding the target log to the synchronization log file, the method further includes: When it is determined that the data synchronization operation of the current cycle is completed, determining identification information of a target storage particle corresponding to the target request according to the target logical start address and the preset data length; The target logical start address and the target data length included in the target log are converted into identification information of the target storage particle, so as to perform a data synchronization operation of a next cycle corresponding to the current cycle.

8. The method according to claim 4, characterized in that The method further comprises: When it is determined that the data synchronization operation corresponding to the target request fails, determining identification information of a target storage particle corresponding to the target request according to the target logical start address and a preset data length; The identification information of the target storage granule is added to the target log to indicate to re-execute the data synchronization operation corresponding to the target request.

9. The method according to claim 8, characterized in that After adding the identification information of the target storage granule to the target log, the method further includes: Start timing; When the timing duration reaches a preset timing duration, each log included in the synchronization log file is traversed; Each time a log is traversed, determine whether the traversed log contains identification information of the storage granule; When it is determined that the traversed log includes identification information of the storage granule, determining the traversed log as a candidate log; When at least one candidate log is determined, each candidate log is classified according to the identification information of the storage granules included in each candidate log to obtain at least one log group, wherein the identification information of the storage granules respectively included in all candidate logs belonging to the same log group is the same; According to identification information of storage particles corresponding to a first log group, a data synchronization operation of the storage particles corresponding to the first log group is performed, wherein the first log group is any one of the at least one log group.

10. The method according to any one of claims 1 to 9, characterized in that: When the current node is a master node in the primary storage system, the method further includes: When it is detected that any node other than the master node in the primary storage system fails, identification information of the failed node is obtained; Determine, according to the identification information of the faulty node and the mapping relationship between the identification information of the pre-built management node and the identification information of the candidate node, a first candidate node corresponding to the faulty node, wherein the first candidate node is any node among other nodes in the primary storage system except the faulty node; The management right of the synchronization log file corresponding to the failed node is transferred to the first candidate node.

11. The method according to claim 10, characterized in that The method further comprises: Determine a target management node corresponding to the faulty node according to the identification information of the faulty node and the mapping relationship; According to a pre-built fault handling mechanism, selecting a second candidate node from other nodes in the primary storage system except the target management node and the faulty node; The second candidate node is determined as a candidate node corresponding to the target management node.

12. The method according to any one of claims 1 to 9, characterized in that: The method further comprises: When performing a data synchronization operation on the primary storage system and the secondary storage system for the first time, traversing each storage granule managed by the current node; Each time a storage particle is traversed, the data stored in the traversed storage particle is synchronized to a standby storage particle corresponding to the traversed storage particle in the auxiliary storage system; When each data synchronization operation of the storage particle is completed, the value of the pre-built synchronization progress variable is updated according to the preset data length; When it is determined that the value of the synchronization progress variable is equal to a preset threshold, the data synchronization operation is stopped, wherein the preset threshold is the total amount of data stored in all storage particles managed by the current node.

13. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for updating the synchronization log according to any one of claims 1 to 12 by executing the computer instructions.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method for updating a synchronization log according to any one of claims 1 to 12.

15. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are used to cause a computer to execute the method for updating a synchronization log according to any one of claims 1 to 12.

Citation Information

Patent Citations

  • Data synchronization method and device and distributed storage system

    CN108616598A

  • Synchronous remote replication method and device of storage system

    CN118550761A