Non-attribution mirror image volume data synchronization method and device, computer equipment and storage medium

Abstract

The invention relates to a non-attribution mirror image volume data synchronization method and device, computer equipment and a storage medium. The method comprises the steps that fault information is obtained in response to the fact that an affiliated main node conducts data read-write processing on a mirror image volume and breaks down, the fault information comprises incomplete operation and a non-affiliated node list, and the non-affiliated node list comprises a plurality of non-affiliated nodes; according to the fault information, integrating the incomplete operations to obtain a global incomplete operation list; based on the global incomplete operation record table, the multiple non-attribution nodes take over the incomplete operation; in response to the fact that all the inexhaustible operations are completed, performing home domain redivision on the non-home node to obtain a new master node; and completing data synchronization and state updating of the non-attribution mirror image volume through the new main node. And completing data synchronization and state updating of the mirror image volume through the new main node. By adopting the method, the data synchronization efficiency can be improved and the data consistency can be ensured.

Description

Technical Field

[0001] The present application relates to the field of computer storage technology, and in particular to a method, device, computer equipment and storage medium for synchronizing data of a non-owned mirror volume. Background Art

[0002] Non-attributed mirror volumes are a flexible data management method in modern distributed storage systems and are widely used in high availability and data redundancy scenarios. In a non-attributed mirror volume, volume data is stored on multiple nodes at the same time, and these nodes do not have a fixed affiliation node, that is, there is no unique node that controls all read and write operations of the volume. This design allows multiple nodes to take over the volume simultaneously or dynamically to avoid single point failures.

[0003] In this architecture, when a system failure occurs, other nodes can quickly take over and restore consistency through data synchronization. However, after a node fails, how to ensure that the mirror volume data of each node can remain consistent during data recovery is a key issue. Non-attributed distributed systems require an efficient synchronization mechanism to ensure that no data inconsistencies occur during data recovery. At the same time, data synchronization of non-attributed mirror volumes requires efficient algorithms to reduce the overhead of data transmission, especially when performing differential synchronization after failure recovery. How to achieve fast transmission of changed data and ensure data consistency without affecting the overall system performance is a technical problem that urgently needs to be solved. Summary of the invention

[0004] Based on this, it is necessary to provide a data synchronization method, device, computer equipment and storage medium that can improve data synchronization efficiency and ensure data consistency in response to the above technical problems.

[0005] On the one hand, a method for synchronizing data of a non-attributed mirror volume is provided, the method comprising:

[0006] In response to a failure in data reading and writing processing of a mirror volume by an owned master node, failure information is obtained, wherein the failure information includes: an unfinished operation and a list of unowned nodes, wherein the list of unowned nodes includes a plurality of unowned nodes;

[0007] According to the fault information, the unfinished operations are integrated to obtain a global unfinished operation list;

[0008] Based on the global unfinished operation record table, the plurality of unattached nodes take over the unfinished operations;

[0009] In response to the completion of all the unfinished operations, the belonging domain of the non-belonging node is re-divided to obtain a new master node; and data synchronization and status update of the non-belonging mirror volume are completed through the new master node.

[0010] In one of the embodiments, re-dividing the home domain of the non-home node to obtain a new master node includes:

[0011] Re-dividing the mirror volume into a plurality of attribution domains;

[0012] The non-attributed nodes are numbered and allocated to each attribution domain in sequence according to the numbering sequence; log resources are divided according to the re-divided attribution domains, and each attribution domain corresponds to a copy of the log resources. In one embodiment, the method further includes:

[0013] Detecting the heartbeat signal sending operation of the owned master node;

[0014] In response to the belonging master node not sending a heartbeat signal, determining that a failure occurs in the data reading and writing processing of the mirror volume by the belonging master node;

[0015] The unfinished operations and the list of unattached nodes are collected, and the information is broadcasted to all nodes and reported.

[0016] In one embodiment, according to the fault information, the unfinished operations are integrated to obtain a global unfinished operation list, including:

[0017] Scanning logs through the plurality of non-attributable nodes to obtain unfinished operations of the plurality of faulty nodes;

[0018] Extract the unfinished operations, integrate multiple unfinished operation record tables and report them;

[0019] Integrate all the unfinished operation record tables to generate a global unfinished operation record table;

[0020] Eliminate duplicate operations in the global unfinished operation record table, and update the global unfinished operation record table. In one embodiment, based on the global unfinished operation record table, a plurality of the homeless nodes take over the unfinished operations, including:

[0021] The homeless node reads a global unfinished operation record table, wherein the global unfinished operation record table includes at least one of the following: an operation type, a logical block address block range, an operation status, and an operation timestamp, the operation type includes: a read operation and a write operation, and the operation status includes: partially completed and uncompleted;

[0022] The homeless node sequentially recovers each unfinished operation according to the unfinished operation type and the unfinished operation state, and executes the unfinished operation in the order of the operation timestamps, wherein the unfinished operation includes at least one of the following: a read operation on the primary copy, a write operation on the primary copy, and marking data differences in a bitmap;

[0023] In response to resuming the write operation to the primary copy, the homeless node checks whether the operation has been recorded in the log;

[0024] In response to the write operation portion of the primary copy having been written, the homeless node continues to complete the remaining write operation from the interrupted portion;

[0025] In response to the primary copy write operation not starting, the non-home node performs a new write operation according to the data in the unfinished operation record table;

[0026] In response to the writing operation of the primary copy being completed, the update operation status is completed, and the update marks the data difference in the bitmap, records it in the log, and reports it;

[0027] In response to resuming the read operation on the primary copy, the homeless node initiates a read request for the logical block address range according to the data in the unfinished operation record table to obtain the required data;

[0028] In response to the owned master node reading part of the data, the unowned node continues reading from the interrupted part, records and reports the complete read data in a log, and updates the current read operation status as completed.

[0029] In one of the embodiments, the method includes: re-dividing the home domain of the non-home node to obtain a new master node, including:

[0030] Extracting node status information reported by the unattached node from log resources of the unattached node corresponding to the assigned domain; judging the status and load capacity of each unattached node according to the node status information reported by the unattached node;

[0031] The non-home nodes are sorted according to a load balancing algorithm, and the best non-home node is obtained as the new master node;

[0032] Identify the data scope that each homeless node is responsible for and its new role;

[0033] The new home domain allocation and the new master node information are recorded in the log.

[0034] In one embodiment, the method includes: completing data synchronization and status update of the mirror volume through the new master node, including:

[0035] The new master node obtains a data mark position difference block between the primary copy and the secondary copy by scanning the bitmap information;

[0036] Transmitting the identified data mark position difference block from the primary copy to the secondary copy, and clearing the bitmap information of the completed transmission;

[0037] The transmitted bitmap information is updated, and the state of the mirror volume is updated to a data available state.

[0038] Another aspect provides a data synchronization device, the device comprising:

[0039] The fault information acquisition module is used to acquire fault information in response to a fault in data reading and writing processing of a mirror volume by an owned master node, wherein the fault information includes: unfinished operations and a list of unowned nodes, wherein the list of unowned nodes includes multiple unowned nodes.

[0040] The list acquisition module is used to integrate the unfinished operations according to the fault information to obtain a global unfinished operation list. The operation takeover module is used to allow multiple non-attributed nodes to take over the unfinished operations based on the global unfinished operation record table.

[0041] The home domain division module is used to, in response to the completion of all the unfinished operations, re-divide the home domain of the non-home node to obtain a new master node.

[0042] The data synchronization module is used to complete the data synchronization and status update of the unattached mirror volume through the new master node.

[0043] In another aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0044] In response to a failure in data reading and writing processing of the mirror volume by the belonging master node, acquiring failure information;

[0045] According to the fault information, the unfinished operations are integrated and divided according to the node domain to obtain an unfinished operation record table;

[0046] Based on the unfinished operation record table, taking over the unfinished operations through the remaining non-attributed nodes;

[0047] In response to the unfinished operations being completed, reorganizing according to the domains to which the remaining nodes belong to obtain a new master node;

[0048] The data synchronization and status update of the mirror volume are completed through the new master node.

[0049] In another aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0050] In response to a failure in data reading and writing processing of the mirror volume by the belonging master node, acquiring failure information;

[0051] According to the fault information, the unfinished operations are integrated and divided according to the node domain to obtain an unfinished operation record table;

[0052] Based on the unfinished operation record table, taking over the unfinished operations through the remaining non-attributed nodes;

[0053] In response to the unfinished operations being completed, reorganizing according to the domains to which the remaining nodes belong to obtain a new master node;

[0054] The data synchronization and status update of the mirror volume are completed through the new master node.

[0055] In the above data recovery method, when the owned master node fails, the system can quickly identify the current unfinished operations by immediately obtaining the fault information and integrating the unfinished operations. By constructing a global unfinished operation record table, it is ensured that each unfinished operation has a clear allocation, so that the unowned node can quickly take over the task, avoiding long-term interruption of data reading and writing, thereby ensuring the high availability of the system. In the process of fault handling, the unfinished operations will be accurately recorded and assigned to the appropriate unowned node for processing, ensuring that all unfinished data operations of the original master node will not be lost or omitted. Through orderly takeover, the integrity and consistency of the operation can be guaranteed, and the risk of data loss or data inconsistency can be prevented. Through the flexible takeover operation of the unowned node, the system can divide the tasks based on the node ownership domain when a failure occurs, balance the load of different nodes, and thus improve the fault tolerance of the entire system. After all unfinished operations are completed, the system re-divides the ownership domain of the unowned node, and selects a new master node. It is ensured that subsequent data reading and writing operations are run in the best performance state, thereby optimizing resource utilization and system efficiency. The data synchronization and status update of the mirror volume are completed through the new master node, ensuring the consistency of the system state after recovery. Whether it is data synchronization or system status adjustment, it is coordinated by the new master node to ensure the integrity of the mirror volume and data consistency. The synchronization process is effectively organized and managed, which can significantly reduce data synchronization time and improve the overall system recovery speed.

[0056] Through this systematic fault handling method, the system can not only quickly restore data read and write operations when the master node fails, but also ensure data consistency and integrity. This design can effectively improve the fault tolerance and fault recovery efficiency of the distributed storage system, and significantly reduce the impact of faults on business system operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 The figure is an application environment diagram of a data synchronization method in one embodiment.

[0058] Figure 2 The figure is a flowchart of a data synchronization method in one embodiment.

[0059] Figure 3 FIG. 4 is a structural block diagram of a data synchronization device in an embodiment.

[0060] Figure 4 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment.

[0061] Figure 5 FIG. 4 is a timing diagram of a data synchronization method in an embodiment.

[0062] Figure 6 A diagram showing the division of node domains in a data synchronization method in one embodiment.

[0063] Figure 7 A data mark bitmap of a data synchronization method in one embodiment. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0065] The present application provides a method for synchronizing data of a non-attributed mirror volume, which can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. When the owned master node fails, the fault information is immediately obtained and the unfinished operations are integrated. A global unfinished operation record table is constructed, and the unfinished operations will be accurately recorded and assigned to appropriate non-owned nodes for processing. Through flexible takeover operations of non-owned nodes, the unfinished operations of the owned master node continue to be executed. After the unfinished operations are completed, the owned domains are redivided, and the non-owned nodes are divided into each owned domain, and a new master node is selected. The data synchronization and status update of the mirror volume are completed through the new master node, wherein the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices, and the server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0066] In one embodiment, Figure 2 As shown, a method for synchronizing data of a non-attributed mirror volume is provided, and the method is applied to Figure 1 The terminal in is used as an example to illustrate, including the following steps:

[0067] Step 202, in response to a failure of a belonging master node in data reading and writing processing on a mirror volume, obtaining failure information, wherein the failure information includes: unfinished operations and a list of non-belonging nodes, wherein the list of non-belonging nodes includes multiple non-belonging nodes.

[0068] Among them, mirror volume is a storage technology that refers to real-time copying and storing the same data on multiple physical or virtual hard disks to ensure data redundancy and high availability.

[0069] Specifically, by recording the unfinished operations before the failure of the master node, it is ensured that these unfinished tasks can be identified and effectively processed, preventing data operation loss or omission, and ensuring data consistency and integrity.

[0070] Step 204: Integrate the unfinished operations according to the fault information to obtain a global unfinished operation list.

[0071] The node belonging domain refers to the classification and organization of nodes in the system according to certain rules or standards to form node groups with similar attributes. Each node belongs to a specific "domain" or "region", which are usually divided based on geographical location, network topology, data center division or functional role.

[0072] Specifically, by integrating the unfinished operations, all unfinished operations are integrated into the global unfinished operation record table, ensuring that the system has a comprehensive understanding of all unfinished operations, preventing any key operations from being missed, and ensuring the integrity and consistency of data processing. Avoiding different nodes from repeatedly processing the same task during the fault recovery process, thereby reducing unnecessary resource consumption and data conflicts.

[0073] Step 206: Based on the global unfinished operation record table, the plurality of non-home nodes take over the unfinished operations.

[0074] The non-belonging node is a backup node of the belonging master node. After the belonging node fails, all remaining nodes become non-belonging nodes.

[0075] Specifically, through flexible takeover of non-attributed nodes, the system takes over the remaining tasks when a failure occurs, balancing the load of different nodes, thereby improving the fault tolerance of the entire system. Distributed processing of unfinished operations can prevent a node from bearing too much recovery pressure after a failure, thereby improving the overall stability of the system.

[0076] Step 208: In response to the completion of all unfinished operations, the home domain of the non-home node is re-divided to obtain a new master node.

[0077] Specifically, after all pending operations are completed, the system re-divides the domain of the non-attributed node and selects a new master node. This process can select the best new master node based on the system's operating status and the performance of the current node, ensuring that subsequent data read and write operations are performed in the best performance state, thereby optimizing resource utilization and system efficiency.

[0078] Step 210: completing data synchronization and status update of the unowned mirror volume through the new primary node.

[0079] Specifically, the new master node completes the data synchronization and status update of the mirrored volume, ensuring the consistency of the system after recovery. Both data synchronization and system status adjustment are coordinated by the new master node, thus ensuring the integrity of the mirrored volume and data consistency. The synchronization process is effectively organized and managed, which can significantly reduce the synchronization time and improve the overall system recovery speed.

[0080] In the above data recovery method, Figure 5 As shown in the figure, when the owned master node fails, the system can quickly identify the current unfinished operations by immediately obtaining the fault information and integrating the unfinished operations. By building a global unfinished operation record table, it is ensured that each unfinished operation has a clear allocation, so that the unowned node can quickly take over the task, avoiding long-term interruption of data reading and writing, thereby ensuring the high availability of the system. In the process of fault handling, the unfinished operations will be accurately recorded and assigned to the appropriate unowned node for processing, ensuring that all unfinished data operations of the original master node will not be lost or omitted. Through orderly takeover, the integrity and consistency of the operation can be guaranteed, and the risk of data loss or data inconsistency can be prevented. Through the flexible takeover operation of the unowned node, the system can divide the tasks based on the node ownership domain when a failure occurs, balance the load of different nodes, and thus improve the fault tolerance of the entire system. After all unfinished operations are completed, the system re-divides the ownership domain of the unowned node and selects a new master node. It ensures that subsequent data reading and writing operations are run in the best performance state, thereby optimizing resource utilization and system efficiency. The data synchronization and status update of the mirror volume are completed through the new master node, ensuring the consistency of the system state after recovery. Whether it is data synchronization or system status adjustment, it is coordinated by the new master node to ensure the integrity of the mirror volume and data consistency. The synchronization process is effectively organized and managed, which can significantly reduce data synchronization time and improve the overall system recovery speed.

[0081] Through this systematic fault handling method, the system can not only quickly restore data read and write operations when the master node fails, but also ensure data consistency and integrity. This design can effectively improve the fault tolerance and fault recovery efficiency of the distributed storage system, and significantly reduce the impact of faults on business system operations.

[0082] In one embodiment, the method further comprises:

[0083] Step 302: Detect the heartbeat signal sending operation of the master node.

[0084] Specifically, by monitoring the heartbeat signal of the owned master node, the system can detect in real time whether the master node fails.

[0085] Step 304: In response to the belonging master node not sending a heartbeat signal, it is determined that a failure occurs in the data reading and writing processing of the mirror volume by the belonging master node.

[0086] Specifically, collecting the list of unfinished operations and unattached nodes ensures that the system can fully grasp the unfinished data reading and writing tasks and the currently available unattached node resources when a failure occurs, ensuring that these tasks can be accurately processed in the subsequent recovery process to avoid data loss.

[0087] Step 306: collect the unfinished operations and the list of nodes without affiliation, broadcast the information to all nodes and report it.

[0088] Specifically, it collects and broadcasts information about unfinished operations to ensure that any unfinished operations will not be ignored or lost when the master node fails, thereby ensuring data integrity and consistency and avoiding data corruption or loss.

[0089] Through heartbeat signal detection, fault information collection and broadcasting, the system can respond quickly when the main node fails, accurately grasp unfinished tasks, and effectively schedule resources to ensure that data is not lost and quickly restore normal operations, while improving the system's automation, fault tolerance and overall stability.

[0090] In one embodiment, according to the fault information, the unfinished operations are integrated to obtain a global unfinished operation list.

[0091] include:

[0092] Step 402: Scan the logs through the non-attributable node to obtain unfinished operations of multiple faulty nodes.

[0093] Specifically, the unfinished operations of the faulty node are obtained through the scanning log of the non-attributed node, ensuring that the system can accurately locate and recover all unfinished operations when the faulty node occurs, avoiding the omission of any critical data processing tasks and ensuring the integrity of data operations.

[0094] Step 404: extract the unfinished operations, integrate multiple unfinished operation record tables and report them.

[0095] Specifically, integrating the incomplete operation record table helps to centrally process operations within the same domain, thereby reducing communication delays between nodes, optimizing resource allocation, and improving the efficiency of data recovery and processing.

[0096] Step 406: Integrate all the unfinished operation record tables to generate a global unfinished operation record table.

[0097] Specifically, the unfinished operation record tables reported by multiple non-belonging nodes are integrated to generate a global unfinished operation record table, so that the system can uniformly manage all unfinished operations, help avoid decentralized operations and complex management, and improve the coordination and efficiency of overall data processing.

[0098] Step 408: Eliminate duplicate operations in the global unfinished operation record table, and update the global unfinished operation record table.

[0099] Specifically, after generating the global unfinished operation record table, duplicate operations are eliminated to avoid the system from repeatedly executing the same task on multiple nodes, thereby reducing unnecessary waste of computing resources and lowering the possibility of data operation conflicts, thereby ensuring efficient operation of the system.

[0100] By scanning the logs of non-attributed nodes, integrating unfinished operations in the attribution domain, and generating and updating the global unfinished operation record table, the system can efficiently and accurately recover the unfinished operations of the failed nodes, avoid duplication, optimize resource allocation, and improve the consistency of data processing, the system's recovery speed and overall performance.

[0101] In one embodiment, based on the global unfinished operation record table, taking over the unfinished operation by the non-home node includes:

[0102] The homeless node reads a global unfinished operation record table, wherein the global unfinished operation record table includes at least one of the following: an operation type, a logical block address block range, an operation status, and an operation timestamp, the operation type includes: a read operation and a write operation, and the operation status includes: partially completed and uncompleted;

[0103] The homeless node sequentially recovers each unfinished operation according to the unfinished operation type and the unfinished operation state, and executes the unfinished operation in the order of the operation timestamps, wherein the unfinished operation includes at least one of the following: a read operation on the primary copy, a write operation on the primary copy, and marking data differences in a bitmap;

[0104] In response to resuming the write operation to the primary copy, the homeless node checks whether the operation has been recorded in the log;

[0105] In response to the write operation portion of the primary copy having been written, the homeless node continues to complete the remaining write operation from the interrupted portion;

[0106] In response to the primary copy write operation not starting, the non-home node performs a new write operation according to the data in the unfinished operation record table;

[0107] In response to the writing operation of the primary copy being completed, the update operation status is completed, and the update marks the data difference in the bitmap, records it in the log, and reports it;

[0108] In response to resuming the read operation on the primary copy, the homeless node initiates a read request for the logical block address range according to the data in the unfinished operation record table to obtain the required data;

[0109] In response to the owned master node reading part of the data, the unowned node continues reading from the interrupted part, records and reports the complete read data in a log, and updates the current read operation status as completed.

[0110] By taking over the unfinished operations according to the non-attributed nodes, reading the detailed records and restoring them in the order of timestamps, the system can efficiently and accurately handle data recovery, optimize resource utilization, and improve operation consistency, thereby greatly improving the overall fault recovery efficiency and data security. The data differences are marked in the bitmap and recorded in the log to ensure that the system can clearly understand which data blocks have been updated after each write operation. This mechanism effectively avoids conflicts caused by data differences during the recovery process and ensures data synchronization and consistency. The log records the recovery process of all unfinished operations, including the complete status of read and write operations, execution time and other information, which is convenient for post-analysis, troubleshooting or system optimization. Through the above refined operation process, the system can maximize data security and consistency when handling the failure of the master node, quickly restore normal operation, and improve the overall stability of the system. These steps enable the system to restore unfinished operations in an orderly and efficient manner, and ensure the consistency, integrity and reliability of operations. Through the cooperation of logs, bitmaps and timestamps, the system can flexibly handle operation recovery under various fault conditions, improve data security and system stability, simplify the recovery process, and improve overall efficiency.

[0111] In one embodiment, re-dividing the home domain of the non-home node to obtain a new master node includes:

[0112] Step 502: extracting node status information reported by the non-belonging node from the log resource of the belonging domain corresponding to the non-belonging node.

[0113] Step 504: Determine the state and load capacity of each of the non-home nodes according to the information reported by the non-home nodes.

[0114] Specifically, through the information reported by the homeless nodes, the system can monitor the status and load capacity of each homeless node in real time. This timely feedback mechanism helps to quickly identify potential problem nodes.

[0115] Step 506: The non-home nodes are sorted according to a load balancing algorithm, and the best non-home node is obtained as the new master node.

[0116] Specifically, the non-home nodes are sorted according to the load balancing algorithm, so that the non-home node with the strongest load capacity can be effectively identified as the new master node.

[0117] Step 508: Identify the data scope that each homeless node is responsible for and its new role.

[0118] Specifically, clearly identifying the scope of data that each non-attributable node is responsible for and its new role helps standardize node responsibilities, ensure that each node has clear tasks during the recovery process, and improve the efficiency of data management.

[0119] Step 510: Record the new home domain allocation and the new master node information.

[0120] Specifically, through clear role allocation and data responsibility, chaos in data management can be effectively prevented, ensuring that the system maintains data consistency and integrity after failure recovery.

[0121] By monitoring the status and load capacity of non-attributed nodes in real time and optimizing node allocation and roles based on load balancing algorithms, the system not only improves resource utilization efficiency and stability, simplifies management processes, and ensures data consistency, but also improves overall fault tolerance and recovery speed. This dynamic management mechanism provides a strong guarantee for the system when facing failures.

[0122] In one embodiment, completing data synchronization and status update of the mirror volume through the new primary node includes:

[0123] The new master node obtains a data mark position difference block between the primary copy and the secondary copy by scanning the bitmap information;

[0124] Transmitting the identified data mark position difference block from the primary copy to the secondary copy, and clearing the bitmap information of the completed transmission;

[0125] The transmitted bitmap information is updated, and the state of the mirror volume is updated to a data available state.

[0126] By scanning the bitmap information, the new master node identifies the difference data blocks and transmits them to the secondary copy, the system can efficiently complete data synchronization and ensure data consistency and availability. The bitmap management mechanism not only optimizes the transmission process and reduces resource waste, but also greatly shortens the fault recovery time and improves the system's performance, stability and scalability. This precise synchronization method provides reliable protection for data management and system recovery.

[0127] In one embodiment, re-dividing the home domain of the non-home node to obtain a new master node includes:

[0128] Re-dividing the mirror volume into a plurality of attribution domains;

[0129] Numbering the non-home nodes, and sequentially allocating them to each home domain according to the numbering sequence;

[0130] The log resources are divided according to the re-divided attribution domains, and each attribution domain corresponds to a copy of the log resources.

[0131] In the storage data backup function, the volume is divided according to the Segment segment, which refers to the general name of the space occupied by the data file, and is evenly distributed to all the domains in the data read and write group in a small to large manner. Domain refers to a logical division or management unit, which is used to organize and manage the redundancy relationship of nodes, data, and storage. For example, Segment 0 belongs to Domain0, Segment 1 belongs to Domain1, Segment 2 belongs to Domain2, Segment3 belongs to Domain3, and Segment4 belongs to Domain0, and so on. The unattached volume is composed of multiple segments, and the segments belong to all domains of the IOGRP according to the number.

[0132] In one embodiment, the unattached volume splits the volume capacity into 32MB segments and belongs to four domains at the same time. The specific domain to which the segment belongs is as follows: Figure 6 As shown in the figure. The bitmap used by the unattached mirror volume will also belong to four domains at the same time. The corresponding structural relationship between the segment and the bitmap is as follows Figure 7 Each copy volume follows Figure 6 and Figure 7 Segment and bitmap division method in IOGRP. Because all nodes in IOGRP are the master nodes of a domain and the backup nodes of another domain, the log resources on the node need to be divided into 4 parts, each corresponding to one domain. Assume that there are four nodes in IOGRP, Node0-Node3, where IOGRP refers to a group of storage nodes or controllers working together, responsible for processing data read and write operations and data distribution.

[0133] For example, each IOGROUP consists of 4 nodes and each IOGROUP has 4 domains. The composition of the nodes in the domain is:

[0134] Domain0: Node0 (Owner node), Node1 (Backup node).

[0135] Domain1: Node1 (Owner node), Node2 (Backup node).

[0136] Domain2: Node2 (Owner node), Node3 (Backup node).

[0137] Domain3: Node3 (Owner node), Node0 (Backup node).

[0138] In one embodiment, the identified data marker position difference block is transmitted from the primary copy to the secondary copy, including:

[0139] compressing the data mark position difference block;

[0140] Specifically, by compressing the data marker position difference blocks, the amount of transmitted data is reduced, thereby greatly reducing the occupation of network bandwidth.

[0141] Through parallel transmission technology, data is transmitted in shards to multiple secondary nodes;

[0142] By sharding data and transmitting it to multiple slave nodes in parallel, the data transmission speed can be greatly improved. Each slave node receives a portion of the data in the same time period, reducing single-point bottlenecks. Especially when processing large-scale data, parallel transmission can significantly shorten the overall transmission time.

[0143] Monitor the transfer status in real time, record the transfer progress and error log.

[0144] By monitoring the status of data transmission in real time, the system can dynamically understand the transmission progress of each data shard and the receiving status of the secondary node. This real-time performance helps to promptly discover and handle problems that may occur during transmission, thereby reducing the failure rate of data transmission. By monitoring the transmission status, the system can dynamically adjust the transmission strategy based on network conditions, node performance, etc. For example, when a node is found to be slow, the priority of its transmission task can be adjusted, or even part of the task can be transferred to other secondary nodes, thereby optimizing the overall transmission process.

[0145] By compressing data mark position difference blocks, transmitting data shards in parallel, and monitoring transmission status in real time, the entire data synchronization and transmission process becomes more efficient, reliable, and scalable. These steps can not only significantly improve data transmission efficiency, but also enhance the fault tolerance and manageability of the transmission process through real-time monitoring and error log mechanisms, thus providing a strong technical guarantee for distributed systems or data-intensive applications.

[0146] In one embodiment, updating the transmitted bitmap information and updating the state of the mirror volume to a data available state includes:

[0147] Detecting the data block transmission result;

[0148] In response to the data inconsistency between the primary copy and the secondary copy, re-marking the data mark position difference block and transmitting it again;

[0149] In response to the data of the primary copy and the secondary copy being consistent, bitmap information is updated and the state of the mirror volume is updated.

[0150] By detecting the data block transmission results, marking the data mark difference blocks and retransmitting them, and updating the bitmap information and mirror volume status when the data is consistent, the system can significantly improve the accuracy, performance and fault tolerance of data synchronization. This process not only ensures the data consistency of the primary and secondary nodes, but also greatly reduces unnecessary transmissions and optimizes the system's resource usage and maintenance efficiency. For large-scale distributed systems and high-availability scenarios, it can provide higher reliability and data security, and update the status of the mirror volume that has resumed normal working status to a data available state, which facilitates the system to judge and analyze the status of the mirror volume.

[0151] In one embodiment, the non-home nodes are sorted according to a load balancing algorithm to obtain the best non-home node as the new master node, including:

[0152] Regularly obtaining the number of connections of the non-home node;

[0153] According to the order of the number of connections of the non-belonging nodes from small to large, the non-belonging node with the least number of connections is the new master node;

[0154] If the connection numbers of the plurality of non-home nodes are the same, the non-home nodes are further sorted according to their response time and load to obtain the new master node.

[0155] By regularly collecting the number of connections of non-attributed nodes and sorting and selecting the best node to switch the master node, the high availability, fault recovery speed and system performance of the system can be effectively improved. At the same time, the automated monitoring and selection process simplifies the management of the system and improves the efficiency of operation and maintenance.

[0156] It should be understood that although Figure 2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0157] In one embodiment, Figure 3 As shown, a data synchronization device is provided, including: a fault information acquisition module 602, a list acquisition module 604, an operation takeover module 606, a home domain division module 608 and a data synchronization module 610, wherein:

[0158] The fault information acquisition module 602 is used to acquire fault information in response to a fault in data reading and writing processing of a mirror volume by an owned master node, wherein the fault information includes: unfinished operations and a list of unowned nodes, wherein the list of unowned nodes includes multiple unowned nodes.

[0159] The list acquisition module 604 is used to integrate the unfinished operations according to the fault information to obtain a global unfinished operation list.

[0160] The operation taking over module 606 is configured to enable the plurality of non-home nodes to take over the unfinished operations based on the global unfinished operation record table.

[0161] The home domain division module 608 is used to re-divide the home domain of the non-home node to obtain a new master node in response to the completion of the unfinished operations.

[0162] The home domain division module 610 is used to complete the data synchronization and status update of the non-home mirror volume through the new master node.

[0163] In one embodiment, the fault information acquisition module 602,

[0164] Used to detect the heartbeat signal sending operation of the owned master node.

[0165] Used to determine that a failure occurs in the data reading and writing processing of the mirror volume by the owned master node in response to the owned master node not sending a heartbeat signal.

[0166] It is used to collect the unfinished operations and the list of nodes without belonging, broadcast the information to all nodes and report it.

[0167] In one embodiment, the list acquisition module 604,

[0168] Used to scan the logs through the multiple non-attributable nodes to obtain the unfinished operations of the multiple faulty nodes;

[0169] Used to extract the unfinished operations, integrate multiple unfinished operation record tables and report them;

[0170] Used to integrate all the unfinished operation record tables to generate a global unfinished operation record table;

[0171] Used to eliminate duplicate operations in the global unfinished operation record table and update the global unfinished operation record table.

[0172] In one embodiment, the data synchronization module 610,

[0173] The new master node obtains a data mark position difference block between the primary copy and the secondary copy by scanning the reported bitmap information;

[0174] Used to transfer the identified data mark position difference block from the primary copy to the secondary copy, and clear the bitmap information of the completed transmission;

[0175] It is used to update the transmitted bitmap information and update the state of the mirror volume to a data available state.

[0176] For the specific definition of the data synchronization device, please refer to the definition of the data synchronization method above, which will not be repeated here. Each module in the above-mentioned data synchronization device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0177] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store mirror volume data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for synchronizing non-attributed mirror volume data is implemented.

[0178] Those skilled in the art will understand that Figure 4The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0179] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0180] In response to a failure in data reading and writing processing of a mirror volume by an owned master node, failure information is obtained, wherein the failure information includes: an unfinished operation and a list of unowned nodes, wherein the list of unowned nodes includes a plurality of unowned nodes;

[0181] According to the fault information, the unfinished operations are integrated to obtain a global unfinished operation list;

[0182] Based on the global unfinished operation record table, the plurality of unattached nodes take over the unfinished operations;

[0183] In response to the unfinished operations being completed, re-dividing the home domain of the non-home node to obtain a new master node;

[0184] The data synchronization and status update of the unowned mirror volume are completed through the new master node.

[0185] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0186] Detecting the heartbeat signal sending operation of the owned master node;

[0187] In response to the belonging master node not sending a heartbeat signal, determining that a failure occurs in the data reading and writing processing of the mirror volume by the belonging master node;

[0188] The unfinished operations and the list of unattached nodes are collected, and the information is broadcasted to all nodes and reported.

[0189] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0190] Re-dividing the mirror volume into a plurality of attribution domains;

[0191] Numbering the non-home nodes, and sequentially allocating them to each home domain according to the numbering sequence;

[0192] The log resources are divided according to the re-divided attribution domains, and each attribution domain corresponds to a copy of the log resources.

[0193] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0194] In response to a failure in data reading and writing processing of a mirror volume by an owned master node, failure information is obtained, wherein the failure information includes: an unfinished operation and a list of unowned nodes, wherein the list of unowned nodes includes a plurality of unowned nodes;

[0195] According to the fault information, the unfinished operations are integrated to obtain a global unfinished operation list;

[0196] Based on the global unfinished operation record table, the plurality of unattached nodes take over the unfinished operations;

[0197] In response to the unfinished operations being completed, re-dividing the home domain of the non-home node to obtain a new master node;

[0198] The data synchronization and status update of the unowned mirror volume are completed through the new master node.

[0199] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0200] The homeless node reads a global unfinished operation record table, wherein the global unfinished operation record table includes at least one of the following: an operation type, a logical block address block range, an operation status, and an operation timestamp, the operation type includes: a read operation and a write operation, and the operation status includes: partially completed and uncompleted;

[0201] The homeless node sequentially recovers each unfinished operation according to the unfinished operation type and the unfinished operation state, and executes the unfinished operation in the order of the operation timestamps, wherein the unfinished operation includes at least one of the following: a read operation on the primary copy, a write operation on the primary copy, and marking data differences in a bitmap;

[0202] In response to resuming the write operation to the primary copy, the homeless node checks whether the operation has been recorded in the log;

[0203] In response to the write operation portion of the primary copy having been written, the homeless node continues to complete the remaining write operation from the interrupted portion;

[0204] In response to the primary copy write operation not starting, the non-home node performs a new write operation according to the data in the unfinished operation record table;

[0205] In response to the writing operation of the primary copy being completed, the update operation status is completed, and the update marks the data difference in the bitmap, records it in the log, and reports it;

[0206] In response to resuming the read operation on the primary copy, the homeless node initiates a read request for the logical block address range according to the data in the unfinished operation record table to obtain the required data;

[0207] In response to the owned master node reading part of the data, the unowned node continues reading from the interrupted part, records and reports the complete read data in a log, and updates the current read operation status as completed.

[0208] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0209] Determine the state and load capacity of each of the unattached nodes according to the information reported by the unattached nodes;

[0210] The non-home nodes are sorted according to a load balancing algorithm, and the best non-home node is obtained as the new master node;

[0211] Identify the data scope that each homeless node is responsible for and its new role;

[0212] The new home domain allocation and the new master node information are recorded in the log.

[0213] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0214] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0215] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for synchronizing data of a non-attributed mirror volume, characterized in that: include, In response to a failure in data reading and writing processing of a mirror volume by an owned master node, failure information is obtained, wherein the failure information includes: an unfinished operation and a list of unowned nodes, wherein the list of unowned nodes includes a plurality of unowned nodes; According to the fault information, the unfinished operations are integrated to obtain a global unfinished operation list; Based on the global unfinished operation record table, the plurality of unattached nodes take over the unfinished operations; In response to the completion of all the unfinished operations, the belonging domain of the non-belonging node is re-divided to obtain a new master node; and data synchronization and status update of the non-belonging mirror volume are completed through the new master node.

2. A method for synchronizing data of a non-attributed mirror volume according to claim 1, characterized in that: Re-dividing the home domain of the non-home node to obtain a new master node includes: Re-dividing the mirror volume into a plurality of attribution domains; The non-attributed nodes are numbered and allocated to each attribution domain in sequence according to the numbering sequence; log resources are divided according to the re-divided attribution domains, and each attribution domain corresponds to a copy of the log resources.

3. A method for synchronizing data of a non-attributed mirror volume according to claim 2, characterized in that: Re-dividing the home domain of the non-home node to obtain a new master node also includes: Extracting node status information reported by the unattached node from log resources of the unattached node corresponding to the assigned domain; judging the status and load capacity of each unattached node according to the node status information reported by the unattached node; The non-home nodes are sorted according to a load balancing algorithm, and the best non-home node is obtained as the new master node; Identify the data scope that each homeless node is responsible for and its new role; The new home domain allocation and the new master node information are recorded in the log.

4. The method for synchronizing data of a non-attributed mirror volume according to claim 1, characterized in that: The method further comprises: Detecting the heartbeat signal sending operation of the owned master node; In response to the belonging master node not sending a heartbeat signal, determining that a failure occurs in the data reading and writing processing of the mirror volume by the belonging master node; The unfinished operations and the list of unattached nodes are collected, and the information is broadcasted to all nodes and reported.

5. The method for synchronizing data of a non-attributed mirror volume according to claim 1, characterized in that: According to the fault information, the unfinished operations are integrated to obtain a global unfinished operation list, including: Scanning logs through the plurality of non-attributable nodes to obtain unfinished operations of the plurality of faulty nodes; Extract the unfinished operations, integrate multiple unfinished operation record tables and report them; Integrate all the unfinished operation record tables to generate a global unfinished operation record table; The repeated operations in the global unfinished operation record table are eliminated, and the global unfinished operation record table is updated.

6. A method for synchronizing data of a non-attributed mirror volume according to claim 5, characterized in that: Based on the global unfinished operation record table, the plurality of non-home nodes taking over the unfinished operations include: The homeless node reads a global unfinished operation record table, wherein the global unfinished operation record table includes at least one of the following: an operation type, a logical block address block range, an operation status, and an operation timestamp, the operation type includes: a read operation and a write operation, and the operation status includes: partially completed and uncompleted; The homeless node sequentially recovers each unfinished operation according to the unfinished operation type and the unfinished operation state, and executes the unfinished operation in the order of the operation timestamps, wherein the unfinished operation includes at least one of the following: a read operation on the primary copy, a write operation on the primary copy, and marking data differences in a bitmap; In response to resuming the write operation to the primary copy, the homeless node checks whether the operation has been recorded in the log; In response to the write operation portion of the primary copy having been written, the homeless node continues to complete the remaining write operation from the interrupted portion; In response to the primary copy write operation not starting, the non-home node performs a new write operation according to the data in the unfinished operation record table; In response to the writing operation of the primary copy being completed, the update operation status is completed, and the update marks the data difference in the bitmap, records it in the log, and reports it; In response to resuming the read operation on the primary copy, the homeless node initiates a read request for the logical block address range according to the data in the unfinished operation record table to obtain the required data; In response to the owned master node reading part of the data, the unowned node continues reading from the interrupted part, records and reports the complete read data in a log, and updates the current read operation status to completed.

7. A method for synchronizing data of a non-attributed mirror volume according to claim 6, characterized in that: The data synchronization and status update of the mirror volume are completed through the new master node, including: The new master node obtains a data mark position difference block between the primary copy and the secondary copy by scanning the bitmap information; Transmitting the identified data mark position difference block from the primary copy to the secondary copy, and clearing the bitmap information of the completed transmission; The transmitted bitmap information is updated, and the state of the unowned mirror volume is updated to a data available state.

8. A data synchronization device, characterized in that: The device comprises: The fault information acquisition module is used to respond to a fault in the data reading and writing processing of the mirror volume by the belonging master node, Fault information is obtained, wherein the fault information includes: unfinished operations and a list of no-belonging nodes, and the list of no-belonging nodes includes a plurality of no-belonging nodes. The list acquisition module is used to integrate the unfinished operations according to the fault information to obtain a global unfinished operation list. The operation takeover module is used to enable the plurality of non-attributed nodes to take over the unfinished operations based on the global unfinished operation record table. The home domain division module is used to, in response to the completion of all the unfinished operations, re-divide the home domain of the non-home node to obtain a new master node. The data synchronization module is used to complete the data synchronization and status update of the unattached mirror volume through the new master node.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images