A method, device, computer equipment and storage medium for synchronizing data of unattached mirror volume

Abstract

The present application relates to a method, apparatus, computer equipment and storage medium for synchronizing data of a non-attributed mirror volume. The method comprises: in response to a failure of a primary node to read and write data to a mirror volume, obtaining fault information, wherein the fault information comprises: unfinished operations and a list of non-attributed nodes, the list of non-attributed nodes comprising a plurality of non-attributed nodes; integrating the unfinished operations according to the fault information to obtain a global list of unfinished operations; based on the global unfinished operation record table, the plurality of non-attributed nodes take over the unfinished operations; in response to the completion of all unfinished operations, redividing the attribution domain of the non-attributed nodes to obtain a new primary node; completing data synchronization and status update of the non-attributed mirror volume through the new primary node. The data synchronization and status update of the mirror volume are completed through the new primary node. The use of this method can improve data synchronization efficiency and ensure data consistency.

Description

Technical Field

[0001] The present application relates to the field of computer storage technology, and in particular to a method, apparatus, computer device, and storage medium for synchronizing data of a non-owned mirror volume. Background Art

[0002] Non-owner mirrored volumes are a flexible data management method in modern distributed storage systems, widely used in high availability and data redundancy scenarios. In a non-owner mirrored volume, volume data is stored simultaneously on multiple nodes, without a fixed owner node. In other words, no single node controls all read and write operations for the volume. This design allows multiple nodes to take over the volume simultaneously or dynamically, eliminating single points of failure.

[0003] In this architecture, when a system failure occurs, other nodes can quickly take over and restore consistency through data synchronization. However, after a node failure, ensuring that the mirrored volume data of each node remains consistent during data recovery is a key issue. Non-homeless distributed systems require an efficient synchronization mechanism to prevent data inconsistencies during the data recovery process. At the same time, data synchronization of non-homeless mirrored volumes requires efficient algorithms to reduce data transmission overhead. Especially when performing differential synchronization after failure recovery, how to quickly transmit changed data and ensure data consistency without affecting overall system performance is a technical problem that urgently needs to be solved. Summary of the Invention

[0004] Based on this, it is necessary to provide a data synchronization method, device, computer equipment and storage medium that can improve data synchronization efficiency and ensure data consistency in response to the above technical problems.

[0005] In one aspect, a method for synchronizing data of a non-owned mirror volume is provided, the method comprising:

[0006] In response to a failure in data reading and writing processing of a mirrored volume by an owned master node, failure information is obtained, wherein the failure information includes: an unfinished operation and a list of unowned nodes, wherein the list of unowned nodes includes a plurality of unowned nodes;

[0007] Integrate the unfinished operations according to the fault information to obtain a global unfinished operation record table;

[0008] Based on the global unfinished operation record table, the plurality of unowned nodes take over the unfinished operations;

[0009] In response to the completion of all the unfinished operations, re-dividing the home domain of the non-home node to obtain a new master node;

[0010] The data synchronization and status update of the unowned mirror volume are completed through the new master node.

[0011] In one embodiment, re-dividing the home domain of the homeless node to obtain a new master node includes:

[0012] Re-dividing the mirror volume into multiple home domains;

[0013] Numbering the non-home nodes and allocating them to each home domain in sequence according to the numbering sequence;

[0014] The log resources are divided according to the re-divided attribution domains, and each attribution domain corresponds to a copy of the log resources.

[0015] In one embodiment, it further includes:

[0016] Detecting the heartbeat signal sending operation of the owned master node;

[0017] In response to the belonging master node not sending a heartbeat signal, determining that a failure occurs in the data reading and writing processing of the mirror volume by the belonging master node;

[0018] Collect the unfinished operations and the list of unattached nodes, broadcast the information to all nodes and report it.

[0019] In one embodiment, the unfinished operations are integrated according to the fault information to obtain a global unfinished operation list, including:

[0020] Scanning logs through the plurality of non-attributable nodes to obtain unfinished operations of the plurality of faulty nodes;

[0021] Extracting the unfinished operations, integrating multiple unfinished operation record tables and reporting them;

[0022] Integrate all the unfinished operation record tables to generate a global unfinished operation record table;

[0023] Eliminate duplicate operations in the global unfinished operation record table, and update the global unfinished operation record table.

[0024] In one embodiment, based on the global unfinished operation record table, the plurality of non-home nodes take over the unfinished operations, including:

[0025] The homeless node reads a global incomplete operation record table, wherein the global incomplete operation record table includes at least one of the following: an operation type, a logical block address block range, an operation status, and an operation timestamp, the operation type includes: a read operation and a write operation, and the operation status includes: partially completed and incomplete;

[0026] The homeless node sequentially recovers each unfinished operation according to the unfinished operation type and the unfinished operation status, and executes the unfinished operation in the order of the operation timestamps, wherein the unfinished operation includes at least one of the following: a read operation on the primary copy, a write operation on the primary copy, and marking data differences in a bitmap;

[0027] In response to resuming the write operation to the primary copy, the homeless node checks whether the operation has been recorded in the log;

[0028] In response to the write operation portion of the primary copy having been written, the no-home node continues to complete the remaining write operation from the interrupted portion;

[0029] In response to the primary copy write operation not being started, the non-home node performs a new write operation according to the data in the incomplete operation record table;

[0030] In response to the completion of the write operation of the primary copy, the update operation status is completed, and the update marks the data difference in the bitmap, records it in the log, and reports it;

[0031] In response to resuming the read operation on the primary copy, the homeless node initiates a read request for the logical block address range according to the data in the unfinished operation record table to obtain required data;

[0032] In response to the owned master node reading part of the data, the unowned node continues reading from the interrupted part, records the complete read data into a log and reports it, and updates the current read operation status to completed.

[0033] In one embodiment, the method includes: re-dividing the home domain of the homeless node to obtain a new master node, including:

[0034] Extracting node status information reported by the unattached node from log resources of the unattached domain corresponding to the unattached node;

[0035] Determining the status and load capacity of each of the unattached nodes according to the node status information reported by the unattached nodes;

[0036] The non-home nodes are sorted according to a load balancing algorithm, and the optimal non-home node is selected as the new master node;

[0037] Identify the data scope that each unowned node is responsible for and its new role;

[0038] The new home domain allocation and the new master node information are recorded in the log.

[0039] In one embodiment, the method includes: completing data synchronization and status update of the mirror volume through the new master node, including:

[0040] The new master node obtains a data mark position difference block between the primary copy and the secondary copy by scanning the bitmap information;

[0041] Transferring the identified data mark position difference block from the primary copy to the secondary copy, and clearing the bitmap information of the completed transfer;

[0042] The transmitted bitmap information is updated, and the state of the mirror volume is updated to a data available state.

[0043] On the other hand, a device for synchronizing data of a non-owned mirror volume is provided, the device comprising:

[0044] The fault information acquisition module is used to obtain fault information in response to a failure of a primary node in reading and writing data on a mirror volume, wherein the fault information includes: unfinished operations and a list of non-belonging nodes, and the list of non-belonging nodes includes multiple non-belonging nodes.

[0045] The list acquisition module is used to integrate the unfinished operations according to the fault information to obtain a global unfinished operation record table.

[0046] The operation takeover module is configured to enable the plurality of non-home nodes to take over the unfinished operations based on the global unfinished operation record table.

[0047] The home domain division module is configured to, in response to completion of all the unfinished operations, re-divide the home domain of the non-home node to obtain a new master node.

[0048] A data synchronization module is used to complete data synchronization and status update of the unowned mirror volume through the new master node.

[0049] In another aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0050] In response to a failure in data reading and writing processing of the mirror volume by the owning master node, obtaining failure information;

[0051] According to the fault information, the unfinished operations are integrated and divided according to the node domain to obtain an unfinished operation record table;

[0052] Based on the unfinished operation record table, taking over the unfinished operations through the remaining non-attributed nodes;

[0053] In response to the completion of all unfinished operations, reorganizing according to the home domains of the remaining nodes to obtain a new master node;

[0054] The data synchronization and status update of the mirror volume are completed through the new master node.

[0055] In another aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0056] In response to a failure in data reading and writing processing of the mirror volume by the owning master node, obtaining failure information;

[0057] According to the fault information, the unfinished operations are integrated and divided according to the node domain to obtain an unfinished operation record table;

[0058] Based on the unfinished operation record table, taking over the unfinished operations through the remaining non-attributed nodes;

[0059] In response to the completion of all unfinished operations, reorganizing according to the home domains of the remaining nodes to obtain a new master node;

[0060] The data synchronization and status update of the mirror volume are completed through the new master node.

[0061] In the above data recovery method, when the owned master node fails, the system can quickly identify unfinished operations by immediately acquiring fault information and integrating outstanding operations. By establishing a global outstanding operation record table, each outstanding operation is clearly assigned, enabling unowned nodes to quickly take over tasks, avoiding prolonged data read and write interruptions and ensuring high system availability. During fault handling, outstanding operations are accurately recorded and assigned to appropriate unowned nodes for processing, ensuring that all unfinished data operations on the original master node are not lost or omitted. This orderly takeover ensures operational integrity and consistency, preventing the risk of data loss or inconsistency. Through flexible takeover of unowned nodes, the system can divide tasks based on node ownership domains in the event of a failure, balancing the load across different nodes and improving the overall system's fault tolerance. After all outstanding operations are completed, the system re-divides the unowned node's ownership domain and selects a new master node. This ensures that subsequent data read and write operations are performed at optimal performance, thereby optimizing resource utilization and system efficiency. Data synchronization and status updates of mirrored volumes are completed by the new master node, ensuring system consistency after recovery. Whether it's data synchronization or system status adjustments, all coordination is done by the new primary node, ensuring the integrity of the mirrored volumes and data consistency. This effectively organized and managed synchronization process significantly reduces data synchronization time and improves overall system recovery speed.

[0062] This systematic fault handling approach not only allows the system to quickly restore data read and write operations when a primary node fails, but also ensures data consistency and integrity. This design effectively improves the fault tolerance and recovery efficiency of distributed storage systems, significantly reducing the impact of failures on business system operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 FIG. 4 is an application environment diagram of a data synchronization method in one embodiment.

[0064] Figure 2 Schematic diagram of a data synchronization method in one embodiment.

[0065] Figure 3 FIG. 4 is a structural block diagram of a data synchronization device in one embodiment.

[0066] Figure 4 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment.

[0067] Figure 5 FIG. 4 is a timing diagram of a data synchronization method in one embodiment.

[0068] Figure 6 A diagram illustrating the division of node domains in a data synchronization method according to an embodiment.

[0069] Figure 7 A data mark bitmap of a data synchronization method in one embodiment. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0071] The present application provides a method for synchronizing data of a non-attributed mirror volume, which can be applied to Figure 1In the application environment shown. The terminal 102 communicates with the server 104 through the network. When a fault occurs in the said attributed master node, the fault information is immediately obtained and the unfinished operations are integrated. A global unfinished operation record table is constructed, and the unfinished operations will be accurately recorded and assigned to the appropriate non-attributed nodes for processing. Through the flexible takeover operation of the non-attributed nodes, the unfinished operations of the said attributed master node are continued to be executed. After the unfinished operations are completed, the attributed domains are redivided, and the non-attributed nodes are divided into various attributed domains, and a new master node is selected. The data synchronization and status update of the mirrored volume are completed through the new master node, wherein the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices, and the server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0072] In one embodiment, Figure 2 As shown, a method for synchronizing data of a non-attributed mirror volume is provided, and the method is applied to Figure 1 The following steps are used as an example to illustrate the terminal:

[0073] Step 202: In response to a failure in data reading and writing processing of a mirrored volume by an owning master node, failure information is obtained, wherein the failure information includes: unfinished operations and a list of unowned nodes, wherein the list of unowned nodes includes multiple unowned nodes.

[0074] Among them, mirror volume is a storage technology that refers to real-time copying and storing the same data on multiple physical or virtual hard disks to ensure data redundancy and high availability.

[0075] Specifically, by recording the unfinished operations before the failure of the master node, it is ensured that these unfinished tasks can be identified and effectively processed, preventing data operations from being lost or omitted, and ensuring data consistency and integrity.

[0076] Step 204 : Integrate the unfinished operations according to the fault information to obtain a global unfinished operation record table.

[0077] Node affiliation domains refer to the classification and organization of system nodes according to certain rules or standards, forming groups of nodes with similar attributes. Each node is assigned to a specific "domain" or "region," which is typically based on geographic location, network topology, data center divisions, or functional roles.

[0078] Specifically, by consolidating the pending operations, all pending operations are integrated into a global pending operation record table, ensuring that the system has a comprehensive understanding of all unfinished operations, preventing the omission of any critical operations, and ensuring the integrity and consistency of data processing. This prevents different nodes from repeatedly processing the same tasks during the fault recovery process, thereby reducing unnecessary resource consumption and data conflicts.

[0079] Step 206: Based on the global unfinished operation record table, the plurality of non-home nodes take over the unfinished operations.

[0080] The unowned node is a backup node of the owned master node. When the owned node fails, all remaining nodes become unowned nodes.

[0081] Specifically, by flexibly taking over unattached nodes, the system can take over remaining tasks in the event of a failure, balancing the load across different nodes and improving the overall system's fault tolerance. Distributed processing of unfinished operations can prevent a single node from bearing excessive recovery pressure after a failure, thereby improving the overall stability of the system.

[0082] Step 208: In response to the completion of all pending operations, the home domain of the non-home node is re-divided to obtain a new master node.

[0083] Specifically, after all outstanding operations are completed, the system re-divides the domains of the unattached nodes and selects a new master node. This process selects the optimal new master node based on the system's operating status and the performance of the current nodes, ensuring that subsequent data read and write operations run at peak performance, thereby optimizing resource utilization and system efficiency.

[0084] Step 210: Complete data synchronization and status update of the unowned mirror volume through the new master node.

[0085] Specifically, the new primary node completes data synchronization and status updates for the mirrored volumes, ensuring system consistency after recovery. Both data synchronization and system status adjustments are coordinated by the new primary node, ensuring the integrity of the mirrored volumes and data consistency. This effectively organized and managed synchronization process significantly reduces synchronization time and improves overall system recovery speed.

[0086] In the above data recovery method, Figure 5As shown, when a fault occurs on the owned master node, the system can quickly identify unfinished operations by immediately acquiring fault information and consolidating outstanding operations. By building a global outstanding operation record table, each outstanding operation is clearly assigned, allowing unowned nodes to quickly take over tasks, avoiding prolonged data read and write interruptions and ensuring high system availability. During fault handling, outstanding operations are accurately recorded and assigned to appropriate unowned nodes for processing, ensuring that all unfinished data operations on the original master node are not lost or omitted. This orderly takeover ensures operational integrity and consistency, preventing the risk of data loss or inconsistency. Through flexible takeover of unowned nodes, the system can partition tasks based on node ownership domains in the event of a fault, balancing the load across different nodes and improving the overall system's fault tolerance. After all outstanding operations are completed, the system re-divides the unowned node's ownership domain and elects a new master node. This ensures that subsequent data read and write operations are performed at optimal performance, optimizing resource utilization and system efficiency. Data synchronization and status updates for mirrored volumes are completed by the new master node, ensuring system consistency after recovery. Whether it's data synchronization or system status adjustments, all coordination is done by the new primary node, ensuring the integrity of the mirrored volumes and data consistency. This effectively organized and managed synchronization process significantly reduces data synchronization time and improves overall system recovery speed.

[0087] This systematic fault handling approach not only allows the system to quickly restore data read and write operations when a primary node fails, but also ensures data consistency and integrity. This design effectively improves the fault tolerance and recovery efficiency of distributed storage systems, significantly reducing the impact of failures on business system operations.

[0088] In one embodiment, the method further comprises:

[0089] Step 302: Detect the heartbeat signal sending operation of the home master node.

[0090] Specifically, by monitoring the heartbeat signal of the owned master node, the system can detect in real time whether the master node fails.

[0091] Step 304: In response to the belonging master node not sending a heartbeat signal, it is determined that a failure occurs in the data reading and writing processing of the mirror volume by the belonging master node.

[0092] Specifically, collecting the list of unfinished operations and unattached nodes ensures that the system can fully grasp the unfinished data reading and writing tasks and the currently available unattached node resources when a failure occurs, ensuring that these tasks can be accurately processed in the subsequent recovery process to avoid data loss.

[0093] Step 306: Collect the unfinished operations and the list of unattached nodes, broadcast the information to all nodes and report it.

[0094] Specifically, it collects and broadcasts information about unfinished operations to ensure that any unfinished operations will not be ignored or lost when the master node fails, thereby ensuring data integrity and consistency and avoiding data corruption or loss.

[0095] Through heartbeat signal detection, fault information collection and broadcasting, the system can quickly respond to master node failures, accurately grasp unfinished tasks, and effectively dispatch resources to ensure that data is not lost and normal operations are quickly restored, while improving the system's automation, fault tolerance, and overall stability.

[0096] In one embodiment, the unfinished operations are integrated according to the fault information to obtain a global unfinished operation list, including:

[0097] Step 402: Scan the logs through the non-attributable node to obtain multiple unfinished operations of the faulty node.

[0098] Specifically, the unfinished operations of the faulty node are obtained by scanning the logs of the non-attributed node, ensuring that the system can accurately locate and recover all unfinished operations when the faulty node occurs, avoiding missing any critical data processing tasks and ensuring the integrity of data operations.

[0099] Step 404: extract the unfinished operations, integrate multiple unfinished operation record tables and report them.

[0100] Specifically, integrating the incomplete operation record table helps to centrally process operations within the same domain, thereby reducing communication delays between nodes, optimizing resource allocation, and improving the efficiency of data recovery and processing.

[0101] Step 406: Integrate all the incomplete operation record tables to generate a global incomplete operation record table.

[0102] Specifically, the unfinished operation record tables reported by multiple non-belonging nodes are integrated to generate a global unfinished operation record table, so that the system can uniformly manage all unfinished operations, help avoid scattered operations and complex management, and improve the coordination and efficiency of overall data processing.

[0103] Step 408: Eliminate duplicate operations in the global unfinished operation record table and update the global unfinished operation record table.

[0104] Specifically, after generating the global incomplete operation record table, by eliminating duplicate operations, the system avoids repeating the same task on multiple nodes, reduces unnecessary waste of computing resources, and reduces the possibility of data operation conflicts, ensuring the efficient operation of the system.

[0105] By scanning the logs of non-attributed nodes, integrating unfinished operations in the attribution domain, and generating and updating the global unfinished operation record table, the system can efficiently and accurately recover the unfinished operations of the faulty nodes, avoid duplication, optimize resource allocation, and improve the consistency of data processing, the system's recovery speed, and overall performance.

[0106] In one embodiment, taking over the unfinished operation by the non-home node based on the global unfinished operation record table includes:

[0107] The homeless node reads a global incomplete operation record table, wherein the global incomplete operation record table includes at least one of the following: an operation type, a logical block address block range, an operation status, and an operation timestamp, the operation type includes: a read operation and a write operation, and the operation status includes: partially completed and incomplete;

[0108] The homeless node sequentially recovers each unfinished operation according to the unfinished operation type and the unfinished operation status, and executes the unfinished operation in the order of the operation timestamps, wherein the unfinished operation includes at least one of the following: a read operation on the primary copy, a write operation on the primary copy, and marking data differences in a bitmap;

[0109] In response to resuming the write operation to the primary copy, the homeless node checks whether the operation has been recorded in the log;

[0110] In response to the write operation portion of the primary copy having been written, the no-home node continues to complete the remaining write operation from the interrupted portion;

[0111] In response to the primary copy write operation not being started, the non-home node performs a new write operation according to the data in the incomplete operation record table;

[0112] In response to the completion of the write operation of the primary copy, the update operation status is completed, and the update marks the data difference in the bitmap, records it in the log, and reports it;

[0113] In response to resuming the read operation on the primary copy, the homeless node initiates a read request for the logical block address range according to the data in the unfinished operation record table to obtain required data;

[0114] In response to the owned master node reading part of the data, the unowned node continues reading from the interrupted part, records the complete read data into a log and reports it, and updates the current read operation status to completed.

[0115] By taking over unfinished operations based on the unowned node, reading detailed records, and restoring them in timestamp order, the system efficiently and accurately handles data recovery, optimizes resource utilization, and improves operational consistency, significantly enhancing overall fault recovery efficiency and data security. Data differences are marked in a bitmap and recorded in a log, ensuring that after each write operation, the system clearly understands which data blocks have been updated. This mechanism effectively avoids conflicts caused by data discrepancies during recovery and ensures data synchronization and consistency. The log records the recovery process of all unfinished operations, including the complete status of read and write operations and execution time, facilitating post-analysis, troubleshooting, and system optimization. This refined operational process maximizes data security and consistency when handling master node failures, quickly restoring normal operations, and improving overall system stability. These steps enable the system to recover unfinished operations in an orderly and efficient manner, ensuring consistency, integrity, and reliability. The coordinated use of logs, bitmaps, and timestamps enables the system to flexibly handle recovery operations in various fault scenarios, improving data security and system stability, while streamlining the recovery process and enhancing overall efficiency.

[0116] In one embodiment, re-dividing the home domain of the homeless node to obtain a new master node includes:

[0117] Step 502: extract the node status information reported by the non-home node from the log resource of the home domain corresponding to the non-home node.

[0118] Step 504: Determine the status and load capacity of each of the non-home nodes based on the information reported by the non-home nodes.

[0119] Specifically, by using the information reported by the homeless nodes, the system can monitor the status and load capacity of each homeless node in real time. This timely feedback mechanism helps to quickly identify potential problem nodes.

[0120] Step 506: The non-home nodes are sorted according to a load balancing algorithm, and the best non-home node is selected as the new master node.

[0121] Specifically, the non-home nodes are sorted according to the load balancing algorithm, so that the non-home node with the strongest load capacity can be effectively identified as the new master node.

[0122] Step 508: Identify the data scope that each homeless node is responsible for and its new role.

[0123] Specifically, clearly identifying the data scope and new role of each non-attributable node helps standardize node responsibilities, ensures that each node has clear tasks during the recovery process, and improves data management efficiency.

[0124] Step 510: Record the new home domain allocation and the new master node information.

[0125] Specifically, through clear role allocation and data responsibility, chaos in data management can be effectively prevented, ensuring that the system maintains data consistency and integrity after failure recovery.

[0126] By monitoring the status and load capacity of unattached nodes in real time and optimizing node allocation and roles based on a load balancing algorithm, the system not only improves resource utilization efficiency and stability, but also simplifies management processes and ensures data consistency. This improves overall fault tolerance and recovery speed. This dynamic management mechanism provides strong protection against system failures.

[0127] In one embodiment, completing data synchronization and status update of the mirror volume through the new master node includes:

[0128] The new master node obtains a data mark position difference block between the primary copy and the secondary copy by scanning the bitmap information;

[0129] Transferring the identified data mark position difference block from the primary copy to the secondary copy, and clearing the bitmap information of the completed transfer;

[0130] The transmitted bitmap information is updated, and the state of the mirror volume is updated to a data available state.

[0131] By scanning bitmap information, the new primary node identifies differential data blocks and transfers them to the secondary replica, enabling efficient data synchronization and ensuring data consistency and availability. This bitmap management mechanism not only optimizes the transmission process and reduces resource waste, but also significantly shortens fault recovery time, improving system performance, stability, and scalability. This precise synchronization method provides reliable assurance for data management and system recovery.

[0132] In one embodiment, re-dividing the home domain of the homeless node to obtain a new master node includes:

[0133] Re-dividing the mirror volume into multiple home domains;

[0134] Numbering the non-home nodes and allocating them to each home domain in sequence according to the numbering sequence;

[0135] The log resources are divided according to the re-divided attribution domains, and each attribution domain corresponds to a copy of the log resources.

[0136] In the storage data backup function, volumes are divided into segments, a general term for data file space that is evenly distributed, from smallest to largest, across all domains in the data read / write group. A domain is a logical division or management unit used to organize and manage node, data, and storage redundancy. For example, Segment 0 belongs to Domain 0, Segment 1 belongs to Domain 1, Segment 2 belongs to Domain 2, Segment 3 belongs to Domain 3, and Segment 4 belongs to Domain 0, and so on. An unattached volume consists of multiple segments, and segments are numbered and assigned to all domains in the IOGRP.

[0137] In one embodiment, the unowned volume splits the volume capacity into 32MB segments and belongs to four domains. The specific segment belongs to which domain. Figure 6 As shown in the figure. The bitmap used by the unowned mirror volume will also belong to four domains at the same time. The corresponding structural relationship between the segment and the bitmap is as follows: Figure 7 Each copy volume follows Figure 6 and Figure 7 Segment and bitmap partitioning. Because all nodes in an IOGRP are master nodes for one domain and backup nodes for another, the log resources on the node need to be divided into four parts, one for each domain. Consider four nodes in an IOGRP, Node0-Node3. An IOGRP refers to a group of collaborative storage nodes or controllers responsible for handling data read and write operations and data distribution.

[0138] For example, each IOGROUP consists of 4 nodes and each IOGROUP has 4 domains. The composition of the nodes in the domain is as follows:

[0139] Domain0: Node0 (Owner node) and Node1 (Backup node).

[0140] Domain1: Node1 (Owner node), Node2 (Backup node).

[0141] Domain2: Node2 (Owner node) and Node3 (Backup node).

[0142] Domain3: Node3 (Owner node) and Node0 (Backup node).

[0143] In one embodiment, the identified data mark position difference block is transferred from the primary copy to the secondary copy, including:

[0144] compressing the data mark position difference block;

[0145] Specifically, by compressing the data marker position difference blocks, the amount of transmitted data is reduced, thereby greatly reducing the network bandwidth usage.

[0146] Through parallel transmission technology, data is fragmented and transmitted to multiple slave nodes;

[0147] By sharding data and transmitting it in parallel to multiple replica nodes, data transmission speeds can be significantly increased. Each replica node receives a portion of the data at the same time, reducing single point bottlenecks. Especially when processing large amounts of data, parallel transmission can significantly shorten overall transmission time.

[0148] Monitor the transfer status in real time, record the transfer progress and error logs.

[0149] By monitoring data transmission status in real time, the system dynamically understands the transmission progress of each data shard and the reception status of its replica nodes. This real-time performance helps promptly identify and address potential transmission issues, thereby reducing data transmission failure rates. By monitoring transmission status, the system can dynamically adjust transmission strategies based on network conditions, node performance, and other factors. For example, if a node is identified as slow, its transmission priority can be adjusted, or even some tasks can be transferred to other replica nodes, thereby optimizing the overall transmission process.

[0150] By compressing data based on location differences, transmitting data in parallel, and monitoring transmission status in real time, the entire data synchronization and transmission process becomes more efficient, reliable, and scalable. These steps not only significantly improve data transmission efficiency but also enhance the fault tolerance and manageability of the transmission process through real-time monitoring and error logging, providing a strong technical guarantee for distributed systems or data-intensive applications.

[0151] In one embodiment, updating the transmitted bitmap information and updating the state of the mirror volume to a data available state includes:

[0152] Detecting data block transmission results;

[0153] In response to the data inconsistency between the primary copy and the secondary copy, re-marking the data mark position difference block and transmitting it again;

[0154] In response to the data of the primary copy and the secondary copy being consistent, bitmap information is updated and the status of the mirrored volume is updated.

[0155] By detecting data block transfer results, marking discrepancies and retransmitting them, and updating bitmap information and mirror volume status when data is consistent, the system significantly improves data synchronization accuracy, performance, and fault tolerance. This process not only ensures data consistency between the primary and secondary nodes, but also significantly reduces unnecessary transmissions, optimizing system resource utilization and maintenance efficiency. For large-scale distributed systems and high-availability scenarios, it provides higher reliability and data security. It updates the status of mirrored volumes that have returned to normal operation to a data-available state, facilitating the system's assessment and analysis of the mirrored volume's status.

[0156] In one embodiment, the non-home nodes are sorted according to a load balancing algorithm to obtain the best non-home node as the new master node, including:

[0157] Regularly obtaining the number of connections of the unattached node;

[0158] Sort the unattached nodes by number of connections from small to large, and select the unattached node with the least number of connections as the new master node;

[0159] If the number of connections of the plurality of non-home nodes is the same, the non-home nodes are further sorted according to their response time and load to obtain the new master node.

[0160] By regularly collecting the number of connections from unattached nodes and sorting them to select the optimal node for master node switching, we can effectively improve system availability, fault recovery speed, and system performance. Furthermore, the automated monitoring and selection process simplifies system management and improves operational efficiency.

[0161] It should be understood that although Figure 2 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 2 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0162] In one embodiment, Figure 3As shown, a device for synchronizing data of a non-owned mirror volume is provided, comprising: a fault information acquisition module 602, a list acquisition module 604, an operation takeover module 606, an ownership domain division module 608, and a data synchronization module 610, wherein:

[0163] The fault information acquisition module 602 is used to acquire fault information in response to a fault in data reading and writing processing of a mirrored volume by an owned master node, wherein the fault information includes: unfinished operations and a list of unowned nodes, wherein the list of unowned nodes includes multiple unowned nodes.

[0164] The list acquisition module 604 is configured to integrate the unfinished operations according to the fault information to obtain a global unfinished operation record table.

[0165] The operation takeover module 606 is configured to enable the plurality of non-home nodes to take over the unfinished operations based on the global unfinished operation record table.

[0166] The home domain division module 608 is configured to, in response to completion of all pending operations, re-divide the home domain of the non-home node to obtain a new master node.

[0167] The home domain division module 608 is configured to complete data synchronization and status update of the non-home mirror volume through the new master node.

[0168] In one embodiment, the fault information acquisition module 602,

[0169] Used to detect the heartbeat signal sending operation of the owned master node.

[0170] Used to determine that a failure occurs in data reading and writing processing of the mirror volume by the owning master node in response to the owning master node not sending a heartbeat signal.

[0171] It is used to collect the unfinished operations and the list of unattached nodes, broadcast the information to all nodes and report it.

[0172] In one embodiment, the list acquisition module 604,

[0173] Used to scan the logs through the multiple non-attributable nodes to obtain the unfinished operations of the multiple faulty nodes;

[0174] Used to extract the unfinished operations, integrate multiple unfinished operation record tables and report them;

[0175] Used to integrate all the unfinished operation record tables to generate a global unfinished operation record table;

[0176] It is used to eliminate repeated operations in the global unfinished operation record table and update the global unfinished operation record table.

[0177] In one embodiment, the data synchronization module 610,

[0178] The new master node obtains a data mark position difference block between the primary copy and the secondary copy by scanning the reported bitmap information;

[0179] for transmitting the identified data mark position difference block from the primary copy to the secondary copy, and clearing the bitmap information of the completed transmission;

[0180] It is used to update the bitmap information after the transmission and update the state of the mirror volume to a data available state.

[0181] For the specific definition of the data synchronization device, please refer to the definition of the data synchronization method above and will not be repeated here. Each module in the above-mentioned data synchronization device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0182] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, memory, network interface, and database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store mirrored volume data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for synchronizing non-attributed mirrored volume data is implemented.

[0183] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0184] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are performed:

[0185] In response to a failure in data reading and writing processing of a mirrored volume by an owned master node, failure information is obtained, wherein the failure information includes: an unfinished operation and a list of unowned nodes, wherein the list of unowned nodes includes a plurality of unowned nodes;

[0186] Integrate the unfinished operations according to the fault information to obtain a global unfinished operation list;

[0187] Based on the global unfinished operation record table, the plurality of unowned nodes take over the unfinished operations;

[0188] In response to the completion of all the unfinished operations, re-dividing the home domain of the non-home node to obtain a new master node;

[0189] The data synchronization and status update of the unowned mirror volume are completed through the new master node.

[0190] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0191] Detecting the heartbeat signal sending operation of the owned master node;

[0192] In response to the belonging master node not sending a heartbeat signal, determining that a failure occurs in the data reading and writing processing of the mirror volume by the belonging master node;

[0193] Collect the unfinished operations and the list of unattached nodes, broadcast the information to all nodes and report it.

[0194] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0195] Re-dividing the mirror volume into multiple home domains;

[0196] Numbering the non-home nodes and allocating them to each home domain in sequence according to the numbering sequence;

[0197] The log resources are divided according to the re-divided attribution domains, and each attribution domain corresponds to a copy of the log resources.

[0198] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0199] In response to a failure in data reading and writing processing of a mirrored volume by an owned master node, failure information is obtained, wherein the failure information includes: an unfinished operation and a list of unowned nodes, wherein the list of unowned nodes includes a plurality of unowned nodes;

[0200] Integrate the unfinished operations according to the fault information to obtain a global unfinished operation list;

[0201] Based on the global unfinished operation record table, the plurality of unowned nodes take over the unfinished operations;

[0202] In response to the completion of all the unfinished operations, re-dividing the home domain of the non-home node to obtain a new master node;

[0203] The data synchronization and status update of the unowned mirror volume are completed through the new master node.

[0204] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0205] The homeless node reads a global incomplete operation record table, wherein the global incomplete operation record table includes at least one of the following: an operation type, a logical block address block range, an operation status, and an operation timestamp, the operation type includes: a read operation and a write operation, and the operation status includes: partially completed and incomplete;

[0206] The homeless node sequentially recovers each unfinished operation according to the unfinished operation type and the unfinished operation status, and executes the unfinished operation in the order of the operation timestamps, wherein the unfinished operation includes at least one of the following: a read operation on the primary copy, a write operation on the primary copy, and marking data differences in a bitmap;

[0207] In response to resuming the write operation to the primary copy, the homeless node checks whether the operation has been recorded in the log;

[0208] In response to the write operation portion of the primary copy having been written, the no-home node continues to complete the remaining write operation from the interrupted portion;

[0209] In response to the primary copy write operation not being started, the non-home node performs a new write operation according to the data in the incomplete operation record table;

[0210] In response to the completion of the write operation of the primary copy, the update operation status is completed, and the update marks the data difference in the bitmap, records it in the log, and reports it;

[0211] In response to resuming the read operation on the primary copy, the homeless node initiates a read request for the logical block address range according to the data in the unfinished operation record table to obtain required data;

[0212] In response to the owned master node reading part of the data, the unowned node continues reading from the interrupted part, records the complete read data into a log and reports it, and updates the current read operation status to completed.

[0213] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0214] Determining the status and load capacity of each of the unattached nodes based on the information reported by the unattached nodes;

[0215] The non-home nodes are sorted according to a load balancing algorithm, and the optimal non-home node is selected as the new master node;

[0216] Identify the data scope that each unowned node is responsible for and its new role;

[0217] The new home domain allocation and the new master node information are recorded in the log.

[0218] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0219] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0220] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the scope of the present application, and such modifications and improvements are all within the scope of protection of the present application.

Claims

1. A method for synchronizing data of a non-attributed mirror volume, characterized in that: include, In response to a failure in data reading and writing processing of a mirrored volume by an owned master node, failure information is obtained, wherein the failure information includes: an unfinished operation and a list of unowned nodes, wherein the list of unowned nodes includes a plurality of unowned nodes; Integrate the unfinished operations according to the fault information to obtain a global unfinished operation record table; Based on the global unfinished operation record table, the plurality of unowned nodes take over the unfinished operations; In response to the completion of all the unfinished operations, the home domain of the non-home node is re-divided to obtain a new master node; and data synchronization and status update of the non-home mirror volume are completed through the new master node.

2. A method for synchronizing data of a non-attributed mirror volume according to claim 1, characterized in that: Re-dividing the home domain of the non-home node to obtain a new master node includes: Re-dividing the mirror volume into multiple home domains; The non-attributable nodes are numbered and allocated to each attributable domain in sequence according to the numbering sequence; log resources are divided according to the re-divided attributable domains, and each attributable domain corresponds to a copy of the log resources.

3. A method for synchronizing data of a non-attributed mirror volume according to claim 2, characterized in that: Re-dividing the home domain of the non-home node to obtain a new master node further includes: Extracting node status information reported by the non-home node from log resources of the home domain corresponding to the non-home node; determining the status and load capacity of each non-home node based on the node status information reported by the non-home node; The non-home nodes are sorted according to a load balancing algorithm, and the optimal non-home node is selected as the new master node; Identify the data scope that each unowned node is responsible for and its new role; The new home domain allocation and the new master node information are recorded in the log.

4. The method for synchronizing data of a non-attributed mirror volume according to claim 1, wherein: The method further comprises: Detecting the heartbeat signal sending operation of the owned master node; In response to the belonging master node not sending a heartbeat signal, determining that a failure occurs in the data reading and writing processing of the mirror volume by the belonging master node; Collect the unfinished operations and the list of unattached nodes, broadcast the information to all nodes and report it.

5. The method for synchronizing data of a non-attributed mirror volume according to claim 1, wherein: According to the fault information, the unfinished operations are integrated to obtain a global unfinished operation list, including: Scanning logs through the plurality of non-attributable nodes to obtain unfinished operations of the plurality of faulty nodes; Extracting the unfinished operations, integrating multiple unfinished operation record tables and reporting them; Integrate all the unfinished operation record tables to generate a global unfinished operation record table; Eliminate duplicate operations in the global unfinished operation record table, and update the global unfinished operation record table.

6. A method for synchronizing data of a non-attributed mirror volume according to claim 5, characterized in that: Based on the global unfinished operation record table, the plurality of non-home nodes taking over the unfinished operations includes: The homeless node reads a global incomplete operation record table, wherein the global incomplete operation record table includes at least one of the following: an operation type, a logical block address block range, an operation status, and an operation timestamp, the operation type includes: a read operation and a write operation, and the operation status includes: partially completed and incomplete; The homeless node sequentially recovers each unfinished operation according to the unfinished operation type and the unfinished operation status, and executes the unfinished operation in the order of the operation timestamps, wherein the unfinished operation includes at least one of the following: a read operation on the primary copy, a write operation on the primary copy, and marking data differences in a bitmap; In response to resuming the write operation to the primary copy, the homeless node checks whether the operation has been recorded in the log; In response to the write operation portion of the primary copy having been written, the no-home node continues to complete the remaining write operation from the interrupted portion; In response to the primary copy write operation not being started, the non-home node performs a new write operation according to the data in the incomplete operation record table; In response to the completion of the write operation of the primary copy, the update operation status is completed, and the update marks the data difference in the bitmap, records it in the log, and reports it; In response to resuming the read operation on the primary copy, the homeless node initiates a read request for the logical block address range according to the data in the unfinished operation record table to obtain the required data; In response to the owned master node reading part of the data, the unowned node continues reading from the interrupted part, records the complete read data into a log and reports it, and updates the current read operation status to completed.

7. A method for synchronizing data of a non-attributed mirror volume according to claim 6, characterized in that: The data synchronization and status update of the mirror volume are completed through the new master node, including: The new master node obtains the data mark position difference block between the primary copy and the secondary copy by scanning the bitmap information; Transferring the identified data mark position difference block from the primary copy to the secondary copy, and clearing the bitmap information of the completed transfer; The transmitted bitmap information is updated, and the state of the unowned mirror volume is updated to a data available state.

8. A device for synchronizing data of a non-attributed mirror volume, characterized in that: The device comprises: The fault information acquisition module is used to respond to a failure in data reading and writing processing of the mirror volume by the home master node, Acquiring fault information, wherein the fault information includes: unfinished operations and a no-home node list, wherein the no-home node list includes a plurality of no-home nodes; A list acquisition module is used to integrate the unfinished operations according to the fault information to obtain a global unfinished operation record table; An operation takeover module, configured to enable the plurality of non-home nodes to take over the unfinished operations based on the global unfinished operation record table; a home domain division module, configured to, in response to completion of all the unfinished operations, re-divide the home domain of the non-home node to obtain a new master node; A data synchronization module is used to complete data synchronization and status update of the unowned mirror volume through the new master node.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images