Metadata mirroring method and device, equipment and medium

Abstract

The invention discloses a metadata mirroring method, a metadata mirroring device, metadata mirroring equipment and a medium, and relates to the technical field of data storage, and the metadata mirroring method comprises the following steps: generating a metadata insertion request, and realizing dynamic aggregation of metadata requests by utilizing a corresponding relationship between a slot structure and a thread to avoid disordered sending of scattered requests; the slot structure maintains a waiting linked list used for temporarily storing mirror image requests, by detecting the state of the waiting linked list, a new request is forwarded in time when the linked list is empty and no processing request exists, after confirmation information returned by the standby end controller is received, the new mirror image requests are packaged in batches according to the set maximum request number when the linked list is not empty, and the packaged new mirror image requests are sent to the standby end controller. The method greatly reduces the number of communication interaction times between the main end controller and the standby end controller, effectively improves the utilization rate of bandwidth resources and the overall mirroring efficiency, provides efficient and stable support for data consistency of the main end and the standby end, and is particularly suitable for metadata synchronization requirements in high-concurrency scenes.

Description

Technical Field

[0001] The present invention relates to the field of data storage technology, and in particular to a metadata mirroring method, apparatus, device and medium. Background Art

[0002] To ensure high metadata availability, storage systems generally employ a mirroring mechanism. In a master-slave controller architecture, after the master controller performs a local insert operation on a piece of metadata, it immediately mirrors the operation to the slave controller to maintain metadata synchronization between the master and slave controllers. However, traditional metadata mirroring methods have efficiency bottlenecks. Because the mirroring process is performed on a single key-value (KV) operation, each time a local metadata insert is completed, a separate mirroring request must be initiated to the slave. This directly leads to a significant increase in the number of communications between controller nodes, and the frequent interactions consume a large amount of communication bandwidth between nodes. Summary of the Invention

[0003] The present invention provides a metadata mirroring method, apparatus, device and medium to at least solve the problem of metadata mirroring efficiency between controller nodes in the related art.

[0004] The present invention provides a metadata mirroring method for a master controller, comprising: Generate a metadata insertion request, distribute the metadata insertion request to a corresponding slot structure according to an assigned thread, and encapsulate the metadata insertion request as a target mirror request; the slot structure maintains a waiting linked list for temporarily storing the mirror request; Detecting a state of the waiting list, and when the state of the waiting list is empty and the slot structure has no mirroring request being processed, inserting the target mirroring request into the waiting list and forwarding the target mirroring request to the standby controller; After the standby controller completes processing the target mirror request and returns a confirmation, re-detecting the status of the waiting list; If the state of the waiting list is not empty, a corresponding number of mirror requests are extracted from the waiting list according to the set maximum number of requests and packaged into new mirror requests, and the new mirror requests are sent to the standby controller; if the waiting list is empty, wait for the next mirror request to be inserted.

[0005] The present invention also provides a metadata mirroring device, comprising: a request processing module, configured to generate a metadata insertion request, distribute the metadata insertion request to a corresponding slot structure according to an assigned thread, and encapsulate the metadata insertion request into a target mirror request; the slot structure maintains a waiting linked list for temporarily storing mirror requests; A first detection module is configured to detect a state of the waiting list; when the state of the waiting list is empty and there is no mirror request being processed in the slot structure, execute an operation of the request forwarding module; The request forwarding module is configured to insert the target mirror request into the waiting linked list and forward the target mirror request to the standby controller; A second detection module is configured to re-detect the status of the waiting list after the backup controller completes processing the target mirroring request and returns a confirmation; if the status of the waiting list is not empty, execute the operation of the request aggregation module; if the waiting list is empty, wait for the insertion of the next mirroring request; The request aggregation module is configured to extract a corresponding number of mirror requests from the waiting list according to a set maximum number of requests, package the requests into new mirror requests, and send the new mirror requests to the standby controller.

[0006] The present invention also provides an electronic device, comprising: a master-end controller and a backup-end controller; The master controller is configured to generate a metadata insertion request, distribute the metadata insertion request to a corresponding slot structure according to an assigned thread, and encapsulate the metadata insertion request into a target mirroring request; the slot structure maintains a waiting linked list for temporarily storing mirroring requests; detect a status of the waiting linked list, and when the waiting linked list is empty and the slot structure has no mirroring request being processed, insert the target mirroring request into the waiting linked list and forward the target mirroring request to the standby controller; The standby controller is configured to process the target mirroring request and return confirmation information to the primary controller after processing; The master-side controller is further configured to recheck the status of the waiting list after receiving the confirmation information; if the status of the waiting list is not empty, extract a corresponding number of mirror requests from the waiting list according to the set maximum number of requests and package them into new mirror requests, and send the new mirror requests to the standby-side controller; if the waiting list is empty, wait for the next mirror request to be inserted.

[0007] The present invention also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned metadata mirroring methods are implemented.

[0008] The metadata mirroring method of the present invention accurately distributes metadata insertion requests to corresponding slot structures through a thread allocation mechanism, and combines the use of a waiting list to temporarily store requests and dynamically manage processing status, thereby bringing significant benefits in many aspects: on the one hand, dynamic aggregation of metadata requests is achieved based on the correspondence between the slot structure and the thread, avoiding the disordered sending of scattered requests. New mirroring requests are batched by setting a maximum number of requests, significantly reducing the number of communication interactions between the primary and standby controllers, effectively improving the utilization of bandwidth resources, and reducing communication overhead. On the other hand, by real-time detection of the waiting list status, new requests are promptly forwarded when the waiting list is empty and there are no requests being processed. After receiving confirmation information returned by the standby controller, batch extraction and processing are performed when the waiting list is not empty. This not only ensures a rapid response to a single urgent request, but also improves the overall mirroring efficiency through batch processing, and realizes flexible adaptation of scattered requests to batch requests. At the same time, the entire process ensures the orderliness and reliability of metadata mirroring, provides efficient and stable support for data consistency between the primary and standby ends, and is particularly suitable for metadata synchronization requirements in high-concurrency scenarios.

[0009] In addition, the present invention also provides a corresponding metadata mirroring device, electronic device and computer-readable storage medium for the metadata mirroring method, which have the same or corresponding technical features as the above-mentioned metadata mirroring method and have the same effects as above. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0011] Figure 1 A flowchart of a metadata mirroring method provided by an embodiment of the present invention; Figure 2 A schematic diagram corresponding to the metadata aggregate image provided by an embodiment of the present invention; Figure 3 A schematic diagram illustrating the correspondence between thread slots and the sequence number mechanism provided in an embodiment of the present invention; Figure 4 A schematic structural diagram of a metadata mirroring device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0012] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0013] It should be noted that, in the description of the present invention, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. The terms "first," "second," etc., in the present invention are used to distinguish similar objects, and are not used to describe a particular order or precedence.

[0014] In order to enable those skilled in the art to better understand the solutions of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0015] In conjunction with the specific application environment architecture or specific hardware architecture on which the execution of the metadata mirroring method depends, the specific application environment architecture or specific hardware architecture is described herein.

[0016] An embodiment of the present invention provides a metadata mirroring method, which is described in detail in conjunction with the execution flow of the metadata mirroring method. Figure 1 A flowchart of a metadata mirroring method provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method is used for the master controller and includes the following steps: S101. Generate a metadata insertion request, distribute the metadata insertion request to a corresponding slot structure according to an assigned thread, and encapsulate the metadata insertion request as a target mirror request; the slot structure maintains a waiting linked list for temporarily storing the mirror request.

[0017] It should be noted that a slot structure refers to storage space used to store thread-related information. For example, a slot structure can store thread status, execution status flags, and associated task queue pointers, facilitating the system's unified scheduling and monitoring of threads. During step S101, after data aggregation and flushing, batches of metadata insertion requests are generated. These metadata insertion requests are distributed to the corresponding slot structures based on the assigned threads, achieving precise binding between requests, threads, and slot structures, avoiding resource contention and improving orderly processing. The wait list maintained in the slot structure can serve as a temporary storage container for mirroring requests, providing buffer space for mirroring requests. This ensures that mirroring requests are not lost due to instantaneous high concurrency and allows for flexible adjustment for subsequent batch processing or dynamic scheduling.

[0018] The metadata of the present invention can be all-flash storage metadata. All-flash storage is a storage system built entirely on flash memory media. Unlike traditional mechanical hard disk storage or hybrid storage, it does not contain any mechanical components, and all data reading and writing relies on flash memory chips. All-flash storage metadata refers to the data used to describe and manage the actual data in an all-flash storage system, which is equivalent to the directory or navigation information of the storage system. It records key information, such as the storage location of the actual data, the attributes of the data, the associations between the data, etc. Of course, the metadata of the present invention can also be other types of metadata, such as metadata of a distributed storage system, which is not limited here.

[0019] S102 , detecting the state of the waiting list. When the state of the waiting list is empty and there is no mirroring request being processed in the slot structure, inserting the target mirroring request into the waiting list and forwarding the target mirroring request to the standby controller.

[0020] It's important to note that the master controller monitors the state of the wait list (whether it's empty) and the slot structure's processing status (whether there are any mirror requests being processed) in real time. These two conditions together form the basis for forwarding decisions, ensuring that forwarding operations don't conflict with the mirroring process. When the wait list is empty and no mirror requests are being processed, indicating that the slot structure is idle, the target mirror request is inserted into the wait list and immediately forwarded to the slave controller. This ensures that individual requests are responded to quickly, avoiding delays, and prevents conflicts or data inconsistencies that could result from concurrent processing through state verification.

[0021] S103: After the standby controller processes the target mirror request and returns a confirmation, it rechecks the status of the waiting list.

[0022] In implementation, after the standby controller completes processing of the target mirror request and returns confirmation information, the master controller does not directly enter the next round of request processing, but instead constructs a responsive processing node by re-detecting the status of the waiting list. In this way, the standby controller uses the processing confirmation as a trigger signal to dynamically sense whether there are newly accumulated mirror requests in the waiting list, ensuring that the master can flexibly select subsequent processing strategies based on the actual queue status.

[0023] S104: If the state of the waiting list is not empty, extract the corresponding number of mirror requests from the waiting list according to the set maximum number of requests and package them into new mirror requests, and send the new mirror requests to the standby controller; if the waiting list is empty, wait for the next mirror request to be inserted.

[0024] During implementation, when executing step S104, the master controller detects that the waiting list is not empty and extracts a corresponding number of mirror requests from the list, based on the set maximum number of requests. These requests are then packaged and integrated into new mirror requests and sent to the slave controller. This batch processing method effectively reduces the number of communications between the master and slave terminals, improving bandwidth utilization and processing efficiency. When the waiting list is empty, the system enters a waiting state to accept the next inserted mirror request, avoiding inefficient resource consumption and the sending of empty requests. By performing different operations based on the linked list status, this method achieves efficient batch processing of accumulated requests while ensuring timely responses to new requests. This balance between resource utilization and processing efficiency ensures the orderliness of the metadata mirroring process.

[0025] It should be added that, when it is detected that the waiting list is in a non-empty state, the present invention can obtain the set maximum number of requests based on the maximum length limit of the mirror message; if the total number of mirror requests in the waiting list exceeds the set maximum number of requests, the mirror requests with the set maximum number of requests are extracted; if the total number of mirror requests in the waiting list does not exceed the set maximum number of requests, all the mirror requests in the waiting list are extracted. The extracted mirror requests are then packaged into new mirror requests and sent to the backup controller. This process not only avoids the sending of empty requests by checking the status of the list, but also adapts to the length limit of the mirror message through a flexible strategy of quota interception or full processing, avoiding data overflow or transmission errors, and achieving a balance between efficient processing and standardized transmission, while ensuring the orderly processing of requests in the waiting list to prevent backlogs.

[0026] The metadata mirroring method provided in the embodiments of the present invention precisely distributes metadata insertion requests to corresponding slot structures through a thread allocation mechanism, and combines this with a wait list to temporarily store requests and dynamically manage processing status, resulting in significant benefits in multiple aspects. First, metadata requests are dynamically aggregated based on the correspondence between slot structures and threads, avoiding the disordered transmission of scattered requests. New mirroring requests are batched by setting a maximum number of requests, significantly reducing the number of communication interactions between the primary and standby controllers, effectively improving bandwidth resource utilization and reducing communication overhead. Second, by real-time monitoring of the wait list status, new requests are promptly forwarded when the wait list is empty and there are no pending requests. Upon receiving confirmation information from the standby controller, batch extraction and processing are performed when the wait list is not empty. This ensures a rapid response to individual urgent requests, improves overall mirroring efficiency through batch processing, and enables flexible adaptation of scattered and batch requests. Furthermore, the entire process ensures the orderliness and reliability of metadata mirroring, providing efficient and stable support for data consistency between the primary and standby ends, making it particularly suitable for metadata synchronization requirements in high-concurrency scenarios.

[0027] Furthermore, in a specific implementation, in the metadata mirroring method provided in an embodiment of the present invention, before executing step S101 to generate a metadata insertion request, it may also include: establishing an equal amount of slot structures according to a preset number of threads, and binding the slot structures to threads one-to-one.

[0028] In implementation, the present invention can apply for slot structure resources according to the number of threads and perform one-to-one binding. In this way, by allocating an independent slot structure to each thread, there is no intersection or interference between threads, which can effectively avoid competition and conflict between multiple threads for the same resource, reduce the synchronization overhead between threads, and improve processing efficiency; at the same time, the one-to-one binding relationship makes the request processing path clearer, facilitates tracking the request flow status corresponding to each thread, reduces the difficulty of problem troubleshooting, and provides stable and efficient basic support for subsequent request aggregation, distribution and mirror forwarding, especially in high concurrency scenarios, can significantly improve the overall processing throughput and stability.

[0029] Furthermore, in a specific implementation, in the metadata mirroring method provided in an embodiment of the present invention, after executing step S101 to encapsulate the metadata insertion request into a target mirroring request, the method may further include: when the state of the waiting list is non-empty, inserting the target mirroring request into the waiting list and returning; when the state of the waiting list is empty but the slot structure is processing the mirroring request, inserting the target mirroring request into the waiting list and returning.

[0030] In practice, the present invention uniformly inserts the target mirror request into the waiting list and returns the operation, ensuring that the request is securely stored and not lost, regardless of whether the waiting list is not empty or the list is empty but the slot is busy, thereby avoiding the risk of request discard due to a busy slot or a non-empty queue. At the same time, the judgment process for request access is simplified, and the orderly accumulation of requests can be achieved without the need for additional complex logic, which not only ensures that the currently processed mirror request is not interfered with, but also reserves pending requests for subsequent batch processing or timely forwarding during idle time.

[0031] Furthermore, in the metadata mirroring method provided in the embodiments of the present invention, the slot structure can also maintain a master sequence number (OwnerSequenceNo) and a backtracking sequence number (BackSequenceNo). The master sequence number serves as an identifier for the mirroring request sent by the master controller, while the backtracking sequence number serves as a reference value for verifying the mirroring request by the backup controller. The master sequence number of the master controller and the backtracking sequence number of the backup controller correspond to each other.

[0032] In practice, the backtracking sequence number is a concept corresponding to the primary sequence number. It serves as a sequence number for reverse operations and complements the primary sequence number, jointly ensuring the integrity of data transfer or object management. The primary sequence number uniquely identifies mirroring requests sent by the primary controller, while the backtracking sequence number serves as a baseline for verifying mirroring requests on the backup controller. The precise correspondence between the primary sequence number and the backup controller's backtracking sequence number ensures the traceability of mirroring requests and the reliability of data synchronization between the primary and backup controllers. Both the primary and backtracking sequence numbers can be initially set to 0.

[0033] Figure 2 Schematic diagram corresponding to the metadata aggregate image provided by the embodiment of the present invention. Figure 2 As shown, the first node and the second node can correspond to the master controller and the backup controller respectively. A waiting list is added to the sending end of the master controller, which can be used to control the adaptive aggregate mirror request. Due to the maximum length limit of the mirror message between nodes, the aggregate mirror will also set the maximum number of mirror requests; through adaptive aggregate mirroring, an efficient mirroring effect with zero waiting and large bandwidth can be achieved.

[0034] Furthermore, in a specific implementation, in the metadata mirroring method provided in an embodiment of the present invention, step S102 forwards the target mirroring request to the standby controller, which may specifically include: reading the current master sequence number stored by the master controller itself from the slot structure; packaging the target mirroring request and the current master sequence number into a mirroring message, and sending the packaged mirroring message to the standby controller so that the standby controller processes the mirroring message; after the mirroring message is sent, the current master sequence number is incremented by a set value; wherein, the standby controller processes the mirroring message, including: after the standby controller receives the packaged mirroring message, it parses the packaged mirroring message to obtain the parsed master sequence number; reads the backtracking sequence number stored by the standby controller itself from the slot structure bound to the current thread; verifies the read backtracking sequence number with the parsed master sequence number; performs corresponding processing on the mirroring message according to the verification result; if the mirroring request is executed, the read backtracking sequence number is incremented by a set value.

[0035] During implementation, the master-side controller packages the mirror message with the current master serial number and automatically increments the set value (such as 1) after sending it. Combined with the verification logic based on the backtracking serial number on the equipment side, this effectively ensures the order and consistency of the metadata mirroring on the master and backup sides, avoiding message loss or disorder problems; the self-increment rule of the serial number enables each interaction to verify the integrity of the message through numerical comparison, providing an accurate verification basis for the reliability of data synchronization; at the same time, the binding relationship between the slot structure and the thread makes serial number management more targeted, reduces cross-thread interference, and improves verification efficiency; the overall process dynamically updates and verifies the serial number, while ensuring the accuracy of data synchronization, realizing the orderly connection of the master and backup operations, providing a solid guarantee for the security and stability of the metadata mirroring, especially suitable for storage scenarios with strict requirements on data consistency.

[0036] Figure 3 Schematic diagram of the thread slot and sequence number mechanism provided by the embodiment of the present invention. Figure 3As shown, each thread is assigned a slot structure with a one-to-one binding. The slot structure stores a primary sequence number and a backtracking sequence number. The primary sequence number of the first node must correspond to the backtracking sequence number of the second node. When a mirroring request is sent, the primary sequence number is packaged into the mirroring message and then incremented by 1. The backtracking sequence number is used to verify the primary sequence number in the mirroring message at the receiving end. If the read backtracking sequence number and the parsed primary sequence number are equal, the message order is determined to be within the specified range (i.e., the message order is legal), and the mirroring request (e.g., writing data, updating metadata) is executed. The backtracking sequence number is then incremented by 1 (synchronized with the sending master controller). If the two are not equal (e.g., the sending end's sequence number is 1, while the receiving end's backtracking sequence number is 0), the message is considered out of order (possibly due to network delays or retransmissions) and is discarded to avoid data conflicts. This verification logic mechanically prevents out-of-order messages from violating data consistency. For example, if the sender sends messages in the order "Message 1 (sequence number 0) to Message 2 (sequence number 1)", if the backup controller receives Message 2 first, it will discard it due to a sequence number mismatch. Subsequent messages cannot be processed until Message 1 arrives and triggers a retroactive sequence number update, ensuring that data operations strictly follow the order sent. This thread slot and sequence number-based mechanism ensures that data operations on the primary and backup controllers are in the same order, eliminating metadata conflicts (such as duplicate writes and overwrite errors) caused by out-of-order operations.

[0037] Furthermore, in specific implementation, the above-mentioned metadata mirroring method provided in the embodiment of the present invention may also include: if a failure occurs in the standby controller, then when the standby controller joins the cluster after recovery, the master sequence number and the backtracking sequence number saved by the master controller itself are set to the initial value, the mirroring operation is restarted and a consistency check is performed; if a failure occurs in the master controller, then after recovery, when the standby controller joins the cluster, the master sequence number and the backtracking sequence number saved by the standby controller itself are set to the initial value, the mirroring operation is restarted and a consistency check is performed.

[0038] During implementation, if any of the primary and backup controllers fails and then recovers and joins the cluster, by resetting the master sequence number and backtracking sequence number of the corresponding end to the initial value, restarting the mirroring operation and performing a consistency check, the abnormal data or sequence number disorder that may have occurred during the failure can be completely cleared, and synchronization errors caused by inconsistent historical states can be avoided from the source; at the same time, the consistency check link can further ensure that the data baselines of the primary and backup ends are unified after recovery, laying a solid foundation for the orderly implementation of subsequent mirroring operations, effectively improving the system's fault tolerance and data synchronization accuracy in fault recovery scenarios, and ensuring the overall stability and continuity of the cluster.

[0039] Furthermore, in specific implementation, the metadata mirroring method provided in the embodiment of the present invention may also include: when the master controller and the backup controller fail at the same time, writing the slot structure-related data into a non-volatile memory; after the master controller and the backup controller recover, reading the slot structure-related data saved by themselves from the non-volatile memory, and restoring the main sequence number and retroactive sequence number before the failure; sending the main sequence number and retroactive sequence number saved by themselves to the backup controller, and receiving the main sequence number and retroactive sequence number saved by the backup control end, and performing a two-way check; judging whether there was an unfinished mirroring request before the failure based on the two-way check result; if so, triggering the re-execution of the unfinished mirroring request to complete the unsynchronized data.

[0040] During implementation, when both the primary and backup controllers fail simultaneously, slot structure data (including the primary and back-end serial numbers in each slot) is written to non-volatile memory (NVM). This allows for reliable preservation of critical operating data prior to the failure when a dual-end failure causes a system interruption, preventing permanent loss of core information due to power outages or crashes, and providing a data foundation for subsequent recovery. Non-volatile memory can be persistent memory, flash memory, or other non-volatile storage devices. After both the primary and backup controllers have completed fault repair and restarted, they each read their own saved slot data from the NVM and restore the primary and back-end serial numbers prior to the failure. This allows for accurate reconstruction of the system state prior to the failure, ensuring that the restored mirroring operation continues with the original process rather than starting from scratch, reducing synchronization confusion caused by state discontinuities.

[0041] After the primary and backup controllers recover, they send their stored sequence number information (primary and retroactive sequence numbers) to each other and perform a two-way verification: the primary controller sends its own primary and retroactive sequence number to the backup controller, and simultaneously receives the backup controller's sequence number; the backup controller does the same, comparing the two serial numbers. Based on the verification results, if an incomplete mirroring request is found before the failure (such as a message sent by the primary controller but not acknowledged by the backup controller), the redo mechanism is triggered to complete the unsynchronized data and ensure data consistency between the primary and backup controllers. After completing the verification and necessary redo, both the primary and backup controllers retain the sequence number recovered from the non-volatile memory (without resetting it to 0). By exchanging and comparing the sequence numbers stored by both parties, this two-way verification mechanism can accurately identify whether there were incomplete mirroring requests at the time of the failure, avoiding missed detections or misjudgments due to information asymmetry. Triggering re-execution of incomplete requests effectively completes the data that was not synchronized during the failure, completely eliminating the potential data inconsistency caused by dual-end failures.

[0042] The overall process solves the data consistency problem when both the primary and backup controllers fail at the same time, significantly improves the system's risk resistance under extreme failures, ensures the continuity and consistency of metadata mirroring, provides solid fault-tolerant support for the stable operation of the cluster, and further improves the reliability of the system.

[0043] In practical applications, the metadata mirroring method of the present invention realizes efficient transmission of mirrored messages and data consistency assurance in multi-threaded scenarios, and is particularly suitable for scenarios with strict requirements on low latency and high reliability, such as all-flash storage and distributed storage systems.

[0044] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0045] An embodiment of the present invention also provides a metadata mirroring device. Figure 4 This is a schematic diagram of the structure of the metadata mirroring device provided by an embodiment of the present invention. This embodiment is based on the perspective of functional modules, such as Figure 4 As shown, the device includes: The request processing module 10 is used to generate metadata insertion requests, distribute the metadata insertion requests to the corresponding slot structure according to the assigned thread, and encapsulate the metadata insertion requests into target image requests; the slot structure maintains a waiting list for temporarily storing image requests; The first detection module 11 is used to detect the status of the waiting list; when the status of the waiting list is empty and there is no mirror request being processed in the slot structure, the operation of the request forwarding module is executed; A request forwarding module 12 is configured to insert the target mirror request into a waiting list and forward the target mirror request to the standby controller; The second detection module 13 is used to re-check the status of the waiting list after the standby controller processes the target mirror request and returns an acknowledgment; if the status of the waiting list is not empty, execute the operation of the request aggregation module; if the waiting list is empty, wait for the next mirror request to be inserted; The request aggregation module 14 is configured to extract a corresponding number of mirror requests from the waiting list according to a set maximum number of requests, package the mirror requests into new mirror requests, and send the new mirror requests to the standby controller.

[0046] In the metadata mirroring device provided in the embodiment of the present invention, the interaction of the four modules can realize dynamic aggregation of metadata requests according to the correspondence between the slot structure and the thread, thereby avoiding the disordered sending of scattered requests. By setting the maximum number of requests to batch-package new mirroring requests, the number of communication interactions between the primary and backup controllers is greatly reduced, effectively improving the utilization rate of bandwidth resources and reducing communication overhead. In addition, by real-time detection of the waiting list status, new requests are forwarded in a timely manner when the waiting list is empty and there are no requests being processed. After receiving the confirmation information returned by the backup controller, batch extraction and processing are performed when the waiting list is not empty. This not only ensures a rapid response to a single urgent request, but also improves the overall mirroring efficiency through batch processing, and realizes flexible adaptation of scattered requests and batch requests. At the same time, it ensures the orderliness and reliability of metadata mirroring, provides efficient and stable support for data consistency between the primary and backup ends, and is particularly suitable for metadata synchronization requirements in high-concurrency scenarios.

[0047] Since the embodiments of the metadata mirroring apparatus correspond to the embodiments of the metadata mirroring method, the description of the features in the corresponding embodiments of the metadata mirroring apparatus can be found in the description of the corresponding embodiments of the metadata mirroring method, and will not be repeated here. The embodiments of the metadata mirroring apparatus and metadata mirroring method have the same beneficial effects as the aforementioned metadata mirroring method.

[0048] Furthermore, in a specific implementation, the metadata mirroring device provided in the embodiment of the present invention may also include: a slot establishment module for establishing an equal amount of slot structures according to a preset number of threads, and binding the slot structures to threads one-to-one. The slot structure may also maintain a master sequence number and a retrospective sequence number. The master sequence number serves as an identifier of the mirroring request sent by the master controller, and the retrospective sequence number serves as a reference value for the backup controller to verify the mirroring request; the master sequence number of the master controller and the retrospective sequence number of the backup controller correspond to each other.

[0049] Furthermore, in a specific implementation, in the metadata mirroring device provided in an embodiment of the present invention, the first detection module 11 can also be used to insert the target mirroring request into the waiting list and return when the status of the waiting list is non-empty; when the status of the waiting list is empty but the slot structure is processing the mirroring request, the target mirroring request is inserted into the waiting list and returned.

[0050] Furthermore, in a specific implementation, in the metadata mirroring device provided in an embodiment of the present invention, the request forwarding module 12 can be specifically used to read the current master sequence number stored by the master controller itself from the slot structure; package the target mirroring request and the current master sequence number into a mirroring message, and send the packaged mirroring message to the standby controller so that the standby controller processes the mirroring message; after the mirroring message is sent, the current master sequence number is incremented by a set value; wherein, the standby controller processes the mirroring message, including: after the standby controller receives the packaged mirroring message, it parses the packaged mirroring message to obtain the parsed master sequence number; reads the backtracking sequence number stored by the standby controller itself from the slot structure bound to the current thread; verifies the read backtracking sequence number with the parsed master sequence number; performs corresponding processing on the mirroring message according to the verification result; if the mirroring request is executed, the read backtracking sequence number is incremented by a set value.

[0051] Furthermore, in a specific implementation, the metadata mirroring device provided in an embodiment of the present invention also includes: a fault recovery module, which is used to set the master sequence number and the backtracking sequence number stored in the master controller itself to the initial value when the slave controller joins the cluster after recovery if a fault occurs in the backup controller, restart the mirroring operation and perform a consistency check; if a fault occurs in the master controller, it will join the cluster after recovery, set the master sequence number and the backtracking sequence number stored in the backup controller itself to the initial value, restart the mirroring operation and perform a consistency check.

[0052] Furthermore, in a specific implementation, in the above-mentioned metadata mirroring device provided in an embodiment of the present invention, the fault recovery module can also be used to write the slot structure-related data into a non-volatile memory when a fault occurs simultaneously in the primary controller and the backup controller; after the primary controller and the backup controller are recovered, the slot structure-related data saved by themselves are read from the non-volatile memory to restore the main sequence number and the retroactive sequence number before the fault; the main sequence number and the retroactive sequence number saved by themselves are sent to the backup controller, and the main sequence number and the retroactive sequence number saved by the backup control end are received to perform a two-way check; based on the two-way check result, it is determined whether there is an unfinished mirroring request before the fault; if so, the unfinished mirroring request is triggered to be re-executed to complete the unsynchronized data.

[0053] An embodiment of the present invention also provides an electronic device, including a master-side controller and a standby-side controller; the master-side controller is used to generate a metadata insertion request, distribute the metadata insertion request to a corresponding slot structure according to an assigned thread, and encapsulate the metadata insertion request into a target mirror request; the slot structure maintains a waiting list for temporarily storing mirror requests; the status of the waiting list is detected, and when the status of the waiting list is empty and there is no mirror request being processed in the slot structure, the target mirror request is inserted into the waiting list and forwarded to the standby-side controller; the standby-side controller is used to process the target mirror request and return a confirmation message to the master-side controller after processing; the master-side controller is further used to re-detect the status of the waiting list after receiving the confirmation message; if the status of the waiting list is not empty, a corresponding number of mirror requests are extracted from the waiting list according to a set maximum number of requests and packaged into new mirror requests, and the new mirror requests are sent to the standby-side controller; if the waiting list is empty, the next mirror request is waited for insertion.

[0054] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps of any of the above-mentioned metadata mirroring method embodiments when running.

[0055] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0056] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of any of the above-mentioned metadata mirroring method embodiments are implemented.

[0057] An embodiment of the present invention also provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned metadata mirroring method embodiments are implemented.

[0058] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0059] The above describes in detail the metadata mirroring method, apparatus, device, and medium provided by the present invention. This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are intended only to facilitate understanding of the present invention's method and core concepts. It should be noted that those skilled in the art may make various improvements and modifications to the present invention without departing from the principles of the present invention, and such improvements and modifications fall within the scope of protection of the present invention.

Claims

1. A metadata mirroring method, characterized in that: For the master controller, including: Generate a metadata insertion request, distribute the metadata insertion request to a corresponding slot structure according to an assigned thread, and encapsulate the metadata insertion request as a target mirror request; the slot structure maintains a waiting linked list for temporarily storing the mirror request; Detecting a state of the waiting list, and when the state of the waiting list is empty and the slot structure has no mirroring request being processed, inserting the target mirroring request into the waiting list and forwarding the target mirroring request to the standby controller; After the standby controller completes processing the target mirror request and returns a confirmation, re-detecting the status of the waiting list; If the state of the waiting list is not empty, a corresponding number of mirror requests are extracted from the waiting list according to the set maximum number of requests and packaged into new mirror requests, and the new mirror requests are sent to the standby controller; if the waiting list is empty, wait for the next mirror request to be inserted.

2. The metadata mirroring method according to claim 1, wherein: Before generating the metadata insert request, also include: An equal amount of slot structures are established according to the preset number of threads, and the slot structures are bound to the threads one-to-one.

3. The metadata mirroring method according to claim 1, wherein: After encapsulating the metadata insertion request into a target image request, the method further includes: When the state of the waiting list is non-empty, inserting the target image request into the waiting list and returning; When the state of the waiting linked list is empty but the slot structure is processing a mirror request, the target mirror request is inserted into the waiting linked list and returned.

4. The metadata mirroring method according to claim 1, wherein: The slot structure also maintains a primary sequence number and a backtracking sequence number; The primary sequence number is used as the identifier of the mirror request sent by the primary controller, and the retrospective sequence number is used as the reference value for the backup controller to verify the mirror request; the primary sequence number of the primary controller and the retrospective sequence number of the backup controller correspond to each other.

5. The metadata mirroring method according to claim 4, wherein: Forwarding the target mirroring request to the standby controller includes: Reading the current master sequence number stored in the master controller from the slot structure; Packing the target mirroring request and the current master sequence number into a mirroring message, and sending the packed mirroring message to the backup controller so that the backup controller processes the mirroring message; after the mirroring message is sent, the current master sequence number is automatically incremented by a set value; Among them, the standby controller processes the mirror message, including: after the standby controller receives the packaged mirror message, it parses the packaged mirror message to obtain the parsed main sequence number; reads the retrospective sequence number saved by the standby controller itself from the slot structure bound to the current thread; verifies the read retrospective sequence number with the parsed main sequence number; according to the verification result, performs corresponding processing on the mirror message; if the mirror request is executed, the read retrospective sequence number is incremented by the set value.

6. The metadata mirroring method according to claim 4, wherein: Also includes: If the standby controller fails, when the standby controller recovers and joins the cluster, the master sequence number and backtracking sequence number saved by the master controller are set to the initial values, the mirroring operation is restarted and a consistency check is performed; If the master controller fails, it joins the cluster after recovery, sets the master sequence number and the backtracking sequence number stored by the backup controller to the initial values, restarts the mirroring operation and performs a consistency check.

7. The metadata mirroring method according to claim 4, wherein: Also includes: When the primary controller and the backup controller fail at the same time, the slot structure related data is written into the non-volatile memory; After the primary controller and the backup controller are restored, the slot structure related data stored in the non-volatile memory are read to restore the primary sequence number and the backtracking sequence number before the failure; Send the primary sequence number and retrospective sequence number stored by itself to the standby controller, and receive the primary sequence number and retrospective sequence number stored by the standby controller, and perform two-way verification; Based on the bidirectional verification results, determine whether there are any unfinished mirroring requests before the failure; if so, trigger the re-execution of the unfinished mirroring request to complete the unsynchronized data.

8. A metadata mirroring device, characterized in that: include: A request processing module, configured to generate a metadata insertion request, distribute the metadata insertion request to a corresponding slot structure according to an assigned thread, and encapsulate the metadata insertion request into a target image request; The slot structure maintains a waiting list for temporarily storing image requests; A first detection module, configured to detect a status of the waiting list; When the state of the waiting linked list is empty and the slot structure has no mirror request being processed, executing the operation of the request forwarding module; The request forwarding module is configured to insert the target mirror request into the waiting linked list and forward the target mirror request to the standby controller; A second detection module is configured to re-detect the status of the waiting list after the backup controller completes processing the target mirroring request and returns a confirmation; if the status of the waiting list is not empty, execute the operation of the request aggregation module; if the waiting list is empty, wait for the insertion of the next mirroring request; The request aggregation module is configured to extract a corresponding number of mirror requests from the waiting list according to a set maximum number of requests, package the requests into new mirror requests, and send the new mirror requests to the standby controller.

9. An electronic device, characterized in that: include: Primary controller and backup controller; The master controller is configured to generate a metadata insertion request, distribute the metadata insertion request to a corresponding slot structure according to an assigned thread, and encapsulate the metadata insertion request into a target mirror request; The slot structure maintains a waiting list for temporarily storing image requests; Detecting a state of the waiting list, and when the state of the waiting list is empty and the slot structure has no mirroring request being processed, inserting the target mirroring request into the waiting list and forwarding the target mirroring request to the standby controller; The standby controller is configured to process the target mirroring request and return confirmation information to the primary controller after processing; The master-side controller is further configured to recheck the status of the waiting list after receiving the confirmation information; if the status of the waiting list is not empty, extract a corresponding number of mirror requests from the waiting list according to the set maximum number of requests and package them into new mirror requests, and send the new mirror requests to the standby-side controller; if the waiting list is empty, wait for the next mirror request to be inserted.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the metadata mirroring method according to any one of claims 1 to 7 are implemented.

Images