Metadata mirroring method, apparatus, device, and medium

Abstract

The application discloses a metadata mirroring method and device, equipment and medium, and relates to the technical field of data storage, which comprises the following steps: generating a metadata insertion request, utilizing the corresponding relationship between a slot structure and a thread to realize dynamic aggregation of the metadata request, and avoiding disordered sending of scattered requests; the slot structure maintains a waiting linked list for temporarily storing mirroring requests; by detecting the state of the waiting linked list, when the linked list is empty and there is no processing request, a new request is forwarded in time; after receiving the confirmation information returned by the backup controller, when the linked list is not empty, new mirroring requests are packaged in batches according to the set maximum number of requests and are sent to the backup controller, the number of communication interactions between the master and backup controllers is greatly reduced, the utilization rate of bandwidth resources and the overall mirroring efficiency are effectively improved, efficient and stable support is provided for the consistency of master and backup data, and the application is especially suitable for metadata synchronization requirements in a high-concurrency scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data storage, and in particular to a metadata mirroring method, device, equipment and medium. BACKGROUND

[0002] To ensure the high availability of metadata, storage systems generally use a mirroring mechanism. In a master-slave controller architecture, after the master controller performs a local insertion operation on certain metadata, it immediately mirrors and synchronizes the operation to the slave controller to maintain synchronization of the metadata at both ends of the master-slave controller. However, the traditional metadata mirroring method has an efficiency bottleneck. Since the mirroring process is carried out in units of single Key-Value (KV) operations, that is, after each local metadata insertion is completed, a separate mirroring request to the slave end is initiated. This directly leads to a significant increase in the number of communications between controller nodes, and frequent interactions occupy a large amount of communication bandwidth between nodes. SUMMARY

[0003] The present application provides a metadata mirroring method, device, equipment and medium to at least solve the problem of metadata mirroring efficiency between controller nodes in related technologies.

[0004] The present application provides a metadata mirroring method for a master controller, comprising:

[0005] Generating a metadata insertion request, distributing the metadata insertion request to a corresponding slot structure according to an assigned thread, and encapsulating the metadata insertion request as a target mirroring request; the slot structure maintains a waiting linked list for temporarily storing mirroring requests;

[0006] Detecting the state of the waiting linked list, and when the state of the waiting linked list is empty and the slot structure has no mirroring request being processed, inserting the target mirroring request into the waiting linked list and forwarding the target mirroring request to a slave controller;

[0007] After the slave controller processes the target mirroring request and returns an acknowledgement, the state of the waiting linked list is re-detected;

[0008] If the state of the waiting linked list is non-empty, then a corresponding number of mirroring requests are extracted from the waiting linked list according to a set maximum number of requests and packaged as a new mirroring request, and the new mirroring request is sent to the slave controller; if the waiting linked list is empty, then the insertion of the next mirroring request is waited for.

[0009] The present application also provides a metadata mirroring device, comprising:

[0010] a request processing module, configured to generate a metadata insertion request, distribute the metadata insertion request to a corresponding slot structure according to an assigned thread, and encapsulate the metadata insertion request into a target mirror request; the slot structure maintains a waiting linked list for temporarily storing mirror requests;

[0011] A first detection module is configured to detect a state of the waiting list; when the state of the waiting list is empty and there is no mirror request being processed in the slot structure, execute an operation of the request forwarding module;

[0012] The request forwarding module is configured to insert the target mirror request into the waiting linked list and forward the target mirror request to the standby controller;

[0013] A second detection module is configured to re-detect the status of the waiting list after the backup controller completes processing the target mirroring request and returns a confirmation; if the status of the waiting list is not empty, execute the operation of the request aggregation module; if the waiting list is empty, wait for the insertion of the next mirroring request;

[0014] The request aggregation module is configured to extract a corresponding number of mirror requests from the waiting list according to a set maximum number of requests, package the requests into new mirror requests, and send the new mirror requests to the standby controller.

[0015] The present invention also provides an electronic device, comprising: a master-end controller and a backup-end controller;

[0016] The master controller is configured to generate a metadata insertion request, distribute the metadata insertion request to a corresponding slot structure according to an assigned thread, and encapsulate the metadata insertion request into a target mirroring request; the slot structure maintains a waiting linked list for temporarily storing mirroring requests; detect a status of the waiting linked list, and when the waiting linked list is empty and the slot structure has no mirroring request being processed, insert the target mirroring request into the waiting linked list and forward the target mirroring request to the standby controller;

[0017] The standby controller is configured to process the target mirroring request and return confirmation information to the primary controller after processing;

[0018] The master-side controller is further configured to recheck the status of the waiting list after receiving the confirmation information; if the status of the waiting list is not empty, extract a corresponding number of mirror requests from the waiting list according to the set maximum number of requests and package them into new mirror requests, and send the new mirror requests to the standby-side controller; if the waiting list is empty, wait for the next mirror request to be inserted.

[0019] The present invention also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned metadata mirroring methods are implemented.

[0020] The metadata mirroring method of the present invention accurately distributes metadata insertion requests to corresponding slot structures through a thread allocation mechanism, and combines the use of a waiting list to temporarily store requests and dynamically manage processing status, thereby bringing significant benefits in many aspects: on the one hand, dynamic aggregation of metadata requests is achieved based on the correspondence between the slot structure and the thread, avoiding the disordered sending of scattered requests. New mirroring requests are batched by setting a maximum number of requests, significantly reducing the number of communication interactions between the primary and standby controllers, effectively improving the utilization of bandwidth resources, and reducing communication overhead. On the other hand, by real-time detection of the waiting list status, new requests are promptly forwarded when the waiting list is empty and there are no requests being processed. After receiving confirmation information returned by the standby controller, batch extraction and processing are performed when the waiting list is not empty. This not only ensures a rapid response to a single urgent request, but also improves the overall mirroring efficiency through batch processing, and realizes flexible adaptation of scattered requests to batch requests. At the same time, the entire process ensures the orderliness and reliability of metadata mirroring, provides efficient and stable support for data consistency between the primary and standby ends, and is particularly suitable for metadata synchronization requirements in high-concurrency scenarios.

[0021] In addition, the present invention also provides a corresponding metadata mirroring device, electronic device and computer-readable storage medium for the metadata mirroring method, which have the same or corresponding technical features as the above-mentioned metadata mirroring method and have the same effects as above. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0023] Figure 1 A flowchart of a metadata mirroring method provided by an embodiment of the present invention;

[0024] Figure 2 A schematic diagram corresponding to the metadata aggregate image provided by an embodiment of the present invention;

[0025] Figure 3 A schematic diagram illustrating the correspondence between thread slots and the sequence number mechanism provided in an embodiment of the present invention;

[0026] Figure 4 A schematic structural diagram of a metadata mirroring device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0028] It should be noted that, in the description of the present invention, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. The terms "first," "second," etc., in the present invention are used to distinguish similar objects, and are not used to describe a particular order or precedence.

[0029] In order to enable those skilled in the art to better understand the solutions of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0030] In conjunction with the specific application environment architecture or specific hardware architecture on which the execution of the metadata mirroring method depends, the specific application environment architecture or specific hardware architecture is described herein.

[0031] An embodiment of the present invention provides a metadata mirroring method, which is described in detail in conjunction with the execution flow of the metadata mirroring method. Figure 1 A flowchart of a metadata mirroring method provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method is used for the master controller and includes the following steps:

[0032] S101. Generate a metadata insertion request, distribute the metadata insertion request to a corresponding slot structure according to an assigned thread, and encapsulate the metadata insertion request as a target mirror request; the slot structure maintains a waiting linked list for temporarily storing the mirror request.

[0033] It should be noted that a slot structure refers to storage space used to store thread-related information. For example, a slot structure can store thread status, execution status flags, and associated task queue pointers, facilitating the system's unified scheduling and monitoring of threads. During step S101, after data aggregation and flushing, batches of metadata insertion requests are generated. These metadata insertion requests are distributed to the corresponding slot structures based on the assigned threads, achieving precise binding between requests, threads, and slot structures, avoiding resource contention and improving orderly processing. The wait list maintained in the slot structure can serve as a temporary storage container for mirroring requests, providing buffer space for mirroring requests. This ensures that mirroring requests are not lost due to instantaneous high concurrency and allows for flexible adjustment for subsequent batch processing or dynamic scheduling.

[0034] The metadata of the present invention can be all-flash storage metadata. All-flash storage is a storage system built entirely on flash memory media. Unlike traditional mechanical hard disk storage or hybrid storage, it does not contain any mechanical components, and all data reading and writing relies on flash memory chips. All-flash storage metadata refers to the data used to describe and manage the actual data in an all-flash storage system, which is equivalent to the directory or navigation information of the storage system. It records key information, such as the storage location of the actual data, the attributes of the data, the associations between the data, etc. Of course, the metadata of the present invention can also be other types of metadata, such as metadata of a distributed storage system, which is not limited here.

[0035] S102 , detecting the state of the waiting list. When the state of the waiting list is empty and there is no mirroring request being processed in the slot structure, inserting the target mirroring request into the waiting list and forwarding the target mirroring request to the standby controller.

[0036] It's important to note that the master controller monitors the state of the wait list (whether it's empty) and the slot structure's processing status (whether there are any mirror requests being processed) in real time. These two conditions together form the basis for forwarding decisions, ensuring that forwarding operations don't conflict with the mirroring process. When the wait list is empty and no mirror requests are being processed, indicating that the slot structure is idle, the target mirror request is inserted into the wait list and immediately forwarded to the slave controller. This ensures that individual requests are responded to quickly, avoiding delays, and prevents conflicts or data inconsistencies that could result from concurrent processing through state verification.

[0037] S103: After the standby controller processes the target mirror request and returns a confirmation, it rechecks the status of the waiting list.

[0038] In implementation, after the standby controller completes processing of the target mirror request and returns confirmation information, the master controller does not directly enter the next round of request processing, but instead constructs a responsive processing node by re-detecting the status of the waiting list. In this way, the standby controller uses the processing confirmation as a trigger signal to dynamically sense whether there are newly accumulated mirror requests in the waiting list, ensuring that the master can flexibly select subsequent processing strategies based on the actual queue status.

[0039] S104: If the state of the waiting list is not empty, extract the corresponding number of mirror requests from the waiting list according to the set maximum number of requests and package them into new mirror requests, and send the new mirror requests to the standby controller; if the waiting list is empty, wait for the next mirror request to be inserted.

[0040] During implementation, when executing step S104, the master controller detects that the waiting list is not empty and extracts a corresponding number of mirror requests from the list, based on the set maximum number of requests. These requests are then packaged and integrated into new mirror requests and sent to the slave controller. This batch processing method effectively reduces the number of communications between the master and slave terminals, improving bandwidth utilization and processing efficiency. When the waiting list is empty, the system enters a waiting state to accept the next inserted mirror request, avoiding inefficient resource consumption and the sending of empty requests. By performing different operations based on the linked list status, this method achieves efficient batch processing of accumulated requests while ensuring timely responses to new requests. This balance between resource utilization and processing efficiency ensures the orderliness of the metadata mirroring process.

[0041] It should be added that, when it is detected that the waiting list is in a non-empty state, the present invention can obtain the set maximum number of requests based on the maximum length limit of the mirror message; if the total number of mirror requests in the waiting list exceeds the set maximum number of requests, the mirror requests with the set maximum number of requests are extracted; if the total number of mirror requests in the waiting list does not exceed the set maximum number of requests, all the mirror requests in the waiting list are extracted. The extracted mirror requests are then packaged into new mirror requests and sent to the backup controller. This process not only avoids the sending of empty requests by checking the status of the list, but also adapts to the length limit of the mirror message through a flexible strategy of quota interception or full processing, avoiding data overflow or transmission errors, and achieving a balance between efficient processing and standardized transmission, while ensuring the orderly processing of requests in the waiting list to prevent backlogs.

[0042] The metadata mirroring method provided by the embodiment of the application has the following advantages: on the one hand, the dynamic aggregation of metadata requests is realized according to the correspondence between the slot structure and the thread, the disordered sending of scattered requests is avoided, the number of communication interactions between the master controller and the backup controller is greatly reduced by setting the maximum number of requests to pack the new mirror image request, the utilization rate of bandwidth resources is effectively improved, and the communication overhead is reduced; on the other hand, the state of the waiting chain table is detected in real time, the new request is forwarded in time when the waiting chain table is empty and there is no processing request, and the batch processing is extracted when the waiting chain table is not empty after receiving the confirmation information returned by the backup controller, so that the fast response of a single urgent request is ensured, the overall mirroring efficiency is improved through batch processing, the flexible adaptation of scattered requests and batch requests is realized, and the orderliness and reliability of metadata mirroring are ensured, thereby providing efficient and stable support for the data consistency of the master and backup ends, and the metadata synchronization demand in a high-concurrency scenario is especially suitable.

[0043] Further, in the metadata mirroring method provided by the embodiment of the application, before the step S101 of generating the metadata insertion request is executed, the following step can also be included: establishing an equal number of slot structures according to a preset number of threads, and binding the slot structures and the threads one by one.

[0044] In the implementation, the application can apply for slot structure resources according to the number of threads, and make one-to-one binding, so that by allocating independent slot structures for each thread, the threads do not cross and interfere with each other, the competition and conflict of multiple threads for the same resource can be effectively avoided, the synchronization overhead between threads is reduced, and the processing efficiency is improved; meanwhile, the one-to-one binding relationship makes the request processing path clearer, facilitates the tracking of the request flow state corresponding to each thread, reduces the problem troubleshooting difficulty, provides stable and efficient foundation support for subsequent request aggregation, distribution and mirroring forwarding, and can significantly improve the throughput and stability of overall processing in a high-concurrency scenario.

[0045] Further, in the metadata mirroring method provided by the embodiment of the application, after the step S101 of encapsulating the metadata insertion request into the target mirror image request is executed, the following steps can also be included: when the state of the waiting chain table is not empty, the target mirror image request is inserted into the waiting chain table and returned; and when the state of the waiting chain table is empty but the slot structure is processing a mirror image request, the target mirror image request is inserted into the waiting chain table and returned.

[0046] In practice, the present invention uniformly inserts the target mirror request into the waiting list and returns the operation, ensuring that the request is securely stored and not lost, regardless of whether the waiting list is not empty or the list is empty but the slot is busy, thereby avoiding the risk of request discard due to a busy slot or a non-empty queue. At the same time, the judgment process for request access is simplified, and the orderly accumulation of requests can be achieved without the need for additional complex logic, which not only ensures that the currently processed mirror request is not interfered with, but also reserves pending requests for subsequent batch processing or timely forwarding during idle time.

[0047] Furthermore, in the metadata mirroring method provided in the embodiments of the present invention, the slot structure can also maintain a master sequence number (OwnerSequenceNo) and a backtracking sequence number (BackSequenceNo). The master sequence number serves as an identifier for the mirroring request sent by the master controller, while the backtracking sequence number serves as a reference value for verifying the mirroring request by the backup controller. The master sequence number of the master controller and the backtracking sequence number of the backup controller correspond to each other.

[0048] In practice, the backtracking sequence number is a concept corresponding to the primary sequence number. It serves as a sequence number for reverse operations and complements the primary sequence number, jointly ensuring the integrity of data transfer or object management. The primary sequence number uniquely identifies mirroring requests sent by the primary controller, while the backtracking sequence number serves as a baseline for verifying mirroring requests on the backup controller. The precise correspondence between the primary sequence number and the backup controller's backtracking sequence number ensures the traceability of mirroring requests and the reliability of data synchronization between the primary and backup controllers. Both the primary and backtracking sequence numbers can be initially set to 0.

[0049] Figure 2 Schematic diagram corresponding to the metadata aggregate image provided by the embodiment of the present invention. Figure 2 As shown, the first node and the second node can correspond to the master controller and the backup controller respectively. A waiting list is added to the sending end of the master controller, which can be used to control the adaptive aggregate mirror request. Due to the maximum length limit of the mirror message between nodes, the aggregate mirror will also set the maximum number of mirror requests; through adaptive aggregate mirroring, an efficient mirroring effect with zero waiting and large bandwidth can be achieved.

[0050] Furthermore, in a specific implementation, in the metadata mirroring method provided in an embodiment of the present invention, step S102 forwards the target mirroring request to the standby controller, which may specifically include: reading the current master sequence number stored by the master controller itself from the slot structure; packaging the target mirroring request and the current master sequence number into a mirroring message, and sending the packaged mirroring message to the standby controller so that the standby controller processes the mirroring message; after the mirroring message is sent, the current master sequence number is incremented by a set value; wherein, the standby controller processes the mirroring message, including: after the standby controller receives the packaged mirroring message, it parses the packaged mirroring message to obtain the parsed master sequence number; reads the backtracking sequence number stored by the standby controller itself from the slot structure bound to the current thread; verifies the read backtracking sequence number with the parsed master sequence number; performs corresponding processing on the mirroring message according to the verification result; if the mirroring request is executed, the read backtracking sequence number is incremented by a set value.

[0051] During implementation, the master-side controller packages the mirror message with the current master serial number and automatically increments the set value (such as 1) after sending it. Combined with the verification logic based on the backtracking serial number on the equipment side, this effectively ensures the order and consistency of the metadata mirroring on the master and backup sides, avoiding message loss or disorder problems; the self-increment rule of the serial number enables each interaction to verify the integrity of the message through numerical comparison, providing an accurate verification basis for the reliability of data synchronization; at the same time, the binding relationship between the slot structure and the thread makes serial number management more targeted, reduces cross-thread interference, and improves verification efficiency; the overall process dynamically updates and verifies the serial number, while ensuring the accuracy of data synchronization, realizing the orderly connection of the master and backup operations, providing a solid guarantee for the security and stability of the metadata mirroring, especially suitable for storage scenarios with strict requirements on data consistency.

[0052] Figure 3 Schematic diagram of the thread slot and sequence number mechanism provided by the embodiment of the present invention. Figure 3As shown, each thread is assigned a slot structure with a one-to-one binding. The slot structure stores a primary sequence number and a backtracking sequence number. The primary sequence number of the first node must correspond to the backtracking sequence number of the second node. When a mirroring request is sent, the primary sequence number is packaged into the mirroring message and then incremented by 1. The backtracking sequence number is used to verify the primary sequence number in the mirroring message at the receiving end. If the read backtracking sequence number and the parsed primary sequence number are equal, the message order is determined to be within the specified range (i.e., the message order is legal), and the mirroring request (e.g., writing data, updating metadata) is executed. The backtracking sequence number is then incremented by 1 (synchronized with the sending master controller). If the two are not equal (e.g., the sending end's sequence number is 1, while the receiving end's backtracking sequence number is 0), the message is considered out of order (possibly due to network delays or retransmissions) and is discarded to avoid data conflicts. This verification logic mechanically prevents out-of-order messages from violating data consistency. For example, if the sender sends messages in the order "Message 1 (sequence number 0) to Message 2 (sequence number 1)", if the backup controller receives Message 2 first, it will discard it due to a sequence number mismatch. Subsequent messages cannot be processed until Message 1 arrives and triggers a retroactive sequence number update, ensuring that data operations strictly follow the order sent. This thread slot and sequence number-based mechanism ensures that data operations on the primary and backup controllers are in the same order, eliminating metadata conflicts (such as duplicate writes and overwrite errors) caused by out-of-order operations.

[0053] Furthermore, in specific implementation, the above-mentioned metadata mirroring method provided in the embodiment of the present invention may also include: if a failure occurs in the standby controller, then when the standby controller joins the cluster after recovery, the master sequence number and the backtracking sequence number saved by the master controller itself are set to the initial value, the mirroring operation is restarted and a consistency check is performed; if a failure occurs in the master controller, then after recovery, when the standby controller joins the cluster, the master sequence number and the backtracking sequence number saved by the standby controller itself are set to the initial value, the mirroring operation is restarted and a consistency check is performed.

[0054] During implementation, if any of the primary and backup controllers fails and then recovers and joins the cluster, by resetting the master sequence number and backtracking sequence number of the corresponding end to the initial value, restarting the mirroring operation and performing a consistency check, the abnormal data or sequence number disorder that may have occurred during the failure can be completely cleared, and synchronization errors caused by inconsistent historical states can be avoided from the source; at the same time, the consistency check link can further ensure that the data baselines of the primary and backup ends are unified after recovery, laying a solid foundation for the orderly implementation of subsequent mirroring operations, effectively improving the system's fault tolerance and data synchronization accuracy in fault recovery scenarios, and ensuring the overall stability and continuity of the cluster.

[0055] Furthermore, in specific implementation, the metadata mirroring method provided in the embodiment of the present invention may also include: when the master controller and the backup controller fail at the same time, writing the slot structure-related data into a non-volatile memory; after the master controller and the backup controller recover, reading the slot structure-related data saved by themselves from the non-volatile memory, and restoring the main sequence number and retroactive sequence number before the failure; sending the main sequence number and retroactive sequence number saved by themselves to the backup controller, and receiving the main sequence number and retroactive sequence number saved by the backup control end, and performing a two-way check; judging whether there was an unfinished mirroring request before the failure based on the two-way check result; if so, triggering the re-execution of the unfinished mirroring request to complete the unsynchronized data.

[0056] During implementation, when both the primary and backup controllers fail simultaneously, slot structure data (including the primary and back-end serial numbers in each slot) is written to non-volatile memory (NVM). This allows for reliable preservation of critical operating data prior to the failure when a dual-end failure causes a system interruption, preventing permanent loss of core information due to power outages or crashes, and providing a data foundation for subsequent recovery. Non-volatile memory can be persistent memory, flash memory, or other non-volatile storage devices. After both the primary and backup controllers have completed fault repair and restarted, they each read their own saved slot data from the NVM and restore the primary and back-end serial numbers prior to the failure. This allows for accurate reconstruction of the system state prior to the failure, ensuring that the restored mirroring operation continues with the original process rather than starting from scratch, reducing synchronization confusion caused by state discontinuities.

[0057] After the primary and backup controllers recover, they send their stored sequence number information (primary and retroactive sequence numbers) to each other and perform a two-way verification: the primary controller sends its own primary and retroactive sequence number to the backup controller, and simultaneously receives the backup controller's sequence number; the backup controller does the same, comparing the two serial numbers. Based on the verification results, if an incomplete mirroring request is found before the failure (such as a message sent by the primary controller but not acknowledged by the backup controller), the redo mechanism is triggered to complete the unsynchronized data and ensure data consistency between the primary and backup controllers. After completing the verification and necessary redo, both the primary and backup controllers retain the sequence number recovered from the non-volatile memory (without resetting it to 0). By exchanging and comparing the sequence numbers stored by both parties, this two-way verification mechanism can accurately identify whether there were incomplete mirroring requests at the time of the failure, avoiding missed detections or misjudgments due to information asymmetry. Triggering re-execution of incomplete requests effectively completes the data that was not synchronized during the failure, completely eliminating the potential data inconsistency caused by dual-end failures.

[0058] The whole process solves the data consistency problem when the master controller and the standby controller fail at the same time, significantly improves the risk resistance of the system under extreme failure, guarantees the continuity and consistency of the metadata mirror, provides solid fault tolerance support for the stable operation of the cluster, and further improves the reliability of the system.

[0059] In actual application, the metadata mirroring method of the application realizes efficient transmission of mirroring messages and data consistency guarantee in a multi-thread scenario, and is especially suitable for a full flash storage, a distributed storage system and other scenes with strict requirements on low delay and high reliability.

[0060] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment.

[0061] Embodiments of the application also provide a metadata mirroring device. Figure 4 The structure diagram of the metadata mirroring device provided by the embodiments of the application is shown in the figure. Figure 4 As shown in the figure, the device comprises:

[0062] The request processing module 10 is used for generating a metadata insertion request, distributing the metadata insertion request to the corresponding slot structure according to the allocated thread, and encapsulating the metadata insertion request as a target mirroring request; the slot structure maintains a waiting linked list for temporarily storing mirroring requests;

[0063] The first detection module 11 is used for detecting the state of the waiting linked list; when the state of the waiting linked list is empty and the slot structure has no mirroring request being processed, the operation of the request forwarding module is executed;

[0064] The request forwarding module 12 is used for inserting the target mirroring request into the waiting linked list and forwarding the target mirroring request to the standby controller;

[0065] The second detection module 13 is used for re-detecting the state of the waiting linked list after the standby controller processes the target mirroring request and returns an acknowledgement; if the state of the waiting linked list is not empty, the operation of the request aggregation module is executed; if the waiting linked list is empty, the next insertion of the mirroring request is waited for;

[0066] The request aggregation module 14 is used for extracting a corresponding number of mirroring requests from the waiting linked list according to the set maximum number of requests and packing them as a new mirroring request, and sending the new mirroring request to the standby controller.

[0067] In the metadata mirroring device provided in the embodiment of the present invention, the interaction of the four modules can realize dynamic aggregation of metadata requests according to the correspondence between the slot structure and the thread, thereby avoiding the disordered sending of scattered requests. By setting the maximum number of requests to batch-package new mirroring requests, the number of communication interactions between the primary and backup controllers is greatly reduced, effectively improving the utilization rate of bandwidth resources and reducing communication overhead. In addition, by real-time detection of the waiting list status, new requests are forwarded in a timely manner when the waiting list is empty and there are no requests being processed. After receiving the confirmation information returned by the backup controller, batch extraction and processing are performed when the waiting list is not empty. This not only ensures a rapid response to a single urgent request, but also improves the overall mirroring efficiency through batch processing, and realizes flexible adaptation of scattered requests and batch requests. At the same time, it ensures the orderliness and reliability of metadata mirroring, provides efficient and stable support for data consistency between the primary and backup ends, and is particularly suitable for metadata synchronization requirements in high-concurrency scenarios.

[0068] Since the embodiments of the metadata mirroring apparatus correspond to the embodiments of the metadata mirroring method, the description of the features in the corresponding embodiments of the metadata mirroring apparatus can be found in the description of the corresponding embodiments of the metadata mirroring method, and will not be repeated here. The embodiments of the metadata mirroring apparatus and metadata mirroring method have the same beneficial effects as the aforementioned metadata mirroring method.

[0069] Furthermore, in a specific implementation, the metadata mirroring device provided in the embodiment of the present invention may also include: a slot establishment module for establishing an equal amount of slot structures according to a preset number of threads, and binding the slot structures to threads one-to-one. The slot structure may also maintain a master sequence number and a retrospective sequence number. The master sequence number serves as an identifier of the mirroring request sent by the master controller, and the retrospective sequence number serves as a reference value for the backup controller to verify the mirroring request; the master sequence number of the master controller and the retrospective sequence number of the backup controller correspond to each other.

[0070] Furthermore, in a specific implementation, in the metadata mirroring device provided in an embodiment of the present invention, the first detection module 11 can also be used to insert the target mirroring request into the waiting list and return when the status of the waiting list is non-empty; when the status of the waiting list is empty but the slot structure is processing the mirroring request, the target mirroring request is inserted into the waiting list and returned.

[0071] Furthermore, in a specific implementation, in the metadata mirroring device provided in an embodiment of the present invention, the request forwarding module 12 can be specifically used to read the current master sequence number stored by the master controller itself from the slot structure; package the target mirroring request and the current master sequence number into a mirroring message, and send the packaged mirroring message to the standby controller so that the standby controller processes the mirroring message; after the mirroring message is sent, the current master sequence number is incremented by a set value; wherein, the standby controller processes the mirroring message, including: after the standby controller receives the packaged mirroring message, it parses the packaged mirroring message to obtain the parsed master sequence number; reads the backtracking sequence number stored by the standby controller itself from the slot structure bound to the current thread; verifies the read backtracking sequence number with the parsed master sequence number; performs corresponding processing on the mirroring message according to the verification result; if the mirroring request is executed, the read backtracking sequence number is incremented by a set value.

[0072] Furthermore, in a specific implementation, the metadata mirroring device provided in an embodiment of the present invention also includes: a fault recovery module, which is used to set the master sequence number and the backtracking sequence number stored in the master controller itself to the initial value when the slave controller joins the cluster after recovery if a fault occurs in the backup controller, restart the mirroring operation and perform a consistency check; if a fault occurs in the master controller, it will join the cluster after recovery, set the master sequence number and the backtracking sequence number stored in the backup controller itself to the initial value, restart the mirroring operation and perform a consistency check.

[0073] Furthermore, in a specific implementation, in the above-mentioned metadata mirroring device provided in an embodiment of the present invention, the fault recovery module can also be used to write the slot structure-related data into a non-volatile memory when a fault occurs simultaneously in the primary controller and the backup controller; after the primary controller and the backup controller are recovered, the slot structure-related data saved by themselves are read from the non-volatile memory to restore the main sequence number and the retroactive sequence number before the fault; the main sequence number and the retroactive sequence number saved by themselves are sent to the backup controller, and the main sequence number and the retroactive sequence number saved by the backup control end are received to perform a two-way check; based on the two-way check result, it is determined whether there is an unfinished mirroring request before the fault; if so, the unfinished mirroring request is triggered to be re-executed to complete the unsynchronized data.

[0074] The embodiment of the present application also provides an electronic device, comprising a master controller and a backup controller; the master controller is used for generating a metadata insertion request, distributing the metadata insertion request into a corresponding slot structure according to an assigned thread, and encapsulating the metadata insertion request into a target image request; the slot structure maintains a waiting linked list used for temporarily storing the image request; the state of the waiting linked list is detected, and when the state of the waiting linked list is empty and the slot structure has no image request being processed, the target image request is inserted into the waiting linked list and the target image request is forwarded to the backup controller; the backup controller is used for processing the target image request and returning an acknowledgement information to the master controller after the processing; the master controller is also used for re-detecting the state of the waiting linked list after receiving the acknowledgement information; if the state of the waiting linked list is not empty, a corresponding number of image requests are extracted from the waiting linked list according to a set maximum number of requests and are packaged into a new image request, and the new image request is sent to the backup controller; if the waiting linked list is empty, the next insertion of the image request is waited for.

[0075] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is configured to execute the steps in any of the above metadata image method embodiments when running.

[0076] In an example embodiment, the above computer readable storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0077] The embodiment of the present application also provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to realize the steps in any of the above metadata image method embodiments.

[0078] The embodiment of the present application also provides another computer program product, which comprises a non-volatile computer readable storage medium, and the non-volatile computer readable storage medium stores a computer program, and the computer program is executed by a processor to realize the steps in any of the above metadata image method embodiments.

[0079] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0080] The above describes in detail the metadata mirroring method, apparatus, device, and medium provided by the present invention. This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are intended only to facilitate understanding of the present invention's method and core concepts. It should be noted that those skilled in the art may make various improvements and modifications to the present invention without departing from the principles of the present invention, and such improvements and modifications fall within the scope of protection of the present invention.

Claims

1. A metadata mirroring method, characterized in that: For the master controller, including: Generate a metadata insertion request, distribute the metadata insertion request to a corresponding slot structure according to an assigned thread, and encapsulate the metadata insertion request as a target mirror request; the slot structure maintains a waiting linked list for temporarily storing the mirror request; Detecting a state of the waiting list, and when the state of the waiting list is empty and the slot structure has no mirroring request being processed, inserting the target mirroring request into the waiting list and forwarding the target mirroring request to the standby controller; After the standby controller completes processing the target mirror request and returns a confirmation, re-detecting the status of the waiting list; If the state of the waiting list is not empty, a corresponding number of mirror requests are extracted from the waiting list according to the set maximum number of requests and packaged into new mirror requests, and the new mirror requests are sent to the standby controller; if the waiting list is empty, wait for the next mirror request to be inserted.

2. The metadata mirroring method according to claim 1, wherein: Before generating the metadata insert request, also include: An equal amount of slot structures are established according to the preset number of threads, and the slot structures are bound to the threads one-to-one.

3. The metadata mirroring method according to claim 1, wherein: After encapsulating the metadata insertion request into a target image request, the method further includes: When the state of the waiting list is non-empty, inserting the target image request into the waiting list and returning; When the state of the waiting linked list is empty but the slot structure is processing a mirror request, the target mirror request is inserted into the waiting linked list and returned.

4. The metadata mirroring method according to claim 1, wherein: The slot structure also maintains a primary sequence number and a backtracking sequence number; The primary sequence number is used as the identifier of the mirror request sent by the primary controller, and the retrospective sequence number is used as the reference value for the backup controller to verify the mirror request; the primary sequence number of the primary controller and the retrospective sequence number of the backup controller correspond to each other.

5. The metadata mirroring method according to claim 4, wherein: Forwarding the target mirroring request to the standby controller includes: Reading the current master sequence number stored in the master controller from the slot structure; Packing the target mirroring request and the current master sequence number into a mirroring message, and sending the packed mirroring message to the backup controller so that the backup controller processes the mirroring message; after the mirroring message is sent, the current master sequence number is automatically incremented by a set value; Among them, the standby controller processes the mirror message, including: after the standby controller receives the packaged mirror message, it parses the packaged mirror message to obtain the parsed main sequence number; reads the retrospective sequence number saved by the standby controller itself from the slot structure bound to the current thread; verifies the read retrospective sequence number with the parsed main sequence number; according to the verification result, performs corresponding processing on the mirror message; if the mirror request is executed, the read retrospective sequence number is incremented by the set value.

6. The metadata mirroring method according to claim 4, wherein: Also includes: If the standby controller fails, when the standby controller recovers and joins the cluster, the master sequence number and backtracking sequence number saved by the master controller are set to the initial values, the mirroring operation is restarted and a consistency check is performed; If the master controller fails, it joins the cluster after recovery, sets the master sequence number and the backtracking sequence number stored by the backup controller to the initial values, restarts the mirroring operation and performs a consistency check.

7. The metadata mirroring method according to claim 4, wherein: Also includes: When the primary controller and the backup controller fail at the same time, the slot structure related data is written into the non-volatile memory; After the primary controller and the backup controller are restored, the slot structure related data stored in the non-volatile memory are read to restore the primary sequence number and the backtracking sequence number before the failure; Send the primary sequence number and retrospective sequence number stored by itself to the standby controller, and receive the primary sequence number and retrospective sequence number stored by the standby controller, and perform two-way verification; Based on the bidirectional verification results, determine whether there are any unfinished mirroring requests before the failure; if so, trigger the re-execution of the unfinished mirroring request to complete the unsynchronized data.

8. A metadata mirroring device, characterized in that: include: A request processing module, configured to generate a metadata insertion request, distribute the metadata insertion request to a corresponding slot structure according to an assigned thread, and encapsulate the metadata insertion request into a target image request; The slot structure maintains a waiting list for temporarily storing image requests; A first detection module, configured to detect a status of the waiting list; When the state of the waiting linked list is empty and the slot structure has no mirror request being processed, executing the operation of the request forwarding module; The request forwarding module is configured to insert the target mirror request into the waiting linked list and forward the target mirror request to the standby controller; A second detection module is configured to re-detect the status of the waiting list after the backup controller completes processing the target mirroring request and returns a confirmation; if the status of the waiting list is not empty, execute the operation of the request aggregation module; if the waiting list is empty, wait for the insertion of the next mirroring request; The request aggregation module is configured to extract a corresponding number of mirror requests from the waiting list according to a set maximum number of requests, package the requests into new mirror requests, and send the new mirror requests to the standby controller.

9. An electronic device, characterized in that: include: Primary controller and backup controller; The master controller is configured to generate a metadata insertion request, distribute the metadata insertion request to a corresponding slot structure according to an assigned thread, and encapsulate the metadata insertion request into a target mirror request; The slot structure maintains a waiting list for temporarily storing image requests; Detecting a state of the waiting list, and when the state of the waiting list is empty and the slot structure has no mirroring request being processed, inserting the target mirroring request into the waiting list and forwarding the target mirroring request to the standby controller; The standby controller is configured to process the target mirroring request and return confirmation information to the primary controller after processing; The master-side controller is further configured to recheck the status of the waiting list after receiving the confirmation information; if the status of the waiting list is not empty, extract a corresponding number of mirror requests from the waiting list according to the set maximum number of requests and package them into new mirror requests, and send the new mirror requests to the standby-side controller; if the waiting list is empty, wait for the next mirror request to be inserted.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the metadata mirroring method according to any one of claims 1 to 7 are implemented.

Images