Method, device and equipment for synchronizing metadata in cluster and readable storage medium
By generating and sending atomic update requests in distributed storage, metadata synchronization without distributed transactions is achieved, which solves the problem of high cost of metadata synchronization in the existing technology, improves system performance and ensures data consistency.
Patent Information
- Application Number
- CN202510093545.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-21
- Publication Date
- 2025-05-16
AI Technical Summary
In distributed storage, the prior art realizes metadata synchronization through distributed transaction mechanisms, resulting in increased communication and persistence costs and reduced system performance.
After the master node completes the metadata update, an atomic update request is generated and sent to the slave node. The slave node updates metadata based on the atomic update request to achieve metadata synchronization without distributed transactions.
It effectively reduces communication and persistence costs, shortens the delay in metadata updates, improves system performance, and ensures data consistency and correctness.
Smart Images
Figure CN120010995A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of data synchronization, and in particular to a method, apparatus, device and readable storage medium for synchronizing metadata within a cluster. Background Art
[0002] In distributed storage, data is usually stored in multiple nodes. Therefore, when data changes (such as adding, deleting or modifying files), related metadata (such as file name, file size, storage location, etc.) also need to be updated and synchronized between different copies (i.e. different nodes) in the cluster. Metadata synchronization ensures that the metadata on all nodes remains consistent, thereby avoiding conflicts and errors during data access.
[0003] In the related art, in distributed storage, for synchronization of metadata in a cluster, a distributed transaction mechanism is usually used to ensure synchronization consistency of metadata, but distributed transactions introduce additional communication and persistence costs, thereby reducing system performance. Summary of the invention
[0004] The present application provides a method, apparatus, device and readable storage medium for synchronizing metadata within a cluster, which can realize metadata synchronization without distributed transactions, so as to reduce communication and persistence costs and shorten the delay of metadata update.
[0005] In a first aspect, an embodiment of the present application provides a method for synchronizing metadata within a cluster, the method comprising:
[0006] When the master node completes the metadata update, control the master node to package the update records in the target data interval corresponding to the metadata update to generate an atomic update request, and send the atomic update request to the slave node;
[0007] The slave node is controlled to perform metadata update based on the atomic update request to achieve metadata synchronization.
[0008] In combination with the first aspect, in one implementation, after the step of controlling the slave node to update metadata based on the atomic update request, the method further includes:
[0009] When it is detected that the first updated metadata in the target data interval on the slave node is inconsistent with the second updated metadata in the target data interval on the master node, the slave node is controlled to update the metadata based on the second updated metadata.
[0010] In combination with the first aspect, in one embodiment, before the step of detecting that the first updated metadata in the target data interval on the slave node is inconsistent with the second updated metadata in the target data interval on the master node, the step further includes:
[0011] Calculating first summary information based on the first updated metadata, and calculating second summary information based on the second updated metadata;
[0012] Determining whether the first summary information is consistent with the second summary information;
[0013] If they are consistent, determining that the first updated metadata is consistent with the second updated metadata;
[0014] If they are inconsistent, it is determined that the first updated metadata and the second updated metadata are inconsistent.
[0015] In combination with the first aspect, in one implementation, after the step of controlling the slave node to update metadata based on the atomic update request, the method further includes:
[0016] When the slave node detects that there is an unexpected mark in the metadata within the first data interval corresponding to the target time interval, control the slave node to obtain the target metadata from the master node based on the target time interval and the first data interval, wherein the unexpected mark is used to indicate that there is an unprocessed atomic update request;
[0017] The slave node is controlled to update metadata based on the target metadata.
[0018] In a second aspect, an embodiment of the present application provides a device for synchronizing metadata within a cluster, the device for synchronizing metadata within a cluster comprising:
[0019] A request generation module, which is used to control the master node to package the update records in the target data interval corresponding to the metadata update when the master node completes the metadata update, so as to generate an atomic update request, and send the atomic update request to the slave node;
[0020] The data synchronization module is used to control the slave node to update metadata based on the atomic update request to achieve metadata synchronization.
[0021] In conjunction with the second aspect, in one implementation, the data synchronization module is further used to:
[0022] When it is detected that the first updated metadata in the target data interval on the slave node is inconsistent with the second updated metadata in the target data interval on the master node, the slave node is controlled to update the metadata based on the second updated metadata.
[0023] In conjunction with the second aspect, in one implementation, the data synchronization module is further used to:
[0024] Calculating first summary information based on the first updated metadata, and calculating second summary information based on the second updated metadata;
[0025] Determining whether the first summary information is consistent with the second summary information;
[0026] If they are consistent, determining that the first updated metadata is consistent with the second updated metadata;
[0027] If they are inconsistent, it is determined that the first updated metadata and the second updated metadata are inconsistent.
[0028] In conjunction with the second aspect, in one implementation, the data synchronization module is further used to:
[0029] When the slave node detects that there is an unexpected mark in the metadata within the first data interval corresponding to the target time interval, control the slave node to obtain the target metadata from the master node based on the target time interval and the first data interval, wherein the unexpected mark is used to indicate that there is an unprocessed atomic update request;
[0030] The slave node is controlled to update metadata based on the target metadata.
[0031] In a third aspect, an embodiment of the present application provides an intra-cluster metadata synchronization device, which includes a processor, a memory, and an intra-cluster metadata synchronization program stored in the memory and executable by the processor, wherein when the intra-cluster metadata synchronization program is executed by the processor, the steps of the intra-cluster metadata synchronization method as described above are implemented.
[0032] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a cluster metadata synchronization program is stored, wherein when the cluster metadata synchronization program is executed by a processor, the steps of the aforementioned cluster metadata synchronization method are implemented.
[0033] The beneficial effects brought by the technical solution provided in the embodiments of the present application include:
[0034] When the master node completes the metadata update, an atomic update request is generated by controlling the master node to package the update records in the target data interval corresponding to the metadata update; since the atomic request operation will not be interrupted by thread scheduling or other factors during execution, the slave node is updated based on the atomic update request to achieve metadata synchronization, which can effectively ensure the consistency and correctness of the data, and the success or failure of the atomic request does not affect the external visibility of the data, so as to ensure the consistency of data visibility. It can be seen that the present application realizes metadata synchronization without distributed transactions through atomic requests, so as to reduce the number of remote procedure calls RPC and persistence required to achieve metadata synchronization based on the distributed transaction mechanism by eliminating transactions, thereby reducing communication and persistence costs, shortening the delay of metadata updates, and improving system performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 This is a flow chart of an embodiment of a method for synchronizing metadata within a cluster of the present application;
[0036] Figure 2 This is a schematic diagram of metadata of each node before aggregation involved in the embodiment of the present application;
[0037] Figure 3 This is a schematic diagram of metadata of each node after aggregation involved in the embodiment of the present application;
[0038] Figure 4 This is a schematic diagram of the hardware structure of the intra-cluster metadata synchronization device involved in the embodiment of the present application. DETAILED DESCRIPTION
[0039] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0040] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.
[0041] In a first aspect, an embodiment of the present application provides a method for synchronizing metadata within a cluster.
[0042] In one embodiment, referring to Figure 1 , Figure 1 This is a flow chart of an embodiment of the method for synchronizing metadata within a cluster of the present application. Figure 1 As shown, the metadata synchronization method within the cluster includes:
[0043] Step S10: When the master node completes the metadata update, the master node is controlled to package the update records in the target data interval corresponding to the metadata update to generate an atomic update request, and the atomic update request is sent to the slave node.
[0044] For example, it should be understood that for the synchronization of metadata in a cluster, a distributed transaction mechanism is usually used to ensure the synchronization consistency of metadata, but the processing flow of distributed transactions is relatively complex, such as transaction initiation, transaction preparation, metadata change, and two-phase commit processes, and transaction requests and responses need to be transmitted between multiple nodes through RPC (Remote Procedure Call), and persistence operations are also required. It can be seen that it will introduce additional communication and persistence costs, thereby reducing system performance.
[0045] It should be understood that for atomic requests, they are either executed in full or not executed at all, that is, once the execution begins, it must be completed completely and will not be interrupted by other operations in the middle. Therefore, in order to eliminate the adverse effects of transactions, this embodiment will implement metadata synchronization through atomic requests to effectively ensure the consistency and correctness of the data.
[0046] Specifically, it can be understood that, during the data aggregation process, the data content will not be changed, but there may be data movement and merging. At this time, the corresponding metadata needs to be updated, and the update is synchronized between the replicas (i.e., nodes) in the cluster; wherein, the metadata mainly records the time point, address range (i.e., address interval or data interval) of the data recorded in the metadata, and the pointer to the specific data. Therefore, for the master node, when it completes the metadata update, it will package the update records (i.e., including modification, merging, and other operation records) in the target data interval (i.e., the same data interval) corresponding to the metadata update into an atomic request, i.e., generate an atomic update request, and send the atomic update request to all slave nodes, so that all slave nodes can achieve metadata synchronization through the atomic update request; it should be noted that the atomic update request can ensure that either all slave nodes are updated successfully or all slave nodes fail to update, so as to ensure the consistency and correctness of the data; at the same time, the success or failure of the atomic update request will not affect the visibility of the data, i.e., regardless of whether the atomic update request is successful or not, the data visible to all nodes is consistent, thereby ensuring the consistency of data visibility.
[0047] The target data interval may be determined according to the data interval involved in the metadata update operation of the master node; for example, see Figure 2As shown, the horizontal axis of the rectangular box where the node is located represents the data interval and the vertical axis represents the time. The master node and the slave node both include metadata 1 and its corresponding data interval is [0,2), metadata 2 and its corresponding data interval is [2,5). Assuming that a new metadata 3 and its corresponding data interval is [1,3), the master node needs to merge metadata 1, metadata 2 and metadata 3 and delete metadata 1 to metadata 3; therefore, see Figure 3 As shown, after the master node merges metadata 1 to metadata 3, it can obtain the updated metadata 4 whose corresponding data interval is [0,5), and deletes metadata 1 to metadata 3; it can be seen that for metadata 1 to metadata 4, the intervals of their update operations are [0,2), [2,5), [1,3) and [0,5) respectively, that is, the data intervals involved in the update operations of the four metadata are all within the range of [0,5), so [0,5) is taken as the target data interval; and this update record includes 4 items: deleting metadata 1, deleting metadata 2, deleting metadata 3 and merging metadata 1 to 3 to generate metadata 4.
[0048] Step S20: Control the slave node to update the metadata based on the atomic update request to achieve metadata synchronization.
[0049] Exemplarily, in this embodiment, all slave nodes will apply the atomic update request they received to achieve metadata synchronization, that is, the slave node will update the metadata stored in itself according to the update record in the received atomic update request to ensure that all slave nodes are either updated successfully or failed to update, thereby ensuring data consistency and accuracy. Specifically, assuming that the update records contained in the atomic update request are to delete metadata 1, delete metadata 2, delete metadata 3, and merge metadata 1 to 3 to generate metadata 4, then each slave node will merge metadata 1 to metadata 3 stored in itself according to the update record of "merge metadata 1 to 3 to generate metadata 4", and obtain the following: Figure 3 The updated metadata 4 is shown, and according to the three update records of “delete metadata 1, delete metadata 2 and delete metadata 3”, metadata 1 to metadata 3 are deleted to achieve metadata synchronization.
[0050] It can be seen that this embodiment realizes metadata synchronization without distributed transactions through atomic requests, so as to reduce the number of remote procedure calls (RPCs) and persistence times required for metadata synchronization based on a distributed transaction mechanism by eliminating transactions. This can not only reduce communication and persistence costs and shorten the latency of metadata updates, but also ensure the correctness of metadata updates, thereby effectively improving system performance.
[0051] Furthermore, in one embodiment, after the step of controlling the slave node to update metadata based on the atomic update request, the step further includes:
[0052] When it is detected that the first updated metadata in the target data interval on the slave node is inconsistent with the second updated metadata in the target data interval on the master node, the slave node is controlled to update the metadata based on the second updated metadata.
[0053] Exemplarily, it should be understood that the update request may be lost or fail, that is, the slave node does not receive the atomic update request or receives the atomic update request but fails to apply the atomic update request. This will cause metadata inconsistencies between different replicas, and then cause the subsequent aggregation process to fail to update the metadata in this area due to conflicts. Therefore, this embodiment will introduce a resynchronization process in the background to avoid problems such as space leakage. Specifically, the resynchronization process can be triggered by a timing method, loss or failure of an update request, etc., such as see Figure 3 As shown, slave node 1 fails when applying the atomic update request, resulting in the metadata on it remaining metadata 1 to metadata 3, and metadata 4 is not generated. At this time, the resynchronization process will be triggered.
[0054] It should be noted that no matter whether the slave node 1 does not receive the atomic update request or the application of the atomic update request fails, from the perspective of the execution action, it is still assumed that the atomic update request has been executed, and the metadata in the target data interval [0,5) on it (that is, metadata 1 to metadata 3) is assumed to be the updated metadata (that is, the first updated metadata); at this time, it is necessary to detect whether the first updated metadata is consistent with the correctly updated metadata (that is, the second updated metadata) in the interval [0,5) on the master node. If they are consistent, it means that the correct metadata synchronization has been completed; if they are inconsistent, it means that the correct metadata synchronization has not been completed, and the slave node 1 is controlled to directly obtain the second updated metadata to update the metadata in the interval [0,5) on it, that is, the slave node 1 deletes the original metadata 1 to 3 in the interval [0,5) on it, and writes the second updated metadata into the interval [0,5), or directly uses the second updated metadata to overwrite metadata 1 to 3 to complete the metadata resynchronization, so that the interval [0,5) on the slave node 1 can continue to accept and process subsequent metadata update requests.
[0055] Furthermore, in one embodiment, before the step of detecting that the first updated metadata in the target data interval on the slave node is inconsistent with the second updated metadata in the target data interval on the master node, the method further includes:
[0056] Calculating first summary information based on the first updated metadata, and calculating second summary information based on the second updated metadata;
[0057] Determining whether the first summary information is consistent with the second summary information;
[0058] If they are consistent, determining that the first updated metadata is consistent with the second updated metadata;
[0059] If they are inconsistent, it is determined that the first updated metadata and the second updated metadata are inconsistent.
[0060] Exemplarily, in this embodiment, when executing the resynchronization process, the consistency of metadata between replicas can be determined by digest comparison and other methods. Specifically, the metadata of the target data interval can be digested by digest algorithms such as CRC (Cyclic Redundancy Check) or MD5 (Message-Digest Algorithm 5), that is, the first digest information corresponding to the interval [0,5) on the slave node 1 and the second digest information corresponding to the interval [0,5) on the master node are calculated respectively, so as to determine the consistency of metadata based on whether the two digests are the same. If the two are the same, the metadata between the nodes is determined to be consistent, otherwise, the metadata between the nodes is determined to be inconsistent. It should be noted that the implementation methods and principles of the CRC algorithm and the MD5 algorithm are common knowledge in the art, and will not be repeated here for the sake of brevity.
[0061] Among them, for the calculation of the first summary information, metadata 1 to metadata 3 can be firstly sorted according to the time information of the first updated metadata (i.e., metadata 1 to metadata 3) in the interval [0,5) on the slave node 1, and then CRC calculation or MD5 calculation is performed according to the time information and address range recorded in metadata 1 to metadata 3 to obtain the first summary information; for the calculation of the second summary information, CRC calculation or MD5 calculation is directly performed according to the time information and address range recorded in the second updated metadata (i.e., metadata 4) in the interval [0,5) on the master node to obtain the second summary information; if the first summary information and the second summary information are different, the slave node 1 is directly updated with metadata 1 to metadata 3 in the interval [0,5) on it through metadata 4 to complete metadata resynchronization.
[0062] Furthermore, in one embodiment, after the step of controlling the slave node to update metadata based on the atomic update request, the step further includes:
[0063] When the slave node detects that there is an unexpected mark in the metadata within the first data interval corresponding to the target time interval, control the slave node to obtain the target metadata from the master node based on the target time interval and the first data interval, wherein the unexpected mark is used to indicate that there is an unprocessed atomic update request;
[0064] The slave node is controlled to update metadata based on the target metadata.
[0065] Exemplarily, it should be understood that in some scenarios (such as the hybrid flash migration scenario), the metadata before and after the aggregation will be marked with corresponding relevant tags. For example, in the hybrid flash migration scenario, the address type of the metadata before and after the migration is different, and the address type is the tag. Therefore, this embodiment can also realize the consistency judgment of the metadata between nodes by tag judgment. Specifically, the interval can be divided according to the time information corresponding to the aggregation round to obtain the corresponding target time interval. For example, if the time range corresponding to the current aggregation round is T1 to T2, [T1, T2] is used as the target time interval; then it is judged whether the metadata of all data intervals (i.e., the first data interval) located in the target time interval on each slave node has unexpected tags. For example, in the hybrid flash migration scenario, the address type of the metadata before the migration is A and the address type of the metadata after the migration is B, then if the address type of the metadata in all the first data intervals is B, it means that there is no unexpected tag, and it is determined that there is no unprocessed atomic update request from the slave node 1 or the synchronization request of the current aggregation round has been successful. In other words, the metadata between the slave node 1 and the master node are consistent.
[0066] Assuming that there is metadata with address type A in the first data interval 1, it means that there is an unexpected mark (that is, the unexpected mark is A), then it is determined that slave node 1 has an unprocessed atomic update request or the synchronization request of the current aggregation round has failed. In other words, the metadata between slave node 1 and the master node are inconsistent; therefore, slave node 1 needs to obtain the target metadata on the first data interval 1 in the [T1, T2] time interval from the master node, and then update the metadata through the target metadata to achieve metadata resynchronization.
[0067] In summary, this application realizes the update of metadata within the cluster without distributed transactions by introducing atomic update requests and metadata resynchronization processes, so that when updating metadata in the data aggregation stage, the consistency and visibility of data to the outside world can be guaranteed without the participation of transactions, and the resynchronization mechanism is used to avoid problems such as space leakage. By eliminating transactions, the number of RPCs and persistence required for metadata synchronization based on the distributed transaction mechanism is reduced, which effectively reduces the communication and persistence costs, and shortens the delay of metadata updates, thereby improving system performance and ensuring the correctness of metadata updates.
[0068] In a second aspect, an embodiment of the present application also provides a metadata synchronization device within a cluster.
[0069] In one embodiment, the intra-cluster metadata synchronization device includes:
[0070] A request generation module, which is used to control the master node to package the update records in the target data interval corresponding to the metadata update when the master node completes the metadata update, so as to generate an atomic update request, and send the atomic update request to the slave node;
[0071] The data synchronization module is used to control the slave node to update metadata based on the atomic update request to achieve metadata synchronization.
[0072] Furthermore, in one embodiment, the data synchronization module is also used for:
[0073] When it is detected that the first updated metadata in the target data interval on the slave node is inconsistent with the second updated metadata in the target data interval on the master node, the slave node is controlled to update the metadata based on the second updated metadata.
[0074] Furthermore, in one embodiment, the data synchronization module is also used for:
[0075] Calculating first summary information based on the first updated metadata, and calculating second summary information based on the second updated metadata;
[0076] Determining whether the first summary information is consistent with the second summary information;
[0077] If they are consistent, determining that the first updated metadata is consistent with the second updated metadata;
[0078] If they are inconsistent, it is determined that the first updated metadata and the second updated metadata are inconsistent.
[0079] Furthermore, in one embodiment, the data synchronization module is also used for:
[0080] When the slave node detects that there is an unexpected mark in the metadata within the first data interval corresponding to the target time interval, control the slave node to obtain the target metadata from the master node based on the target time interval and the first data interval, wherein the unexpected mark is used to indicate that there is an unprocessed atomic update request;
[0081] The slave node is controlled to update metadata based on the target metadata.
[0082] Among them, the functional implementation of each module in the above-mentioned intra-cluster metadata synchronization device corresponds to the various steps in the above-mentioned intra-cluster metadata synchronization method embodiment, and its functions and implementation processes are no longer repeated here.
[0083] In a third aspect, an embodiment of the present application provides a metadata synchronization device within a cluster. The metadata synchronization device within the cluster may be a device with data processing functions, such as a personal computer (PC), a laptop computer, or a server.
[0084] Reference Figure 4 , Figure 4 The schematic diagram of the hardware structure of the intra-cluster metadata synchronization device involved in the embodiment of the present application is shown in FIG. In the embodiment of the present application, the intra-cluster metadata synchronization device may include a processor, a memory, a communication interface, and a communication bus.
[0085] The communication bus may be of any type and is used to interconnect the processor, the memory, and the communication interface.
[0086] The communication interface includes input / output (I / O) interfaces, physical interfaces, and logical interfaces, which are used to interconnect devices within the metadata synchronization device within the cluster, and interfaces used to interconnect the metadata synchronization device within the cluster with other devices (such as other computing devices or user devices). The physical interface can be an Ethernet interface, a fiber interface, an ATM interface, etc.; the user device can be a display, a keyboard, etc.
[0087] The memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.
[0088] The processor may be a general-purpose processor, and the general-purpose processor may call the intra-cluster metadata synchronization program stored in the memory and execute the intra-cluster metadata synchronization method provided in the embodiment of the present application. For example, the general-purpose processor may be a central processing unit (CPU). The method executed when the intra-cluster metadata synchronization program is called may refer to the various embodiments of the intra-cluster metadata synchronization method of the present application, and will not be repeated here.
[0089] Those skilled in the art will understand that Figure 4 The hardware structure shown in the figure does not constitute a limitation on the present application, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.
[0090] In a fourth aspect, an embodiment of the present application also provides a computer-readable storage medium.
[0091] The readable storage medium of the present application stores an intra-cluster metadata synchronization program, wherein when the intra-cluster metadata synchronization program is executed by a processor, the steps of the intra-cluster metadata synchronization method as described above are implemented.
[0092] Among them, the method implemented when the intra-cluster metadata synchronization program is executed can refer to the various embodiments of the intra-cluster metadata synchronization method of the present application, and will not be repeated here.
[0093] It should be noted that the serial numbers of the above-mentioned embodiments of the present application are only for description and do not represent the advantages or disadvantages of the embodiments.
[0094] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. The terms "first", "second" and "third" are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit "first", "second" and "third" to different types.
[0095] In the description of the embodiments of the present application, "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a specific way.
[0096] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; the “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.
[0097] In some processes described in the embodiments of the present application, multiple operations or steps that appear in a specific order are included, but it should be understood that these operations or steps may not be executed in the order in which they appear in the embodiments of the present application or in parallel, and the sequence number of the operation is only used to distinguish the different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed in sequence or in parallel, and these operations or steps may be combined.
[0098] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD) as described above, and includes a number of instructions for a terminal device to execute the methods described in each embodiment of the present application.
[0099] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.
Claims
1. A method for synchronizing metadata within a cluster, characterized in that: The intra-cluster metadata synchronization method comprises: When the master node completes the metadata update, control the master node to package the update records in the target data interval corresponding to the metadata update to generate an atomic update request, and send the atomic update request to the slave node; The slave node is controlled to perform metadata update based on the atomic update request to achieve metadata synchronization.
2. The method for synchronizing metadata within a cluster according to claim 1, characterized in that: After the step of controlling the slave node to update metadata based on the atomic update request, the method further includes: When it is detected that the first updated metadata in the target data interval on the slave node is inconsistent with the second updated metadata in the target data interval on the master node, the slave node is controlled to update the metadata based on the second updated metadata.
3. The method for synchronizing metadata within a cluster as claimed in claim 2, characterized in that: Before the step of detecting that the first updated metadata in the target data interval on the slave node is inconsistent with the second updated metadata in the target data interval on the master node, the method further includes: Calculating first summary information based on the first updated metadata, and calculating second summary information based on the second updated metadata; Determining whether the first summary information is consistent with the second summary information; If they are consistent, determining that the first updated metadata is consistent with the second updated metadata; If they are inconsistent, it is determined that the first updated metadata and the second updated metadata are inconsistent.
4. The method for synchronizing metadata within a cluster according to claim 1, wherein: After the step of controlling the slave node to update metadata based on the atomic update request, the method further includes: When the slave node detects that there is an unexpected mark in the metadata within the first data interval corresponding to the target time interval, control the slave node to obtain the target metadata from the master node based on the target time interval and the first data interval, wherein the unexpected mark is used to indicate that there is an unprocessed atomic update request; The slave node is controlled to update metadata based on the target metadata.
5. A metadata synchronization device within a cluster, characterized in that: The intra-cluster metadata synchronization device comprises: A request generation module, which is used to control the master node to package the update records in the target data interval corresponding to the metadata update when the master node completes the metadata update, so as to generate an atomic update request, and send the atomic update request to the slave node; The data synchronization module is used to control the slave node to update metadata based on the atomic update request to achieve metadata synchronization.
6. The intra-cluster metadata synchronization device according to claim 5, characterized in that: The data synchronization module is also used for: When it is detected that the first updated metadata in the target data interval on the slave node is inconsistent with the second updated metadata in the target data interval on the master node, the slave node is controlled to update the metadata based on the second updated metadata.
7. The intra-cluster metadata synchronization device according to claim 6, characterized in that: The data synchronization module is also used for: Calculating first summary information based on the first updated metadata, and calculating second summary information based on the second updated metadata; Determining whether the first summary information is consistent with the second summary information; If they are consistent, determining that the first updated metadata is consistent with the second updated metadata; If they are inconsistent, it is determined that the first updated metadata and the second updated metadata are inconsistent.
8. The intra-cluster metadata synchronization device according to claim 5, characterized in that: The data synchronization module is also used for: When the slave node detects that there is an unexpected mark in the metadata within the first data interval corresponding to the target time interval, control the slave node to obtain the target metadata from the master node based on the target time interval and the first data interval, wherein the unexpected mark is used to indicate that there is an unprocessed atomic update request; The slave node is controlled to update metadata based on the target metadata.
9. A metadata synchronization device within a cluster, characterized in that: The intra-cluster metadata synchronization device includes a processor, a memory, and an intra-cluster metadata synchronization program stored in the memory and executable by the processor, wherein when the intra-cluster metadata synchronization program is executed by the processor, the steps of the intra-cluster metadata synchronization method as described in any one of claims 1 to 4 are implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an intra-cluster metadata synchronization program, wherein when the intra-cluster metadata synchronization program is executed by a processor, the steps of the intra-cluster metadata synchronization method according to any one of claims 1 to 4 are implemented.