Metadata consistency management method and device
By introducing state labels to metadata, the problems of latency and network congestion in metadata synchronization in distributed systems are solved, achieving low-latency and precise control of metadata consistency management, and ensuring data consistency among client nodes.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- TSINGHUA UNIVERSITY
- Filing Date
- 2026-01-23
- Publication Date
- 2026-05-12
AI Technical Summary
In distributed systems, existing metadata synchronization methods suffer from update delays and network congestion, failing to balance low latency and precise control, leading to data inconsistency and errors.
Different status labels are introduced to indicate whether metadata is being updated. These status labels actively drive client nodes and management nodes to collaboratively update and synchronize metadata, ensuring metadata consistency among client nodes.
It achieves low-latency and precise control over metadata consistency management, avoiding data errors and network congestion, and ensuring metadata consistency among client nodes.
Smart Images
Figure CN122019666A_ABST
Abstract
Description
Technical Field
[0001] The embodiments in this specification relate to the field of computer technology, and in particular to a method and apparatus for metadata consistency management. Background Technology
[0002] In distributed systems, metadata plays an important management role. For example, metadata can control information such as table structure, data structure, tag definition, permissions, and indexes.
[0003] To improve access performance, distributed systems typically cache metadata on the client-side (meta-client) side. However, metadata is characterized by relatively infrequent updates but frequent reads, and multiple clients may simultaneously access and modify the same metadata. Therefore, it is necessary to promptly synchronize modified metadata from one client to other clients to ensure consistency across all clients. Otherwise, data errors can occur due to inconsistencies in the same metadata.
[0004] In related technologies, two synchronization methods are provided: passive synchronization and broadcast synchronization. Passive synchronization refers to the client periodically pulling the latest metadata through heartbeats to update its local metadata. This method suffers from update latency (local metadata may lag during the heartbeat cycle, causing the client to read outdated data). Broadcast synchronization involves broadcasting changes to the local metadata of any client to all clients across the network. This method cannot precisely control the scope, potentially causing network congestion.
[0005] Therefore, there is an urgent need for a metadata consistency management solution that balances low latency and precise control. Summary of the Invention
[0006] This specification provides a method and apparatus for metadata consistency management.
[0007] According to a first aspect of the embodiments of this specification, a metadata consistency management method is provided, applied to a distributed system, the distributed system including a management node providing metadata services and client nodes using the metadata services; the client nodes locally cache metadata synchronized from the management node, and the cached metadata items include a status label indicating the cache status; the method includes: The target client node initiates a write operation on the target metadata cached locally, modifies the status label of the target metadata cached locally to the first state, and initiates a remote call request to the management node for consistency management of the target metadata; wherein, the first state indicates that the metadata is being updated and is unreadable; In response to the remote call request, the management node queries the current status of the target metadata recorded locally; if the current status of the target metadata is the second status, it obtains the latest data of the target metadata from the target client node and notifies other client nodes to modify the status label of the target metadata cached locally to the first status; wherein, the second status indicates that the metadata is readable; In response to notifications from other client nodes, if all notifications indicate that the status label of the locally cached target metadata has been modified to the first status, the management node modifies the current status of the locally recorded target metadata from the second status to the first status and notifies the other client nodes to update the locally cached target metadata to the latest data.
[0008] Optionally, after the management node queries the current status of the target metadata recorded locally in response to the remote call request, the method further includes: If the current state of the target metadata is the first state, then the latest data of the target metadata is obtained from the target client node.
[0009] Optionally, the method further includes: After updating the target metadata cached locally to the latest data, the other client nodes modify the status label of the target metadata cached locally from the first status to the second status. In response to notification responses returned by the other client nodes, if all notification responses update the locally cached target metadata to the latest data, the management node modifies the current state of the locally recorded target metadata from the first state to the second state, and notifies the target client node to modify the status label of the locally cached target metadata from the first state to the second state.
[0010] Optionally, the metadata server of the distributed system maintains a list of client nodes corresponding to each piece of metadata; The step of notifying other client nodes to modify the status label of the locally cached target metadata to the first status includes: The management node queries the list of client nodes corresponding to the target metadata and notifies all client nodes in the list other than the target client node to modify the status label of the locally cached target metadata to the first status.
[0011] Optionally, after the management node responds to the notification responses returned by the other client nodes, if all the notification responses indicate that the status label of the locally cached target metadata has been modified to the first status, the method further includes: Persist the latest data of the target metadata obtained from the target client node.
[0012] Optionally, the method further includes: The target client node initiates a read operation on the target metadata and queries the status tag of the target metadata cached locally; If the status label is in the first state, the read operation is blocked until the status label is in the second state, at which point the target metadata is read.
[0013] Optionally, the method further includes: If the status label is in the third state, a caching request for the target metadata is initiated to the management node; wherein, the third state indicates that the metadata is not cached locally; In response to the cache request, the management node queries the current status of the target metadata recorded locally. If the current status of the target metadata is a first status, the cache request is blocked until the current status is a second status, at which point the management node maintains the latest target metadata locally and sends it to the target client node. The target client node receives the latest target metadata issued by the management node, caches the latest target metadata locally, and changes the status label of the target metadata from the third status to the second status.
[0014] Optionally, the management node uses a hash table structure to maintain the current state of all metadata and a list of client nodes corresponding to the metadata.
[0015] According to a second aspect of the embodiments of this specification, a metadata consistency management apparatus is provided, applied to a distributed system, the distributed system including a management node providing metadata services and client nodes using the metadata services; the client nodes locally cache metadata synchronized from the management node, and the cached metadata items include a status label indicating the cache status; the apparatus includes: The write operation unit involves the target client node initiating a write operation on the target metadata cached locally, modifying the status label of the target metadata cached locally to a first state, and initiating a remote call request to the management node for consistency management of the target metadata; wherein, the first state indicates that the metadata is being updated and is unreadable; The notification unit, in response to the remote call request, queries the current status of the target metadata recorded locally; if the current status of the target metadata is the second status, it obtains the latest data of the target metadata from the target client node and notifies other client nodes to modify the status label of the target metadata cached locally to the first status; wherein, the second status indicates that the metadata is readable; In the synchronization unit, the management node responds to the notification responses returned by the other client nodes. If the notification responses all indicate that the status label of the locally cached target metadata has been modified to the first status, then the current status of the locally recorded target metadata is modified from the second status to the first status, and the other client nodes are notified to update the locally cached target metadata to the latest data.
[0016] According to a third aspect of the embodiments of this specification, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to perform the metadata consistency management method described in any of the preceding embodiments.
[0017] This specification provides an embodiment of a metadata consistency management scheme. By introducing different status labels to metadata to indicate whether the metadata is being updated, the scheme can proactively drive client nodes and management nodes to perform metadata consistency collaborative updates and synchronization when the metadata is in the first state, thereby ensuring the consistency of metadata among client nodes. Attached Figure Description
[0018] Figure 1 This is an architecture diagram of a distributed system provided in one embodiment of this specification; Figure 2 This is a flowchart of a metadata consistency management method provided in one embodiment of this specification; Figure 3 This is a schematic diagram of a metadata writing process provided in one embodiment of this specification; Figure 4 This is a schematic diagram of the structure of the hash table maintained by the management node according to one embodiment of this specification; Figure 5 This is a schematic diagram of a metadata reading process provided in one embodiment of this specification; Figure 6 This is a hardware structure diagram of a metadata consistency management device provided in one embodiment of this specification; Figure 7 This is a module of the metadata consistency management device provided in one embodiment of this specification. Detailed Implementation
[0019] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this specification as detailed in the appended claims.
[0020] The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of this specification. The singular forms “a,” “the,” and “the” as used in this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.
[0021] It should be understood that although the terms first, second, third, etc., may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this specification, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."
[0022] As mentioned earlier, metadata is characterized by relatively infrequent updates but frequent reads, and multiple clients may simultaneously access and modify the same metadata. Therefore, it is necessary to promptly synchronize modified metadata from one client to other clients to ensure consistency across all clients. Otherwise, data errors may occur due to inconsistencies in the same metadata.
[0023] In related technologies, two synchronization methods are provided: passive synchronization and broadcast synchronization. Passive synchronization refers to the client periodically pulling the latest metadata through heartbeats to update its local metadata. This method suffers from update latency (local metadata may lag during the heartbeat cycle, causing the client to read outdated data). Broadcast synchronization involves broadcasting changes to the local metadata of any client to all clients across the network. This method cannot precisely control the scope, potentially causing network congestion.
[0024] Therefore, there is an urgent need for a metadata consistency management solution that balances low latency and precise control.
[0025] In view of this, the present invention proposes a metadata consistency management scheme that balances low latency and precise control. By introducing different status labels to metadata to indicate whether metadata is being updated, the status labels of metadata, such as when metadata is in the first state, can be used to actively drive client nodes and management nodes to perform metadata consistency collaborative updates and synchronization, thereby ensuring the consistency of metadata among client nodes.
[0026] Please refer to the following: Figure 1 The system architecture diagram of the distributed system shown is as follows: the distributed system includes a management node 1 that provides metadata services and a client node 2 that uses the metadata services.
[0027] In a distributed system, all nodes can connect to each other to execute relevant business logic (such as...). Figure 1 (As shown by the dashed line segment). Regarding metadata consistency management, the metadata consistency management method provided in this specification can be used between management node 1 and client node 2 to achieve timely synchronization and updating of metadata (e.g., ...). Figure 1 (The solid line segment shown in the image).
[0028] The difference from traditional distributed systems is that in the distributed system described in this specification, management node 1 can introduce a status label for each metadata. Based on this, when each client node 2 caches metadata from management node 1, it will also cache the status label, thereby adding a status label indicating the cached status to the cached metadata entries recorded locally by client node 2.
[0029] It is understood that the status labels are used to represent different states of the metadata. For example, when the metadata is being updated, the status label of the metadata can be set to the first state, so the first state can indicate that the metadata is being updated and is unreadable; For example, when the metadata is not being updated, the status label of the metadata can be set to the second state, so the second state can indicate that the metadata is readable; For example, when metadata is not cached locally on the client node, the status label of the metadata can be set to the third state, so the third state can indicate that the metadata is not cached locally.
[0030] Different status labels are used to indicate whether metadata is being updated. This allows the metadata status labels, such as when the metadata is in the first state, to be used to actively drive client nodes and management nodes to perform consistent metadata updates and synchronization, thereby ensuring the consistency of metadata among client nodes.
[0031] Please refer to the following. Figure 2 The diagram shown illustrates a metadata consistency management method, which may include the following steps: Step 210: The target client node initiates a write operation on the target metadata cached locally, modifies the status label of the target metadata cached locally to the first status, and initiates a remote call request to the management node for consistency management of the target metadata; wherein, the first status indicates that the metadata is being updated and is unreadable.
[0032] Step 220: In response to the remote call request, the management node queries the current status of the target metadata recorded locally; if the current status of the target metadata is the second status, it obtains the latest data of the target metadata from the target client node and notifies other client nodes to modify the status label of the target metadata cached locally to the first status; wherein, the second status indicates that the metadata is readable.
[0033] Step 230: In response to the notification responses returned by the other client nodes, if the notification responses all indicate that the status label of the locally cached target metadata has been modified to the first status, then the current status of the locally recorded target metadata is modified from the second status to the first status, and the other client nodes are notified to update the locally cached target metadata to the latest data.
[0034] This embodiment can be applied to the aforementioned Figure 1 The distributed system shown includes a management node that provides metadata services and client nodes that use the metadata services; the client nodes locally cache metadata synchronized from the management node, and the cached metadata entries include status labels indicating the cache status.
[0035] As shown above, the first state can indicate that the metadata is being updated and is not readable; the second state can indicate that the metadata is readable; and the third state can indicate that the metadata is not cached locally.
[0036] For example, in the following embodiments, the dirty field will be used as the first state, the shared field as the second state, and the unshared field as the third state for ease of description; of course, other fields can be used as different state labels in actual applications, and this specification does not limit this.
[0037] The target client node can be any client node in the distributed system, and the target metadata can be any metadata cached on the target client node itself.
[0038] The target client node performs data operations on the locally cached target metadata according to actual needs, and then triggers relevant operation processes based on the type of data operation.
[0039] For example, if it is necessary to update the target metadata, a write operation can be initiated targeting the target metadata. The following section will explain... Figure 3 The following diagram illustrates the metadata writing process: Step 301: The target client node initiates a write operation on the target metadata cached locally; Step 302: The target client node modifies the status label of the locally cached target metadata to the first status, dirty; Step 303: The target client node initiates an RPC (Remote Procedure Call) request for the target metadata to the management node.
[0040] Step 304: In response to the remote call request, the management node queries the current status of the target metadata recorded locally. If the current status is the second status, shared, proceed to step 305.
[0041] Step 305: The management node obtains the latest target metadata (target metadata after the write operation is performed) from the target client node and notifies other client nodes to change the status label of the locally cached target metadata to the first status, dirty.
[0042] Step 307: The management node waits for other client nodes to return notification responses.
[0043] If all notification responses indicate that the status label of the locally cached target metadata has been modified to the first status "dirty", then step 309 is executed. Furthermore, if step 304 finds that the current status is the second status "shared", then the current status of the target metadata recorded locally by the management node also needs to be modified from the second status "shared" to the first status "dirty" to prevent the management node from using the old target metadata (e.g., blocking the provision of old target metadata to newly joined client nodes in the distributed system until the current status of the target metadata is modified back to the second status "shared").
[0044] Step 309: The management node will obtain the latest target metadata from the target client node and notify other client nodes so that the other client nodes can update their locally cached target metadata (old data) with the latest data.
[0045] The above embodiments utilize status labels to control metadata synchronization during write operations, ensuring that all client nodes maintain metadata consistency. Furthermore, PRC requests are only triggered when necessary (i.e., when metadata is in its first state), avoiding frequent broadcasts that could cause network congestion.
[0046] In an exemplary embodiment, after the management node in step 220 above queries the current status of the target metadata recorded locally in response to the remote call request, the process may further include: If the current state of the target metadata is the first state, then the latest data of the target metadata is obtained from the target client node.
[0047] like Figure 3 As shown, if the current status queried in step 304 is the first status "dirty", then execute step 306: obtain the latest data of the target metadata from the target client node.
[0048] In practical applications, the current state of the target metadata recorded locally by the management node is not necessarily the second state (shared); it could also be the first state (dirty). This occurs when the management node is still in the process of updating the target metadata. Therefore, to obtain the latest data, the management node needs to directly retrieve the latest target metadata from the target client node being updated. Furthermore, since the management node is still in the process of updating the target metadata, it means that other client nodes are also in the process of updating the target metadata, meaning that the state label of the target metadata cached locally by other client nodes has been modified to the first state (dirty). Therefore, the difference between step 306 and step 305 is that the management node does not need to notify other client nodes to modify the locally cached state label to the first state (dirty).
[0049] In an exemplary embodiment, after the management node in step 230 responds to the notification responses returned by the other client nodes, if all the notification responses indicate that the status label of the locally cached target metadata has been modified to the first status, the process may further include: Persist the latest data of the target metadata obtained from the target client node.
[0050] like Figure 3 As shown, step 308 can also be performed after step 307 and before step 309: the management node persists the latest data of the target metadata.
[0051] By persisting the latest target metadata, management nodes can concurrently handle the consistent management of different or the same target metadata from multiple client nodes.
[0052] In one exemplary embodiment, step 230 may further include: After updating the target metadata cached locally to the latest data, the other client nodes modify the status label of the target metadata cached locally from the first status to the second status. In response to notification responses returned by the other client nodes, if all notification responses update the locally cached target metadata to the latest data, the management node modifies the current state of the locally recorded target metadata from the first state to the second state, and notifies the target client node to modify the status label of the locally cached target metadata from the first state to the second state.
[0053] In practical applications, after using status tags to synchronize target metadata between client nodes, the status tags maintained locally by the relevant nodes are modified to the first state. Since the target metadata cannot be read in the first state, it is necessary to restore the status tags maintained locally by the relevant nodes to the second state so that the updated target metadata can be read and used. The relevant nodes here include the target client node, the management node, and other client nodes.
[0054] In one exemplary embodiment, the metadata server of the distributed system maintains a list of client nodes corresponding to each piece of metadata; The step of notifying other client nodes to modify the status label of the locally cached target metadata to the first status includes: The management node queries the list of client nodes corresponding to the target metadata and notifies all client nodes in the list other than the target client node to modify the status label of the locally cached target metadata to the first status.
[0055] To address the issue that existing broadcast synchronization methods cannot accurately control the scope, this manual maintains a corresponding list of client nodes for each piece of metadata. This allows for the precise location of each client node that needs to update metadata, thereby reducing unnecessary communication and lowering communication costs.
[0056] In one exemplary embodiment, the management node uses a hash table structure to maintain the current state of all metadata and a list of client nodes corresponding to the metadata.
[0057] Please refer to the following. Figure 4 The diagram shows the structure of the hash table maintained by the management node. This hash table can be a Map data structure, specifically recording the current state (state label) of all metadata using a key-value mapping relationship, as well as the list of client nodes. <clientid>).like Figure 4 As shown, the key can be a unique value calculated based on the current state of the metadata and the list of client nodes. The value mapped to each key is the current state of the metadata and the list of client nodes. Through this hash table structure, the management node can determine the cache state of each client node in real time, facilitating accurate state coordination and notification during updates and avoiding the performance overhead of broadcast synchronization.
[0058] In one exemplary embodiment, the method further includes: The target client node initiates a read operation on the target metadata and queries the status tag of the target metadata cached locally; If the status label is in the first state, the read operation is blocked until the status label is in the second state, at which point the target metadata is read. As previously mentioned, the target client node performs data operations on the locally cached target metadata according to actual needs, and then triggers relevant operation processes based on the type of data operation. Besides the aforementioned write operations, read operations are also possible. For example, when it is necessary to read the target metadata, the target client node can initiate a read operation on the target metadata. The following section will discuss this in conjunction with... Figure 5 The following diagram illustrates the metadata reading process: Step 501: The target client node initiates a read operation on the target metadata; Step 502: The target client node queries the status label of the target metadata cached locally. If the status label is the second status "shared", proceed to step 503; if the status label is the first status "dirty", proceed to step 504.
[0059] Step 503: Read the cached target metadata from the local cache. This prioritizes local processing of read operations, reducing the pressure on the management node.
[0060] Step 504: Block the read operation until the status label is in the second state, then read the target metadata from the local cache.
[0061] Since the first state "dirty" indicates that the target metadata is being updated, we need to wait for the target metadata to be updated before the state label changes to the second state.
[0062] By blocking read operations, old target metadata can be avoided from being read from the local cache.
[0063] In an exemplary embodiment, after the target client node initiates a read operation on the target metadata and queries the status tag of the target metadata cached locally, the process may further include: If the status label is in the third state, a caching request for the target metadata is initiated to the management node; wherein, the third state indicates that the metadata is not cached locally; In response to the cache request, the management node queries the current status of the target metadata recorded locally. If the current status of the target metadata is a first status, the cache request is blocked until the current status is a second status, at which point the management node maintains the latest target metadata locally and sends it to the target client node. The target client node receives the latest target metadata issued by the management node, caches the latest target metadata locally, and changes the status label of the target metadata from the third status to the second status.
[0064] Continue to refer to Figure 5 In step 502, when querying the status label of the target metadata cached locally, it may be either the second status (shared) or the first status (dirty) mentioned above, or it may be the third status (unshared). The third status indicates that the target client node does not cache the target metadata locally, in which case step 505 needs to be executed: the target client node initiates a cache request to the management node.
[0065] Step 506: The management node queries the current status of the target metadata of the local record; if the current status is the first state dirty, then proceed to step 507; if the current status is the first state dirty, then proceed to step 508.
[0066] Step 507: The management node blocks cache requests and waits for the current state to change to the second state.
[0067] When the current state is the first state "dirty", it means that the target metadata is in the update phase. At this time, the target metadata cannot be read, so it is necessary to wait for the target metadata to complete the update.
[0068] Step 508: The management node adds the target client node to the client node list corresponding to the target metadata and returns the latest data of the target metadata to the target client node.
[0069] Step 509: The target metadata caches the latest target metadata locally and changes the target metadata's status label from the third state (unshared) to the second state (shared). At this point, since the target client node's local cache status label has changed to the second state (shared), a read operation can execute step 503 to read the latest target metadata from the local cache.
[0070] In summary, this specification provides an implementation of metadata consistency management that balances low latency and precise control. By introducing different status labels to metadata to indicate whether it is being updated, the status labels (e.g., when metadata is in the first state) can be used to proactively drive client nodes and management nodes to perform collaborative updates and synchronization for metadata consistency, thereby ensuring metadata consistency among client nodes.
[0071] Corresponding to the aforementioned embodiments of the metadata consistency management method, this specification also provides embodiments of a metadata consistency management device. These device embodiments can be implemented in software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of its host device reading the corresponding computer program from non-volatile memory into memory for execution. From a hardware perspective, such as... Figure 6 The diagram shown is a hardware structure diagram of the device containing the metadata consistency management device described in this specification. (Except for...) Figure 6 In addition to the processor, network interface, memory, and non-volatile memory shown, the device in the embodiment may also include other hardware depending on the actual communication function, which will not be described in detail here.
[0072] Please see Figure 7 This is a block diagram of a metadata consistency management device provided in an embodiment of this specification, applied to a distributed system. The distributed system includes a management node that provides metadata services and client nodes that use the metadata services. The client nodes locally cache metadata synchronized from the management node, and the cached metadata entries include status tags indicating the cache status. The device includes: The write operation unit 710 initiates a write operation on the target metadata cached locally, modifies the status label of the target metadata cached locally to a first state, and sends a remote call request to the management node for consistency management of the target metadata; wherein, the first state indicates that the metadata is being updated and is unreadable; Notification unit 720: In response to the remote call request, the management node queries the current status of the target metadata recorded locally; if the current status of the target metadata is the second status, it obtains the latest data of the target metadata from the target client node and notifies other client nodes to modify the status label of the locally cached target metadata to the first status; wherein, the second status indicates that the metadata is readable; Synchronization unit 730: In response to notification responses from other client nodes, if all notification responses indicate that the status tag of the locally cached target metadata has been modified to the first status, then the management node modifies the current status of the locally recorded target metadata from the second status to the first status, and notifies the other client nodes to update the locally cached target metadata to the latest data. Optionally, after querying the current status of the target metadata in the local record, the notification unit 720 is further configured to obtain the latest data of the target metadata from the target client node if the current status of the target metadata is a first status.
[0073] Optionally, the device further includes: The status modification unit, after the other client nodes update the locally cached target metadata to the latest data, modifies the status label of the locally cached target metadata from the first state to the second state; the management node responds to the notification response returned by the other client nodes, and if the notification response is to update the locally cached target metadata to the latest data, then modifies the current state of the locally recorded target metadata from the first state to the second state, and notifies the target client node to modify the status label of the locally cached target metadata from the first state to the second state.
[0074] Optionally, the metadata server of the distributed system maintains a list of client nodes corresponding to each piece of metadata; The notification unit 720 notifies other client nodes to modify the status label of the locally cached target metadata to the first status. It is further used by the management node to query the client node list corresponding to the target metadata and notify all client nodes in the client node list, except for the target client node, to modify the status label of the locally cached target metadata to the first status.
[0075] Optionally, after the notification response indicates that the status label of the locally cached target metadata has been modified to the first status, the synchronization unit 730 is further configured to persist the latest data of the target metadata obtained from the target client node.
[0076] Optionally, the device further includes: The read operation unit initiates a read operation on the target metadata by querying the status tag of the target metadata in the local cache. If the status tag is in the first state, the read operation is blocked until the status tag is in the second state, at which point the target metadata is read.
[0077] Optionally, the read operation unit is further configured to initiate a cache request for the target metadata to the management node if the status label is in the third state; wherein the third state indicates that the metadata is not cached this time; Accordingly, the device further includes: In the distribution unit, the management node responds to the cache request by querying the current status of the target metadata recorded locally. If the current status of the target metadata is a first status, the cache request is blocked until the current status is a second status, at which point the latest target metadata is maintained locally and distributed to the target client node. The caching unit allows the target client node to receive the latest target metadata from the management node, cache the latest target metadata locally, and change the status label of the target metadata from the third status to the second status.
[0078] Optionally, the management node uses a hash table structure to maintain the current state of all metadata and a list of client nodes corresponding to the metadata.
[0079] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.
[0080] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.
[0081] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this specification according to actual needs. Those skilled in the art can understand and implement this without creative effort.
[0082] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.
[0083] Other embodiments of this specification will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This specification is intended to cover any variations, uses, or adaptations that follow the general principles of this specification and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this specification are indicated by the following claims.
[0084] It should be understood that this specification is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this specification is limited only by the appended claims.< / clientid>
Claims
1. A metadata consistency management method, characterized in that, This is applied to a distributed system, which includes a management node that provides metadata services and client nodes that use the metadata services. The client node locally caches metadata synchronized from the management node, and the cached items of the metadata include status tags indicating the cache status; the method includes: The target client node initiates a write operation on the target metadata cached locally, modifies the status label of the target metadata cached locally to the first state, and initiates a remote call request to the management node for consistency management of the target metadata; wherein, the first state indicates that the metadata is being updated and is unreadable; In response to the remote call request, the management node queries the current status of the target metadata recorded locally; if the current status of the target metadata is the second status, it obtains the latest data of the target metadata from the target client node and notifies other client nodes to modify the status label of the target metadata cached locally to the first status; wherein, the second status indicates that the metadata is readable; In response to notifications from other client nodes, if all notifications indicate that the status label of the locally cached target metadata has been modified to the first status, the management node modifies the current status of the locally recorded target metadata from the second status to the first status and notifies the other client nodes to update the locally cached target metadata to the latest data.
2. The method according to claim 1, characterized in that, After the management node responds to the remote call request and queries the current status of the target metadata recorded locally, the process further includes: If the current state of the target metadata is the first state, then the latest data of the target metadata is obtained from the target client node.
3. The method according to claim 1, characterized in that, The method further includes: After updating the target metadata cached locally to the latest data, the other client nodes modify the status label of the target metadata cached locally from the first status to the second status. In response to notification responses returned by the other client nodes, if all notification responses update the locally cached target metadata to the latest data, the management node modifies the current state of the locally recorded target metadata from the first state to the second state, and notifies the target client node to modify the status label of the locally cached target metadata from the first state to the second state.
4. The method according to claim 1, characterized in that, The metadata server of the distributed system maintains a list of client nodes corresponding to each piece of metadata. The step of notifying other client nodes to modify the status label of the locally cached target metadata to the first status includes: The management node queries the list of client nodes corresponding to the target metadata and notifies all client nodes in the list other than the target client node to modify the status label of the locally cached target metadata to the first status.
5. The method according to claim 1, characterized in that, After the management node responds to the notification responses returned by the other client nodes, if all the notification responses indicate that the status label of the locally cached target metadata has been modified to the first status, the method further includes: Persist the latest data of the target metadata obtained from the target client node.
6. The method according to claim 1, characterized in that, The method further includes: The target client node initiates a read operation on the target metadata and queries the status tag of the target metadata cached locally; If the status label is in the first state, the read operation is blocked until the status label is in the second state, at which point the target metadata is read.
7. The method according to claim 6, characterized in that, The method further includes: If the status label is in the third state, a caching request for the target metadata is initiated to the management node; wherein, the third state indicates that the metadata is not cached locally; In response to the cache request, the management node queries the current status of the target metadata recorded locally. If the current status of the target metadata is a first status, the cache request is blocked until the current status is a second status, at which point the management node maintains the latest target metadata locally and sends it to the target client node. The target client node receives the latest target metadata issued by the management node, caches the latest target metadata locally, and changes the status label of the target metadata from the third status to the second status.
8. The method according to claim 4, characterized in that, The management node uses a hash table structure to maintain the current state of all metadata and a list of client nodes corresponding to the metadata.
9. A metadata consistency management device, characterized in that, This is applied to a distributed system, which includes a management node that provides metadata services and client nodes that use the metadata services. The client node locally caches metadata synchronized from the management node, and the cached entries for the metadata include status tags indicating the cache status; the apparatus includes: The write operation unit involves the target client node initiating a write operation on the target metadata cached locally, modifying the status label of the target metadata cached locally to a first state, and initiating a remote call request to the management node for consistency management of the target metadata; wherein, the first state indicates that the metadata is being updated and is unreadable; The notification unit, in response to the remote call request, queries the current status of the target metadata recorded locally; if the current status of the target metadata is the second status, it obtains the latest data of the target metadata from the target client node and notifies other client nodes to modify the status label of the target metadata cached locally to the first status; wherein, the second status indicates that the metadata is readable; In the synchronization unit, the management node responds to the notification responses returned by the other client nodes. If the notification responses all indicate that the status label of the locally cached target metadata has been modified to the first status, then the current status of the locally recorded target metadata is modified from the second status to the first status, and the other client nodes are notified to update the locally cached target metadata to the latest data.
10. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to perform the method according to any one of claims 1-8.