Data request response method and device, electronic equipment and storage medium
By comparing resource identifiers, cluster mapping version numbers, lock versions, and monotonic clock values in a distributed storage system, expired requests are identified and intercepted, solving the problem of old instance requests overwriting new data in a distributed storage system and improving the system's business continuity and data consistency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-30
- Publication Date
- 2026-04-03
AI Technical Summary
In distributed storage systems, data inconsistency arises from in-transit data requests from older instances overwriting new data, while existing technologies suffer from switching blocking or low efficiency in blacklist synchronization.
By receiving the resource identifier, client cluster mapping version number, lock version, and monotonic clock value from the data request, the storage service node compares the valid version range, the maximum lock version, and the monotonic clock value to determine whether the data request has expired and intercepts expired requests locally.
It enables the rapid identification and interception of expired requests without centralized control, preventing old requests from overwriting new data and improving business continuity and data consistency.
Smart Images
Figure CN121786048A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and more specifically, to a data request and response method, apparatus, electronic device, and computer-readable storage medium. Background Technology
[0002] In distributed storage systems, a master-slave instance mechanism is typically used to ensure data consistency. This means that only one active instance of the same resource is allowed to provide services at any given time. When a node fails or a network partition occurs, the service needs to be switched from the old instance to the new instance. During this process, if the old instance has pending data requests while the new instance has already started processing new write operations, subsequent data requests arriving on the old instance may overwrite the new data, causing data inconsistency or even corruption.
[0003] In related technologies, one approach is to wait for all pending data requests from the old instance to complete before starting the new instance. However, this method can cause switching blockage when data requests cannot return due to network packet loss or other reasons, affecting business continuity. Another approach is to maintain a blacklist on the server side to reject requests from the old instance. However, this method relies on a centralized control node to synchronize the blacklist to all storage nodes, which is difficult to implement efficiently in a purely distributed environment and has poor timeliness.
[0004] Therefore, how to effectively identify and intercept expired data requests to prevent old requests from overwriting new data is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention
[0005] The purpose of this application is to provide a data request response method, apparatus, electronic device, and computer-readable storage medium, which effectively identify and intercept expired data requests to prevent old requests from overwriting new data.
[0006] To achieve the above objectives, this application provides a data request and response method, applied to a storage service node in a distributed storage system, the method comprising:
[0007] Receive data requests sent by clients; wherein, the data requests carry a resource identifier, a client cluster mapping version number, a lock version, and a monotonic clock value, the resource identifier is used to identify the storage instance to which the requested data belongs, the client cluster mapping version number is used to describe the cluster topology version when the client constructs the request, the lock version is used to describe the distributed lock version currently held by the client, and the monotonic clock value is used to describe the monotonic clock reading of the client at the time of instance startup;
[0008] Based on the resource identifier, query the corresponding valid version range, the maximum locked version, and the monotonic clock value corresponding to the maximum locked version in the locally maintained version record;
[0009] Whether the data request is an expired request is determined based on one of the first comparison result, the second comparison result, and the third comparison result; wherein, the first comparison result is the comparison result between the client cluster mapping version number and the valid version range, the second comparison result includes: the comparison result between the lock version in the data request and the maximum lock version, and the first comparison result, and the third comparison result includes: the comparison result between the monotonic clock value in the data request and the monotonic clock value corresponding to the maximum lock version, and the second comparison result;
[0010] If the data request is expired, the data request is intercepted; otherwise, the data request is responded to.
[0011] The step of determining whether the data request is an expired request based on one of the first comparison result, the second comparison result, and the third comparison result includes:
[0012] Compare the client cluster mapping version number with the valid version range;
[0013] If the client cluster mapping version number is within the valid version range, then compare the lock version in the data request with the maximum lock version;
[0014] If the lock version in the data request is less than the maximum lock version, the data request is determined to be an expired request.
[0015] If the lock version in the data request is greater than the maximum lock version, then the data request is determined to be a non-expired request;
[0016] If the lock version in the data request is equal to the maximum lock version, then compare the monotonic clock value in the data request with the monotonic clock value corresponding to the maximum lock version.
[0017] If the monotonic clock value in the data request is less than the monotonic clock value corresponding to the maximum lock version, then the data request is determined to be an expired request.
[0018] If the monotonic clock value in the data request is greater than or equal to the monotonic clock value corresponding to the maximum lock version, then the data request is determined to be a non-expired request.
[0019] If the client cluster mapping version number is outside the valid version range, the data request is determined to be an expired request.
[0020] Wherein, the lower limit of the effective version range is the minimum cluster mapping version number when the storage service node becomes the primary replica of the placement group, and the upper limit of the effective version range is the current cluster mapping version number.
[0021] Accordingly, comparing the client cluster mapping version number with the valid version range includes:
[0022] Determine whether the client cluster mapping version number is greater than or equal to the minimum cluster mapping version number and less than or equal to the current cluster mapping version number;
[0023] If so, then the client cluster mapping version number is determined to be within the valid version range;
[0024] If not, then the client cluster mapping version number is determined to be outside the valid version range.
[0025] The method further includes:
[0026] When the storage service node restarts or the primary replica of the reset group is switched to become the new primary replica, the lower limit of the effective version range in the version record is updated to the cluster mapping version number after the restart or switch, and the maximum lock version corresponding to the resource identifier and the monotonic clock value corresponding to the maximum lock version in the version record are reset to the initial value.
[0027] The step of querying the corresponding valid version range, the maximum locked version, and the monotonic clock value corresponding to the maximum locked version from the locally maintained version record based on the resource identifier further includes:
[0028] If the client cluster mapping version number is within the valid version range and the maximum lock version corresponding to the resource identifier and the monotonic clock value corresponding to the maximum lock version are found to be the initial value, then the data request is directly determined to be a non-expired request.
[0029] This also includes:
[0030] If the client cluster mapping version number is outside the valid version range, an error response indicating that the data request is suspected of being expired is returned to notify the client to retry the data request within the validity period of the distributed lock when the data request is a retryable request.
[0031] If the lock version in the data request is less than the maximum lock version, or if the lock version in the data request is equal to the maximum lock version and the monotonic clock value in the data request is less than the monotonic clock value corresponding to the maximum lock version, then an error response indicating that the data request is explicitly expired is returned.
[0032] The process of responding to the data request further includes:
[0033] Update the maximum lock version corresponding to the resource identifier in the version record to the lock version carried in the data request, and update the monotonic clock value corresponding to the maximum lock version in the version record to the monotonic clock value carried in the data request.
[0034] To achieve the above objectives, this application provides a data request and response apparatus applied to a storage service node in a distributed storage system, the apparatus comprising:
[0035] A receiving module is used to receive data requests sent by a client; wherein, the data request carries a resource identifier, a client cluster mapping version number, a lock version, and a monotonic clock value. The resource identifier is used to identify the storage instance to which the requested data belongs. The client cluster mapping version number is used to describe the cluster topology version when the client constructs the request. The lock version is used to describe the distributed lock version currently held by the client. The monotonic clock value is used to describe the monotonic clock reading of the client at the instance startup time.
[0036] The query module is used to query the corresponding valid version range, the maximum locked version, and the monotonic clock value corresponding to the maximum locked version in the locally maintained version record based on the resource identifier.
[0037] The determining module is configured to determine whether the data request is an expired request based on one of the first comparison result, the second comparison result, and the third comparison result; wherein, the first comparison result is the comparison result between the client cluster mapping version number and the valid version range, the second comparison result includes: the comparison result between the lock version in the data request and the maximum lock version, and the first comparison result, and the third comparison result includes: the comparison result between the monotonic clock value in the data request and the monotonic clock value corresponding to the maximum lock version, and the second comparison result;
[0038] An interception module is used to intercept the data request when the data request is an expired request;
[0039] The response module is used to respond to the data request when the data request is a non-expired request.
[0040] To achieve the above objectives, this application provides an electronic device, comprising:
[0041] Memory, used to store computer programs;
[0042] A processor for implementing the steps of the data request-response method described above when executing the computer program.
[0043] To achieve the above objectives, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the data request-response method described above.
[0044] In this application, the client cluster mapping version number reflects the cluster topology state when a data request is issued. Storage service nodes can quickly eliminate invalid requests under outdated topologies caused by primary / replica switching or OSD restarts by comparing the version number with the valid version range recorded locally. The lock version directly relates to the instance's legitimacy; only instances holding the latest distributed lock are allowed to write. Storage service nodes can determine whether a data request originates from a replaced old instance by comparing the lock version in the data request with the maximum lock version recorded locally. Furthermore, when lock versions are the same, a monotonic clock value is introduced as a secondary criterion to ensure that requests are processed in chronological order within the same lock lease, preventing out-of-order writes due to retransmissions or delayed arrivals. Since the above judgments are based entirely on the information carried in the request itself and the local state of the storage node, they do not rely on external coordination or global synchronization. Each storage service node can independently and in real-time intercept expired requests. Therefore, even if a data request from an old instance arrives after being delayed in the network, as long as the new instance has already performed an update operation by acquiring the new lock version, the old request will be reliably intercepted due to the lag in lock version or monotonic clock value. This effectively prevents the risk of data overwriting and achieves precise filtering of expired data requests without centralized control. This application also discloses a data request response device, an electronic device, and a computer-readable storage medium, which can achieve the same technical effects.
[0045] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this application. Attached Figure Description
[0046] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. The drawings are used to provide a further understanding of this disclosure and constitute a part of the specification. They are used together with the following detailed description to explain this disclosure, but do not constitute a limitation of this disclosure. In the drawings:
[0047] Figure 1 This is a flowchart illustrating a data request and response method according to an exemplary embodiment;
[0048] Figure 2A flowchart illustrating another data request and response method according to an exemplary embodiment;
[0049] Figure 3 A component architecture diagram of an application embodiment provided in this application;
[0050] Figure 4 A schematic diagram illustrating a scenario where the primary replica OSD of the placement group has been switched or restarted in one application embodiment provided in this application;
[0051] Figure 5 This is a schematic diagram illustrating a scenario where the primary copy OSD of the placement group has not changed in one application embodiment provided in this application;
[0052] Figure 6 This is a structural diagram illustrating a data request response apparatus according to an exemplary embodiment;
[0053] Figure 7 This is a structural diagram of an electronic device according to an exemplary embodiment. Detailed Implementation
[0054] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.
[0055] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.
[0056] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0057] This application discloses a data request response method that effectively identifies and intercepts expired data requests to prevent old requests from overwriting new data.
[0058] See Figure 1 A flowchart illustrating a data request and response method according to an exemplary embodiment is shown below. Figure 1 As shown, it includes:
[0059] S101: Receive a data request sent by a client; wherein the data request carries a resource identifier, a client cluster mapping version number, a lock version, and a monotonic clock value. The resource identifier is used to identify the storage instance to which the requested data belongs. The client cluster mapping version number is used to describe the cluster topology version when the client constructs the request. The lock version is used to describe the distributed lock version currently held by the client. The monotonic clock value is used to describe the monotonic clock reading of the client at the instance startup time.
[0060] This embodiment is applied to a distributed storage system, and the execution entity is a storage service node in the distributed storage system.
[0061] In this step, the storage service node receives data requests from clients through the network interface, parses their header metadata, and extracts the resource identifier, client cluster mapping version number, lock version, and monotonic clock value.
[0062] It should be noted that, in this embodiment, each data request sent to the distributed storage service node must carry a four-tuple in its request header, which is abstractly represented as (resource_key, send_osdmap_epoch, version_primary, version_secondary). Among them, resource_key is the unique identifier of the target resource for which the interception policy needs to be implemented. In the PLogFS (PersistentLog-based FileSystem) instance I / O interception scenario, this field specifically uses the file system instance ID (fsid) as resource_key to identify which PLogFS instance the request is applied to; send_osdmap_epoch represents the cluster topology view version number used by the client when constructing the request, that is, the epoch value of its local OSDMap, reflecting the active nodes and data distribution status in the cluster at that time; version_primary is the primary version number, which corresponds to the distributed lock version number (version) currently held by the client in the PLogFS scenario, and is used to identify whether it is the current legitimate primary replica; version_secondary is the secondary version number, which is taken from the local monotonic clock value (T) read at the time of client process startup in the PLogFS scenario, and is used to distinguish instances with different lifecycles within the same lock version.
[0063] S102: Based on the resource identifier, query the corresponding valid version range, the maximum locked version, and the monotonic clock value corresponding to the maximum locked version in the locally maintained version record;
[0064] In this step, after receiving a data request, the storage service node extracts the resource identifier and then uses it as the key to look up the corresponding version record in a local hash table or index structure. The version record is a piece of state metadata maintained by the storage service node for each managed resource (uniquely identified by the resource identifier) in local memory or persistent logs. This includes the valid version range, the maximum lock version, and the monotonic clock value corresponding to the maximum lock version. The valid version range refers to the cluster mapping version range currently recognized by the system as a legitimate primary replica. The maximum lock version is the highest legitimate lock version number received by the resource to date, representing the identity of the latest primary replica. The monotonic clock value corresponding to the maximum lock version refers to the monotonic clock reading reported by the corresponding primary replica at its own startup time when the maximum lock version is first accepted, used to distinguish instances with different lifecycles within the same lock version.
[0065] S103: Determine whether the data request is an expired request based on one of the first comparison result, the second comparison result, and the third comparison result; wherein, the first comparison result is the comparison result between the client cluster mapping version number and the valid version range, the second comparison result includes: the comparison result between the lock version in the data request and the maximum lock version, and the first comparison result, and the third comparison result includes: the comparison result between the monotonic clock value in the data request and the monotonic clock value corresponding to the maximum lock version, and the second comparison result;
[0066] In this step, the following checks are performed: whether the client cluster mapping version number is within the valid version range; whether the lock version carried in the data request is compared with the maximum lock version; and whether the monotonic clock value carried in the data request is compared with the monotonic clock value corresponding to the maximum lock version. If the client cluster mapping version number is outside the valid version range, or the lock version carried in the data request is less than the maximum lock version, or the lock version in the data request is equal to the maximum lock version but the monotonic clock value in the data request is less than the monotonic clock value corresponding to the maximum lock version, then the data request is determined to be an expired request. Conversely, if the client cluster mapping version number is within the valid version range, or the lock version carried in the data request is greater than the maximum lock version, or the lock version in the data request is equal to the maximum lock version but the monotonic clock value in the data request is greater than the monotonic clock value corresponding to the maximum lock version, then the data request is determined to be a non-expired request.
[0067] As a feasible implementation, determining whether the data request is an expired request based on one of the first comparison result, the second comparison result, and the third comparison result includes: comparing the client cluster mapping version number with the valid version range; if the client cluster mapping version number is within the valid version range, then comparing the lock version in the data request with the maximum lock version; if the lock version in the data request is less than the maximum lock version, then determining the data request is an expired request; if the lock version in the data request is greater than the maximum lock version, then determining the data request is a non-expired request; if the lock version in the data request is equal to the maximum lock version, then comparing the monotonic clock value in the data request with the monotonic clock value corresponding to the maximum lock version; if the monotonic clock value in the data request is less than the monotonic clock value corresponding to the maximum lock version, then determining the data request is an expired request; if the monotonic clock value in the data request is greater than or equal to the monotonic clock value corresponding to the maximum lock version, then determining the data request is a non-expired request; if the client cluster mapping version number is outside the valid version range, then determining the data request is an expired request.
[0068] In practice, the system first checks if the client cluster mapping version number falls within the valid version range. If not, the data request is directly deemed expired without further comparison. If the client cluster mapping version number is within the valid version range, a double comparison is performed: First, if the requested lock version is less than the maximum lock version, it indicates the request originated from the old primary replica, thus the data request is deemed expired. Second, if the lock versions are equal but the requested monotonic clock value is less than the recorded monotonic clock value, it indicates that although the request belongs to the same logical primary replica, it was generated in its previous runtime cycle (due to the monotonic clock resetting after each restart), making it a residual request, and thus also deemed expired. Conversely, if the requested lock version is greater than the maximum lock version, it indicates the new primary replica has taken over; or if the lock versions are equal but the requested monotonic clock value is greater than the recorded monotonic clock value, it indicates a new request within the current lifecycle, thus the data request is deemed non-expired.
[0069] As a feasible implementation, the lower limit of the effective version range is the minimum cluster mapping version number when the storage service node becomes the primary replica of the placement group, and the upper limit of the effective version range is the current cluster mapping version number. Accordingly, comparing the client cluster mapping version number with the effective version range includes: determining whether the client cluster mapping version number is greater than or equal to the minimum cluster mapping version number and less than or equal to the current cluster mapping version number; if so, then the client cluster mapping version number is determined to be within the effective version range; if not, then the client cluster mapping version number is determined to be outside the effective version range.
[0070] In a distributed storage system, a Placement Group (PG) is a data shard unit used to organize objects. The minimum cluster mapping version number refers to the OSDMapepoch that first took effect during the node's current primary replica term, i.e., the starting topology version of its primary replica tenure; the current cluster mapping version number is the latest OSDMapepoch released by the system.
[0071] In practice, the effective version range is dynamically defined as [min_epoch, current_epoch], where min_epoch is set during primary replica switching or node restart, and current_epoch increments with cluster topology updates. To determine if the client cluster mapping version number is within the effective version range, it is only necessary to verify whether the client cluster mapping version number epoch satisfies min_epoch ≤ epoch ≤ current_epoch.
[0072] As a feasible implementation method, this embodiment further includes: when the storage service node restarts or the primary replica of the reset group is switched to become the new primary replica, the lower limit of the effective version range in the version record is updated to the cluster mapping version number after the restart or switch, and the maximum lock version corresponding to the resource identifier and the monotonic clock value corresponding to the maximum lock version in the version record are reset to the initial value.
[0073] In practice, whenever a storage node becomes the new primary replica due to a restart or election, its local version record is reset: the lower limit of the effective version range is set to the current OSDMap epoch, which is the cluster version at the time of the switch, and the maximum lock version and its corresponding monotonic clock value are cleared to zero.
[0074] As a feasible implementation, after querying the corresponding valid version range, maximum lock version, and monotonic clock value corresponding to the maximum lock version in the locally maintained version record based on the resource identifier, the method further includes: if the client cluster mapping version number is within the valid version range and the maximum lock version corresponding to the resource identifier and the monotonic clock value corresponding to the maximum lock version are found to be initial values, then the data request is directly determined to be a non-expired request.
[0075] In practice, when the client cluster mapping version number is confirmed to be within a valid version range, but the maximum lock version and corresponding monotonic clock value in the local version record are still at their initial values, it indicates that the resource has not processed any requests during the current primary replica's term. In this case, regardless of the specific lock version and monotonic clock value carried by the data request, it is considered the first legitimate request, directly determined as a non-expired request, and responded to.
[0076] S104: If the data request is an expired request, then intercept the data request; otherwise, respond to the data request.
[0077] In this step, the storage service node executes branch logic based on the determination result of step S103. If the data request is an expired request, it is immediately discarded. Otherwise, if the data request is not expired, it is regarded as coming from the current legitimate and active primary replica and is responded to according to the standard process.
[0078] As a possible implementation, after responding to the data request, the method further includes: updating the maximum lock version corresponding to the resource identifier in the version record to the lock version carried in the data request, and updating the monotonic clock value corresponding to the maximum lock version in the version record to the monotonic clock value carried in the data request.
[0079] In practice, if the data request is a non-expired request and a response is completed, the local version record is updated using the lock version and monotonic clock value carried by the data request to ensure that the version record always reflects the status of the latest legitimate primary copy, providing an accurate comparison benchmark for subsequent requests.
[0080] Therefore, in this embodiment, the client cluster mapping version number reflects the cluster topology when a data request is issued. Storage service nodes can quickly eliminate invalid requests due to outdated topologies caused by primary / replica switching or OSD restarts by comparing the version number with the valid version range recorded locally. The lock version directly relates to the instance's legitimacy; only instances holding the latest distributed lock are allowed to write. Storage service nodes can determine whether a data request originates from a replaced old instance by comparing the lock version in the data request with the maximum lock version recorded locally. Furthermore, when lock versions are the same, a monotonic clock value is introduced as a secondary criterion to ensure that requests are processed in chronological order within the same lock lease, preventing out-of-order writes due to retransmissions or delayed arrivals. Since the above judgments are based entirely on the information carried by the request itself and the local state of the storage node, without relying on external coordination or global synchronization, each storage service node can independently and in real-time intercept expired requests. Therefore, even if the data request from the old instance arrives after being delayed in the network, as long as the new instance has performed an update operation by acquiring the new lock version, the old request will be reliably intercepted due to the lock version or the monotonic clock value being outdated. This effectively prevents the risk of data overwriting and achieves accurate filtering of expired data requests without centralized control.
[0081] This application discloses a data request and response method. Compared to the previous embodiment, this embodiment further explains and optimizes the technical solution. Specifically:
[0082] See Figure 2A flowchart illustrating another data request-response method according to an exemplary embodiment, such as Figure 2 As shown, it includes:
[0083] S201: Receive a data request sent by the client; the data request carries a resource identifier, client cluster mapping version number, lock version, and monotonic clock value;
[0084] S202: Based on the resource identifier, query the corresponding valid version range, the maximum locked version, and the monotonic clock value corresponding to the maximum locked version in the locally maintained version record;
[0085] S203: Compare the client cluster mapping version number with the valid version range; if the client cluster mapping version number is within the valid version range, proceed to S204; if the client cluster mapping version number is outside the valid version range, proceed to S208.
[0086] S204: Compare the lock version in the data request with the maximum lock version; if the lock version in the data request is less than the maximum lock version, proceed to S206; if the lock version in the data request is greater than the maximum lock version, proceed to S207; if the lock version in the data request is equal to the maximum lock version, proceed to S205.
[0087] S205: Compare the monotonic clock value in the data request with the monotonic clock value corresponding to the maximum lock version; if the monotonic clock value in the data request is less than the monotonic clock value corresponding to the maximum lock version, proceed to S206; if the monotonic clock value in the data request is greater than or equal to the monotonic clock value corresponding to the maximum lock version, proceed to S207.
[0088] S206: Determine that the data request is expired, intercept the data request, and return an error response that explicitly states the data request is expired;
[0089] In this step, when a data request is deemed expired due to an outdated lock version or obsolete monotonic clock value, an explicit expiration error code (such as E_REQUEST_EXPIRED) is returned to the client. This indicates that the request context has permanently expired, and the client should not retry the original request. Instead, it should retrieve the latest metadata, refresh the lock state, and construct a new request. This avoids wasting system resources on ineffective retries.
[0090] S207: Determine that the data request is not expired and respond to the data request;
[0091] S208: Determine if the data request is expired, intercept the data request, and return an error response indicating that the data request is likely expired, so as to notify the client to retry the data request within the validity period of the distributed lock when the data request is a retryable request.
[0092] In this step, when a data request is deemed expired because the cluster mapping version number is outside the valid range, this may be due to a brief network partition or topology synchronization delay, rather than a primary replica switch. A suspected expiration error code (such as E_REQUEST_MAYBE_EXPIRED) is returned to the client, indicating that if the request is idempotent and still within the current distributed lock lease period, it can be safely retried. Based on this, the client can choose to retry immediately instead of rebuilding the context, reducing latency.
[0093] Therefore, this embodiment achieves a refined expiration request handling strategy by distinguishing between two types of error responses: explicit expiration and suspected expiration. The former prevents stale writes from compromising data consistency, while the latter supports safe retries to improve availability, balancing strong consistency and high availability requirements. This approach is particularly suitable for distributed environments with unstable networks or frequent topology changes.
[0094] The following describes an application embodiment provided by this application, with the component architecture diagram as follows: Figure 3 As shown, the system includes controller A, controller B, and a shared disk enclosure. Controller A is the current master node, responsible for handling I / O (Input / Output) requests initiated by upper-layer business services. Controller B is a standby node, in a ready state. Each controller internally runs the PLogFS (Persistent Log-based File System) file system and interacts with the underlying OSD (Object Storage Daemon) through the Plock (Persistent Lock) client to obtain or update the distributed lock state. The system shares the same set of physical storage resources; that is, multiple OSD instances are connected to the shared disk enclosure, and all OSDs are mounted with hard drives for persistent storage.
[0095] When an upper-layer business service sends an I / O request to controller A, the request is first processed by PLogFS, and its Plock client generates a four-tuple containing the resource identifier, cluster mapping version number, lock version, and monotonic clock value, which is then sent to the corresponding OSD. The OSD containing the "Data PG Master" in the diagram represents the primary replica currently carrying the target data, and its responsibilities include receiving and processing write requests. Simultaneously, this OSD also acts as a "Lock PG Slave," serving as a subordinate replica of the lock management logic for synchronizing lock states. The Plock client of controller B then sends a lock request to its associated OSD (marked as the "Lock PG Master") to apply for or verify the legitimacy of the distributed lock, thereby achieving lock state coordination between the master and slave.
[0096] I / O requests originate from controller A and are routed through PLogFS to the OSD where the data PG master resides. Lock requests are coordinated between controllers via a dual-controller arbitration mechanism to ensure that only one controller holds a valid lock. During a master-slave switchover, I / O requests from the original master node (e.g., controller A) may arrive at the OSD controlled by the new master node (e.g., controller B) delayed due to network latency or other reasons. In this case, an I / O interception mechanism needs to be implemented on the OSD where the "data PG master" resides. This mechanism compares the four-tuple information carried in the request with the locally maintained version record to determine if it is an expired request. If it is determined to be expired, execution is rejected to prevent write operations from the old instance from overwriting data already written by the new master node, thereby avoiding data inconsistency issues caused by split-brain.
[0097] If the primary replica OSD of the relocation group has been switched or restarted, such as... Figure 4As shown. The old plogfs instance initiates a lock request to the distributed lock master (step 1), and returns lock OK after acquiring the lock (step 2). At this time, the old instance records the current lock version ver1 and monotonic clock T1 in its local memory (step 3), and sends a four-tuple containing the resource identifier fsid, client cluster mapping version number client_epoch1, lock version ver1, and monotonic clock T1 to the metadata PG master (step 4). The metadata PG master verifies whether client_epoch1 is greater than or equal to its recorded last_primary_start (i.e., the OSD Map epoch when the last primary replica started). If it passes, it allows to continue (step 5); then it compares whether the requested (ver1,T1) is greater than the (ver_s,Ts) currently maintained by the server, which is initially (0,0), so the check passes (step 6), and updates the server status to (ver1,T1) (step 7), finally returning the list result (step 8). However, when performing the deletion operation later, due to the failure of private network communication (step 9), the delete request is stuck in the network protocol stack and cannot be delivered in time. Meanwhile, the new PLogFS instance successfully acquires the distributed lock (steps 10-11) and begins processing new write requests. It records the new lock version ver2 and the monotonic clock T2 locally (step 12), and sends a new list request to the metadata PG master (step 13), carrying client_epoch2 (the current OSD Map epoch), ver2, and T2. The metadata PG master verifies again that client_epoch2 ≥ last_primary_start (step 14), and confirms the new request is "newer" by comparing (ver2, T2) > (ver1, T1) (step 15), updates the server state to (ver2, T2) (step 16), and returns the list result (step 17). At this point, due to a PG master switch or master node restart, the original server state (ver_s, Ts) is reset to (0, 0) (step 18). When an old delete request (containing fsid, client_epoch1, ver1, and T1) stuck in the protocol stack finally reaches the metadata PG master, the system re-checks whether its client_epoch1 is greater than or equal to last_primary_start. However, at this point, last_primary_start has been promoted to the current OSD Mapepoch, and client_epoch1 is clearly less than this value (step 20), causing the check to fail. Therefore, the request is determined to be expired, and the PLOG_OP_MAY_STALE error code is returned (step 21).Upon receiving the error, the old instance checks whether it is still within the validity period of the lock lease: if it has not timed out, it resends the delete request (steps 23-24); if it has timed out, it directly returns the error (step 25).
[0098] If the primary copy OSD of the relocation group has not changed, such as Figure 5As shown. The old plogfs instance initiates a lock request to the distributed lock master (step 1), and returns lock OK after acquiring the lock (step 2). The old instance records the current lock version ver1 and monotonic clock T1 in local memory (step 3), and sends a list request to the metadata PG master, carrying a four-tuple (fsids, client_epoch1, ver1, T1), where client_epoch1 is the OSD Map epoch when the request was generated (step 4). The metadata PG master verifies whether client_epoch1 is greater than or equal to its recorded last_primary_start (i.e., the epoch when the last primary replica started). If it passes, it allows to continue (step 5); then it compares whether the requested (ver1, T1) is greater than the current server state (ver_s, Ts) (initially (0, 0)). After the check passes, it updates the server state to (ver1, T1) (step 6) and returns the list result (step 7). However, due to private network communication problems, the subsequent deletion operation fails (step 8), and the next delete request is stuck in the protocol stack and cannot be delivered in time (step 9). Meanwhile, the new plogfs instance successfully acquires the distributed lock (steps 10-11) and records the new lock version ver2 and monotonic clock T2 locally (step 12). The new instance sends a list request to the metadata PG master, carrying client_epoch2, ver2, and T2 (step 13). The metadata PG master verifies that client_epoch2 ≥ last_primary_start (step 14) and confirms that it is "newer" by comparing (ver2, T2) > (ver1, T1) (step 15), updates the server status to (ver2, T2) (step 16), and returns the list result (step 17). At this point, the stuck old delete request (containing fsid, client_epoch1, ver1, and T1) finally reaches the metadata PG master (step 17). The system first verifies whether its client_epoch1 is greater than or equal to last_primary_start, which usually passes (step 18); however, a core check is then performed: the current server state (ver_s, Ts) has been updated to (ver2, T2), while the request carries (ver1, T1) < (ver2, T2), therefore the check fails (step 19). The request is determined to be expired and a PLOG_OP_STALE error code is returned (step 20). Upon receiving this error, the old instance returns it to the upper-layer application (step 21).
[0099] As can be seen, in a split-brain scenario, regardless of the stage of in-transit I / O, as long as the new instance has performed an update operation on a resource, the old instance's I / O will be intercepted and rejected when it arrives at the OSD server. The instance failover process does not need to care about the status of in-transit I / O or wait for it to complete, greatly improving the timeliness of instance failover and shortening the business interruption time caused by fault failover. This embodiment does not rely on a central control node and can achieve global in-transit I / O interception capability in a distributed environment.
[0100] The following describes a data request response apparatus provided in an embodiment of this application. The data request response apparatus described below can be referred to in conjunction with the data request response method described above.
[0101] See Figure 6 A structural diagram of a data request response apparatus is shown according to an exemplary embodiment, as follows: Figure 6 As shown, it includes:
[0102] The receiving module 100 is used to receive a data request sent by a client; wherein the data request carries a resource identifier, a client cluster mapping version number, a lock version, and a monotonic clock value. The resource identifier is used to identify the storage instance to which the requested data belongs. The client cluster mapping version number is used to describe the cluster topology version when the client constructs the request. The lock version is used to describe the distributed lock version currently held by the client. The monotonic clock value is used to describe the monotonic clock reading of the client at the instance startup time.
[0103] The query module 200 is used to query the corresponding valid version range, the maximum locked version, and the monotonic clock value corresponding to the maximum locked version in the locally maintained version record based on the resource identifier.
[0104] The determining module 300 is configured to determine whether the data request is an expired request based on one of the first comparison result, the second comparison result, and the third comparison result; wherein, the first comparison result is the comparison result between the client cluster mapping version number and the valid version range; the second comparison result includes: the comparison result between the lock version in the data request and the maximum lock version, and the first comparison result; the third comparison result includes: the comparison result between the monotonic clock value in the data request and the monotonic clock value corresponding to the maximum lock version, and the second comparison result.
[0105] The interception module 400 is used to intercept the data request when the data request is an expired request;
[0106] The response module 500 is used to respond to the data request when the data request is a non-expired request.
[0107] Therefore, in this embodiment, the client cluster mapping version number reflects the cluster topology when a data request is issued. Storage service nodes can quickly eliminate invalid requests due to outdated topologies caused by primary / replica switching or OSD restarts by comparing the version number with the valid version range recorded locally. The lock version directly relates to the instance's legitimacy; only instances holding the latest distributed lock are allowed to write. Storage service nodes can determine whether a data request originates from a replaced old instance by comparing the lock version in the data request with the maximum lock version recorded locally. Furthermore, when lock versions are the same, a monotonic clock value is introduced as a secondary criterion to ensure that requests are processed in chronological order within the same lock lease, preventing out-of-order writes due to retransmissions or delayed arrivals. Since the above judgments are based entirely on the information carried by the request itself and the local state of the storage node, without relying on external coordination or global synchronization, each storage service node can independently and in real-time intercept expired requests. Therefore, even if the data request from the old instance arrives after being delayed in the network, as long as the new instance has performed an update operation by acquiring the new lock version, the old request will be reliably intercepted due to the lock version or the monotonic clock value being outdated. This effectively prevents the risk of data overwriting and achieves accurate filtering of expired data requests without centralized control.
[0108] Based on the above embodiments, as a preferred embodiment, the determining module 300 includes:
[0109] The first comparison unit is used to compare the client cluster mapping version number with the valid version range;
[0110] The second comparison unit is used to compare the lock version in the data request with the maximum lock version when the client cluster mapping version number is within the valid version range.
[0111] The first determination unit is used to determine that the data request is an expired request when the lock version in the data request is less than the maximum lock version;
[0112] The second determination unit is used to determine that the data request is a non-expired request when the lock version in the data request is greater than the maximum lock version;
[0113] The third comparison unit is used to compare the monotonic clock value in the data request with the monotonic clock value corresponding to the maximum lock version when the lock version in the data request is equal to the maximum lock version.
[0114] The third determination unit is used to determine that the data request is an expired request when the monotonic clock value in the data request is less than the monotonic clock value corresponding to the maximum lock version.
[0115] The fourth determination unit is used to determine that the data request is a non-expired request when the monotonic clock value in the data request is greater than or equal to the monotonic clock value corresponding to the maximum lock version.
[0116] The fifth determination unit is used to determine that the data request is an expired request when the client cluster mapping version number is outside the valid version range.
[0117] Based on the above embodiments, as a preferred implementation, the lower limit of the effective version range is the minimum cluster mapping version number when the storage service node becomes the primary replica of the placement group, and the upper limit of the effective version range is the current cluster mapping version number.
[0118] Accordingly, the first comparison unit is specifically used to: determine whether the client cluster mapping version number is greater than or equal to the minimum cluster mapping version number and less than or equal to the current cluster mapping version number; if yes, then determine that the client cluster mapping version number is within the valid version range; if no, then determine that the client cluster mapping version number is outside the valid version range.
[0119] Based on the above embodiments, as a preferred embodiment, the device further includes:
[0120] The first update module is used to update the lower limit of the effective version range in the version record to the cluster mapping version number after the storage service node restarts or the primary replica of the reset group is switched to become the new primary replica, and to reset the maximum lock version corresponding to the resource identifier and the monotonic clock value corresponding to the maximum lock version in the version record to the initial value.
[0121] Based on the above embodiments, as a preferred embodiment, the determining module 300 further includes:
[0122] The fourth determination unit is used to directly determine that the data request is a non-expired request when the client cluster mapping version number is within the valid version range and the maximum lock version corresponding to the resource identifier and the monotonic clock value corresponding to the maximum lock version are found to be the initial value.
[0123] Based on the above embodiments, as a preferred embodiment, the determining module 300 further includes:
[0124] The first return unit is used to return an error response indicating that the data request is suspected to be expired when the client cluster mapping version number is outside the valid version range, so as to notify the client to retry the data request within the validity period of the distributed lock when the data request is a retryable request.
[0125] The second return unit is used to return an error response indicating that the data request has expired when the lock version in the data request is less than the maximum lock version, or when the lock version in the data request is equal to the maximum lock version and the monotonic clock value in the data request is less than the monotonic clock value corresponding to the maximum lock version.
[0126] Based on the above embodiments, as a preferred embodiment, it further includes:
[0127] The second update module is used to update the maximum lock version corresponding to the resource identifier in the version record to the lock version carried in the data request after responding to the data request, and to update the monotonic clock value corresponding to the maximum lock version in the version record to the monotonic clock value carried in the data request.
[0128] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.
[0129] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiments of this application, the embodiments of this application also provide an electronic device. Figure 7 This is a structural diagram of an electronic device according to an exemplary embodiment, such as... Figure 7 As shown, the electronic device includes:
[0130] Communication interface 1 enables information exchange with other devices, such as network devices;
[0131] Processor 2 is connected to communication interface 1 to enable information interaction with other devices and, when running a computer program, executes the data request and response methods provided by one or more of the above-mentioned technical solutions. The computer program is stored in memory 3.
[0132] Of course, in practical applications, the various components in an electronic device are coupled together through bus system 4. It can be understood that bus system 4 is used to achieve communication and connection between these components. In addition to the data bus, bus system 4 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 7 The general will label all buses as Bus System 4.
[0133] The memory 3 in this embodiment is used to store various types of data to support the operation of the electronic device. Examples of such data include any computer program used to operate on the electronic device.
[0134] It is understood that memory 3 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 3 described in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.
[0135] The methods disclosed in the embodiments of this application can be applied to processor 2, or implemented by processor 2. Processor 2 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 2 or by instructions in the form of software. The processor 2 may be a general-purpose processor, DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 2 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory 3. Processor 2 reads the program in memory 3 and completes the steps of the aforementioned method in combination with its hardware.
[0136] When processor 2 executes the program, it implements the corresponding processes in the various methods of the embodiments of this application. For the sake of brevity, these will not be described in detail here.
[0137] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a memory 3 that stores a computer program, which can be executed by a processor 2 to complete the steps described in the aforementioned method. The computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM.
[0138] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.
[0139] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.
[0140] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.
Claims
1. A data request and response method, characterized in that, The method, applied to a storage service node in a distributed storage system, includes: Receive data requests sent by clients; wherein, the data requests carry a resource identifier, a client cluster mapping version number, a lock version, and a monotonic clock value, the resource identifier is used to identify the storage instance to which the requested data belongs, the client cluster mapping version number is used to describe the cluster topology version when the client constructs the request, the lock version is used to describe the distributed lock version currently held by the client, and the monotonic clock value is used to describe the monotonic clock reading of the client at the time of instance startup; Based on the resource identifier, query the corresponding valid version range, the maximum locked version, and the monotonic clock value corresponding to the maximum locked version in the locally maintained version record; Whether the data request is an expired request is determined based on one of the first comparison result, the second comparison result, and the third comparison result; wherein, the first comparison result is the comparison result between the client cluster mapping version number and the valid version range, the second comparison result includes: the comparison result between the lock version in the data request and the maximum lock version, and the first comparison result, and the third comparison result includes: the comparison result between the monotonic clock value in the data request and the monotonic clock value corresponding to the maximum lock version, and the second comparison result; If the data request is expired, the data request is intercepted; otherwise, the data request is responded to.
2. The data request and response method according to claim 1, characterized in that, Determining whether the data request is an expired request based on one of the first comparison result, the second comparison result, and the third comparison result includes: Compare the client cluster mapping version number with the valid version range; If the client cluster mapping version number is within the valid version range, then compare the lock version in the data request with the maximum lock version; If the lock version in the data request is less than the maximum lock version, the data request is determined to be an expired request. If the lock version in the data request is greater than the maximum lock version, then the data request is determined to be a non-expired request; If the lock version in the data request is equal to the maximum lock version, then compare the monotonic clock value in the data request with the monotonic clock value corresponding to the maximum lock version. If the monotonic clock value in the data request is less than the monotonic clock value corresponding to the maximum lock version, then the data request is determined to be an expired request. If the monotonic clock value in the data request is greater than or equal to the monotonic clock value corresponding to the maximum lock version, then the data request is determined to be a non-expired request. If the client cluster mapping version number is outside the valid version range, the data request is determined to be an expired request.
3. The data request and response method according to claim 2, characterized in that, The lower limit of the effective version range is the minimum cluster mapping version number when the storage service node becomes the primary replica of the placement group, and the upper limit of the effective version range is the current cluster mapping version number. Accordingly, comparing the client cluster mapping version number with the valid version range includes: Determine whether the client cluster mapping version number is greater than or equal to the minimum cluster mapping version number and less than or equal to the current cluster mapping version number; If so, then the client cluster mapping version number is determined to be within the valid version range; If not, then the client cluster mapping version number is determined to be outside the valid version range.
4. The data request and response method according to claim 3, characterized in that, The method further includes: When the storage service node restarts or the primary replica of the reset group is switched to become the new primary replica, the lower limit of the effective version range in the version record is updated to the cluster mapping version number after the restart or switch, and the maximum lock version corresponding to the resource identifier and the monotonic clock value corresponding to the maximum lock version in the version record are reset to the initial value.
5. The data request and response method according to claim 4, characterized in that, After querying the corresponding valid version range, the maximum locked version, and the monotonic clock value corresponding to the maximum locked version from the locally maintained version record based on the resource identifier, the method further includes: If the client cluster mapping version number is within the valid version range and the maximum lock version corresponding to the resource identifier and the monotonic clock value corresponding to the maximum lock version are found to be the initial value, then the data request is directly determined to be a non-expired request.
6. The data request and response method according to claim 2, characterized in that, Also includes: If the client cluster mapping version number is outside the valid version range, an error response indicating that the data request is suspected of being expired is returned to notify the client to retry the data request within the validity period of the distributed lock when the data request is a retryable request. If the lock version in the data request is less than the maximum lock version, or if the lock version in the data request is equal to the maximum lock version and the monotonic clock value in the data request is less than the monotonic clock value corresponding to the maximum lock version, then an error response indicating that the data request is explicitly expired is returned.
7. The data request and response method according to claim 1, characterized in that, Following the response to the data request, the method further includes: Update the maximum lock version corresponding to the resource identifier in the version record to the lock version carried in the data request, and update the monotonic clock value corresponding to the maximum lock version in the version record to the monotonic clock value carried in the data request.
8. A data request and response device, characterized in that, The device is used as a storage service node in a distributed storage system, and includes: A receiving module is used to receive data requests sent by a client; wherein, the data request carries a resource identifier, a client cluster mapping version number, a lock version, and a monotonic clock value. The resource identifier is used to identify the storage instance to which the requested data belongs. The client cluster mapping version number is used to describe the cluster topology version when the client constructs the request. The lock version is used to describe the distributed lock version currently held by the client. The monotonic clock value is used to describe the monotonic clock reading of the client at the instance startup time. The query module is used to query the corresponding valid version range, the maximum locked version, and the monotonic clock value corresponding to the maximum locked version in the locally maintained version record based on the resource identifier. The determining module is configured to determine whether the data request is an expired request based on one of the first comparison result, the second comparison result, and the third comparison result; wherein, the first comparison result is the comparison result between the client cluster mapping version number and the valid version range, the second comparison result includes: the comparison result between the lock version in the data request and the maximum lock version, and the first comparison result, and the third comparison result includes: the comparison result between the monotonic clock value in the data request and the monotonic clock value corresponding to the maximum lock version, and the second comparison result; An interception module is used to intercept the data request when the data request is an expired request; The response module is used to respond to the data request when the data request is a non-expired request.
9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the data request-response method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed, implements the steps of the data request-response method as described in any one of claims 1 to 7.