Performance-aware cross-client cache co-access method and apparatus

Abstract

The application provides a performance-aware cross-client cache collaborative access method and device, which is applied to the client side of a distributed file system. The method comprises the following steps: after the client successfully reads file data and caches the file data locally, an interval cache description item is established with a continuous data interval as a minimum management unit and is synchronized to other clients; in response to a data reading request, if the local cache does not hit target data, interval cache description items published by other clients are retrieved, and it is determined whether there is valid remote data interval that completely covers the target data; if there is, the expected time consumption for obtaining the target data across clients and the expected time consumption for reading the target data from an object storage target are estimated, the target data obtaining mode is decided based on the two time consumptions, and the target data is obtained in the decided mode. Through the above method, the sharing of cache resources among clients can be realized, the redundant I / O pressure of the object storage target is reduced, and the reading performance is prevented from being degraded.

Description

Technical Field

[0001] This application relates to the field of distributed storage technology, and in particular to a performance-aware cross-client cache collaborative access method and apparatus. Background Technology

[0002] In applications of distributed file systems, high demands are placed on their data read performance. For example, in AI distributed training scenarios, multiple training nodes repeatedly read the same training samples in each training epoch, which places extremely high demands on the data read performance of the distributed file system.

[0003] Currently, clients typically use local caching mechanisms (such as page cache, persistent client-side cache, etc.) to cache read data. However, client-side caches are all private to a single node. When multiple clients frequently read the same data from the same file, even if one client has just read the data, other clients will still access the object storage target again because the cache is not present on their local node. As a result, each client needs to independently initiate a read request for the same data to the same object storage target (OST), causing the object storage target's resources to be consumed by a large number of duplicate requests, which can easily become a system performance bottleneck. Summary of the Invention

[0004] In view of this, embodiments of this application provide a performance-aware cross-client cache collaborative access method and apparatus to achieve collaborative utilization of cache resources among clients, reduce redundant I / O pressure on object storage targets, and maintain read performance without degradation.

[0005] This application provides a performance-aware cross-client cache collaborative access method, which is applied to the client side of a distributed file system. The method includes: After a client successfully reads file data through the distributed file system and caches it locally, it establishes a range cache description item locally with a continuous data range as the smallest management unit and synchronizes it to other clients; the range cache description item does not contain the actual file, and the range cache description item is used to describe the cache information of the file data on the client; In response to a data read request, if the target data requested by the data read request is not found in the local cache, the range cache description item published by other clients is retrieved to determine whether there is a valid remote data range that completely covers the data read request. If the valid remote data range exists, estimate the expected time Tpeer for acquiring the target data across clients and the expected time Tost for reading the target data from the object storage target; decide on the target data acquisition method based on Tpeer and Tost. If the decision is made that the target data acquisition method is cross-client acquisition, then the target data shall be acquired in the cross-client manner; If the decision is made that the target data acquisition method is a non-cross-client acquisition method, then the target data is directly obtained from the object storage target.

[0006] This application also provides a performance-aware cross-client cache collaborative access device, which is applied to any client of a distributed file system. The device includes: A module is created to establish a range cache description item locally and synchronize it to other clients after a client successfully reads file data through a distributed file system and caches it locally, using a continuous data range as the smallest management unit. The range cache description item does not contain the file data; instead, it describes the cache information of the file data on the client. The retrieval module is used to respond to a data read request. If the target data requested by the data read request is not found in the local cache, it retrieves the range cache description items published by other clients to determine whether there is a valid remote data range that completely covers the data read request. The decision module is used to estimate, if the valid remote data range exists, the expected time Tpeer for acquiring the target data across clients and the expected time Tost for reading the target data from the object storage target; and to decide on the target data acquisition method based on Tpeer and Tost. The acquisition module is used to acquire the target data in a cross-client manner if the target data acquisition method is determined to be cross-client; and to acquire the target data directly from the object storage target if the target data acquisition method is determined to be non-cross-client.

[0007] This application also provides an electronic device, including: a processor and a machine-readable storage medium for storing machine-executable instructions, wherein the machine-executable instructions, when run by the machine-readable storage medium, cause the processor to perform the steps of the above method.

[0008] This application also provides a machine-readable storage medium storing machine-executable instructions that, when executed, enable the implementation of the steps described above.

[0009] As can be seen from the above technical solution, in this embodiment, after the client successfully reads file data through the distributed file system and caches it locally, it establishes a range cache descriptor with continuous data ranges as the smallest management unit and synchronizes it to other clients. Since the range cache descriptor does not contain the file data itself, only the lightweight range cache descriptor is transmitted during the synchronization process, without transmitting the actual data. This achieves cross-client synchronization of cache status with minimal network overhead, breaking the limitations of traditional technologies where client caches are isolated from each other and cannot know each other's cache status. In this way, in response to data read requests, it determines whether there is a valid remote data range that completely covers the target data. It then diverts repeated read requests that were originally concentrated on the object storage target to other clients holding the valid remote data range, thereby significantly reducing the number of direct accesses to the backend object storage target, effectively alleviating the I / O pressure on the object storage target, significantly saving backend storage bandwidth resources, and preventing it from becoming a system performance bottleneck due to resource exhaustion.

[0010] Furthermore, when a valid remote data range exists, instead of blindly allowing other clients within that range to retrieve the target data, the system estimates the expected time (Tpeer) for cross-client retrieval of the target data and the expected time (Tost) for reading the target data from the object storage target. Based on the comparison between Tpeer and Tost, it decides on the target data retrieval method. If a cross-client retrieval method is chosen, the target data is retrieved using this method; otherwise, if a non-cross-client retrieval method is chosen, the target data is retrieved directly from the object storage target. This ensures that cross-client retrieval is only selected when the read cost is less than reading from the object storage target, thus avoiding read performance degradation. Attached Figure Description

[0011] Figure 1 A network architecture diagram of the distributed file system provided in the embodiments of this application; Figure 2 A flowchart illustrating the method provided in the embodiments of this application; Figure 3 A flowchart illustrating the decision target data acquisition method provided in this application embodiment; Figure 4 Another schematic flowchart illustrating the method provided in this application embodiment; Figure 5 This is a schematic diagram of the device provided in the embodiments of this application; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0012] To make the methods provided in this application easier to understand, the methods provided in this application will be described in detail below with reference to the accompanying drawings and embodiments.

[0013] Before introducing the method provided in the embodiments of this application, let's first combine... Figure 1 The existing technical problems are explained: See Figure 1 , Figure 1 This is a schematic diagram of the network architecture of the distributed file system provided in an embodiment of this application. Figure 1 As shown, this distributed file system includes a Metadata Server (MDS), multiple Object Storage Targets (OSTs), and multiple clients. The MDS manages the metadata of the files in the file system, including the file directory structure, file attributes, and file stripe layout information. The OST stores the actual data content of the files, and the clients, as access nodes, provide data access interfaces for upper-layer applications.

[0014] In related technologies, when a client receives a data read request, it first sends a metadata request to the MDS to obtain the stripe layout information of the file corresponding to the data read request. The stripe layout information includes at least which OSTs the file's data is distributed on, as well as the offset and length of each data segment on the corresponding OST. Based on the stripe layout information returned by the MDS, the client establishes a connection with the corresponding OST and reads the required data.

[0015] To reduce I / O latency and alleviate the access pressure on the OST, clients typically use local caching mechanisms (such as page cache, persistent client cache, etc.) to cache the data they have already read. However, the client caches of existing distributed file systems are all private to a single node. When multiple clients frequently read the same data from the same file, even if one client has just read the data, other clients will still access the OST again because the data is not cached on their own nodes. This results in a large number of duplicate requests consuming the OST's resources, which can easily become a system performance bottleneck.

[0016] For example, suppose client A has just read a segment of data from file F from OST1 and cached it locally. Now, client B also needs to read the same segment of data from file F. Since client B does not cache this data locally, it can only send the same read request to OST1. If multiple clients, such as client C and client D, simultaneously need to read this data, each client must independently send a read request to OST1. OST1 will then have to process multiple identical requests repeatedly, resulting in a significant consumption of its bandwidth and IOPS resources.

[0017] Based on this, embodiments of this application provide a performance-aware cross-client cache collaborative access method and apparatus to achieve collaborative utilization of cache resources among clients, reduce redundant I / O pressure on object storage targets, and maintain read performance without degradation.

[0018] See Figure 2 , Figure 2 This is a flowchart illustrating the method provided in an embodiment of this application. The method is applied to the client side of a distributed file system.

[0019] like Figure 2 As shown, the method may include the following steps: S201: After a client successfully reads file data through the distributed file system and caches it locally, it establishes a range cache description item locally with a continuous data range as the smallest management unit and synchronizes it to other clients.

[0020] In this embodiment, the interval cache descriptor does not contain the actual file; it is used to describe the cached information of file data on the client. The specific process for creating the interval cache descriptor will be illustrated later and will not be elaborated here.

[0021] S202, in response to a data read request, if the target data requested by the data read request is not found in the local cache, then retrieve the range cache description items published by other clients to determine whether there is a valid remote data range that completely covers the data read request.

[0022] In this embodiment, determining whether the local cache has hit the target data requested by the data read request adopts the technical means in the related technologies of distributed file systems, and this embodiment of the application is not specifically limited.

[0023] The specific implementation of retrieving range cache descriptions published by other clients to determine whether there is a valid remote data range that completely covers the data read request will be explained in detail later, and will not be elaborated here.

[0024] S203, if a valid remote data range exists, estimate the expected time Tpeer for obtaining target data across clients and the expected time Tost for reading target data from object storage target; decide on the target data acquisition method based on Tpeer and Tost.

[0025] In other words, if no valid remote data range exists, the target data will be retrieved directly from the object storage target.

[0026] In this embodiment, the specific estimation methods of Tpeer and Tost, as well as the data acquisition methods for decision-making targets based on Tpeer and Tost, will be explained in detail later, and will not be repeated here.

[0027] S204. If the decision is made that the target data acquisition method is cross-client acquisition, then the target data is acquired in the cross-client manner. If the decision is made that the target data acquisition method is non-cross-client acquisition, then the target data is acquired directly from the object storage target.

[0028] This concludes the process. Figure 2 The process is shown below.

[0029] pass Figure 2 As shown in the flowchart, in this embodiment, after the client successfully reads file data through the distributed file system and caches it locally, it establishes a range cache descriptor with continuous data ranges as the smallest management unit and synchronizes it to other clients. Since the range cache descriptor does not contain the file data itself, only the lightweight range cache descriptor is transmitted during the synchronization process, without transmitting the actual data. This achieves cross-client synchronization of cache status with minimal network overhead, breaking the limitations of traditional technologies where client caches are isolated from each other and cannot know each other's cache status. In this way, in response to data read requests, it determines whether there is a valid remote data range that completely covers the target data. It then diverts repeated read requests that were originally concentrated on the object storage target to other clients holding the valid remote data range, thereby significantly reducing the number of direct accesses to the backend object storage target, effectively alleviating the I / O pressure on the object storage target, significantly saving backend storage bandwidth resources, and preventing it from becoming a system performance bottleneck due to resource exhaustion.

[0030] Furthermore, when a valid remote data range exists, instead of blindly allowing other clients within that range to retrieve the target data, the system estimates the expected time (Tpeer) for cross-client retrieval of the target data and the expected time (Tost) for reading the target data from the object storage target. Based on the comparison between Tpeer and Tost, it decides on the target data retrieval method. If a cross-client retrieval method is chosen, the target data is retrieved using this method; otherwise, if a non-cross-client retrieval method is chosen, the target data is retrieved directly from the object storage target. This ensures that cross-client retrieval is only selected when the read cost is less than reading from the object storage target, thus avoiding read performance degradation.

[0031] Furthermore, since the above method does not require modification of the distributed file system's metadata server and object storage target, but only adds collaborative access logic on each client side, it has good compatibility and deployment flexibility.

[0032] The following is a detailed explanation of the specific process for establishing the interval cache descriptor in step S201 above: The source of file data successfully read by the client includes two scenarios: one is reading from the object storage target (OST) of the distributed file system, and the other is obtaining it from other clients through cross-client methods. Regardless of the source of the data, after successfully obtaining the file data and caching it locally, the client establishes a corresponding interval cache descriptor with a continuous data interval as the smallest management unit.

[0033] Any contiguous data range refers to a continuous storage range of file data on the OST. For example, {FID:0x12345, OST:5, offset:1024, len:4096} means that the data of file FID 0x12345 is stored on OST5, starting from offset 1024 bytes and with a length of 4096 bytes.

[0034] A range cache descriptor describes the caching information of file data on this client; that is, it notifies other clients in the distributed file system that the client has cached the file data locally. The range cache descriptor does not contain the file data itself, but it includes at least: the file identifier (File IdentifierDescriptor, FID) to which the file data belongs, the continuous data range of the file data on the OST (represented by OST ID, starting offset, and length len), and validity status information. The validity status information indicates whether the cached data is available. It is marked as valid when the range cache descriptor is created, and is updated from valid to invalid when the file data is modified or the range cache descriptor's lifespan exceeds a preset lifespan.

[0035] Continuing from the previous example, the interval cache descriptor created for the aforementioned continuous data interval is represented as follows: {FID:0x12345, OST:5, offset:1024, len:4096, valid:1, checksum:0xABCD}.

[0036] Wherein, FID:0x12345 represents the globally unique identifier of the file to which the file data belongs in the distributed file system; OST:5 indicates that the file data is stored on OST5; ​​offset:1024 indicates that the starting offset of this continuous data range on OST5 is 1024 bytes; len:4096 indicates that the length of this continuous data range is 4096 bytes. valid:1 indicates that the cached data is currently valid; checksum:0xABCD represents the checksum corresponding to this data range, which is used to verify the correctness of data during cross-node transmission.

[0037] After establishing a range cache descriptor, the client synchronizes it with other clients in the distributed file system, enabling other clients to perceive the cached data on their end. In this embodiment, since the range cache descriptor does not contain the file data itself, only the lightweight range cache descriptor is transmitted during synchronization, without transmitting the actual data, thereby achieving cross-client synchronization of cached states with minimal network overhead.

[0038] The above provides a detailed explanation of the specific process for establishing the interval cache descriptor in step S201.

[0039] The following section elaborates on the specific implementation of step S202 above, which involves retrieving range cache descriptions published by other clients to determine whether there exists a valid remote data range that completely covers the data read request.

[0040] When the target data requested by the data read request is not found in the local cache, the client initiates a remote retrieval process to search for range cache descriptors from other clients besides itself. For example, in one specific implementation, the client searches for descriptors from range cache descriptors published by other clients that meet the following conditions: the validity status information is valid, the indicated file identifier matches the file identifier of the target data, and the indicated continuous data range completely covers the continuous data range corresponding to the target data. The remote data range corresponding to the found range cache descriptor is determined as the valid remote data range.

[0041] Optionally, as an example, the above-mentioned determination of the remote data range corresponding to the found range cache descriptor as a valid remote data range is performed under the premise that only one range cache descriptor that meets the conditions is found.

[0042] If multiple interval cache descriptors satisfying the above conditions are found, a target client is selected from the synchronization source clients of these clients. Specifically, the target client is selected from multiple clients according to the priority order of same-switch priority and Remote Direct Memory Access (RDMA) online priority. That is, clients on the same network switch as the current client are preferred; if no client on the same switch exists, clients supporting RDMA and online are preferred. After determining the target client, the remote data interval corresponding to the interval cache descriptor published by the target client is determined as the valid remote data interval. Through the above selection method, the client with the shortest network distance and the highest transmission efficiency can be selected as the data source, further optimizing the performance of cross-client data acquisition.

[0043] If no matching range cache descriptor is found, it is determined that there is no valid remote data range, and the client directly determines to retrieve the target data from the object storage target.

[0044] The above provides a detailed explanation of the specific implementation method for retrieving interval cache description items published by other clients in step S202 to determine whether there is a valid remote data interval that completely covers the data read request.

[0045] The following is combined with Figure 3 The paper elaborates on the specific estimation methods for Tpeer and Tost, as well as the data acquisition methods for decision-making objectives based on Tpeer and Tost.

[0046] See Figure 3 , Figure 3 A flowchart illustrating the method for acquiring decision target data provided in this application embodiment.

[0047] like Figure 3 As shown, the process may include the following steps: S301, respectively estimate the expected time Tpeer for obtaining target data across clients and the expected time Tost for reading target data from object storage target.

[0048] In this embodiment, if a valid remote data range exists, the client does not blindly obtain the target data from the target client holding the valid remote data range, but instead estimates the expected time consumption of the two acquisition paths respectively.

[0049] Tpeer is used to characterize the expected time required to retrieve target data from a target client across nodes. Tpeer is determined based on the following parameters: the size of the target data, the current available bandwidth between the current client and the target client (the client holding the valid remote data range), the network round-trip time (RTT) between the current client and the target client, and the queuing latency of the target client's read I / O operations.

[0050] Specifically, the size of the target data is positively correlated with Tpeer. The current available bandwidth between the current client and the target client is negatively correlated with Tpeer. The network round-trip time (RTT) between the current client and the target client is positively correlated with Tpeer. The queuing latency of the target client's read I / O operations is positively correlated with Tpeer.

[0051] Alternatively, Tpeer is determined by the following formula: Tpeer = DataSize / BW + RTT + Q_peer Where DataSize is the size of the target data; BW represents the currently available bandwidth between this client and the target client. RTT is the network round-trip time between this client and the target client; Q_peer represents the queuing latency for read I / O operations on the target client.

[0052] The current available bandwidth between this client and the target client, the network round-trip time (RTT) between this client and the target client, and the queuing latency of the target client's read I / O operations are all parameters collected periodically. Parameters for each period are obtained separately. When performing Tpeer estimation, the client selects the parameter values ​​collected in the most recent period as the basis.

[0053] For example, the current available bandwidth between this client and the target client is obtained through periodic measurements by this client. Within each detection period, this client initiates a bandwidth speed test with the target client to obtain the available bandwidth value for the current period. Specifically, if the Remote Direct Memory Access (RDMA) protocol is supported between this client and the target client, the current available bandwidth of the RDMA link is obtained using the ib_read_bw command; if only the Transmission Control Protocol (TCP) protocol is supported, the current available bandwidth of the TCP link is obtained using the iperf3 command. When performing Tpeer estimation, this client uses the bandwidth value obtained in the most recent period.

[0054] The network round-trip time (RTT) between this client and the target client is obtained by this client through periodic heartbeat probing. Specifically, lightweight heartbeat messages are maintained between nodes in the client cluster. Within each detection period (e.g., every 200ms), if the client and the target client support the RDMA protocol, probing is performed using the ib_send_bw command; if only the TCP protocol is supported, probing is performed using ordinary UDP or TCP small packets. This client calculates the current RTT based on the sending and receiving times of the probing messages and takes the average value within the current period as the RTT value for that period; when performing Tpeer estimation, the average RTT value of the most recent period is used.

[0055] The queuing latency of read I / O operations from the target client is periodically collected by the target client and broadcast to this client. This client stores the most recently received queuing latency of read I / O operations from the target client. Specifically, the target client records the read time each time it hits its local cache and stores the recorded read time in a local circular queue (e.g., storing the most recent 100 records). Within each detection period (e.g., every 5 seconds), the target client determines its own queuing latency of read I / O operations based on the average of a preset number of read times stored in the circular queue and broadcasts it to other clients in the distributed file system via heartbeat messages. This client receives and stores the queuing latency broadcast by the target client and uses the most recently received queuing latency of read I / O operations from that target client when performing Tpeer estimation.

[0056] Tost is used to characterize the expected time required to read target data from the object storage target. Tost is determined based on the following parameters: the size of the target data, the current available bandwidth of the object storage target, the request queue latency of the object storage target, and the scheduling latency of the metadata server (MDS) of the distributed file system. Here, the object storage target refers to the object storage target where the target data resides; the MDS of the distributed file system can be determined.

[0057] Among them, the size of the target data is positively correlated with Tost, the current available bandwidth is negatively correlated with Tost, the request queue waiting latency is positively correlated with Tost, and the scheduling latency is positively correlated with Tost.

[0058] Tost is determined by the following formula: Tost=DataSize / BW_ost+Q_ost+S_mds Where DataSize is the size of the target data; BW_ost is the current available bandwidth of the object storage target; Q_ost is the request queue wait time for the object storage target; S_mds represents the scheduling latency of the MDS.

[0059] The current available bandwidth of the object storage target, the request queue waiting latency of the object storage target, and the scheduling latency of the MDS are all parameters obtained through periodic data collection. When performing Tost estimation, the client selects the values ​​of the above parameters collected in the most recent period as the basis.

[0060] The current available bandwidth of the object storage target is obtained by this client through a call to the distributed file system's programming interface. Specifically, this client calls the llapi interface, passing in the stripe layout information of the target file, which returns the real-time load bandwidth value of the corresponding OST. When performing OST estimation, this client uses the OST bandwidth value obtained in the most recent period.

[0061] The request queue waiting latency of the object storage target is obtained by this client by reading the read I / O operation time statistics of the object storage target. Specifically, the read I / O operation time statistics are read I / O operation time histogram statistics. This client calls the llapi interface to read the brw_stats block read and write statistics of the target OST, extracts the average time of read operations from the histogram as the average total time of all read I / O operations, extracts the midpoint of the minimum latency interval as the base time without queuing, and uses the difference between the average total time of all read I / O operations and the base time without queuing as the request queue waiting latency. The average total time of all I / O operations changes dynamically with the cluster's concurrent request volume, so the latency value of each period reflects the current load status in real time. When performing Tost estimation, this client uses the request queue waiting latency obtained in the most recent period.

[0062] The scheduling latency of MDS is periodically collected by MDS and sent to this client, which stores the most recently received MDS scheduling latency. Specifically, MDS obtains the average response time of metadata operations by reading the ` / proc / fs / lustre / mdc / mdcxxx / stats` file, uses this as the MDS scheduling latency, and sends it to each client via heartbeat messages or synchronization messages. This client uses the most recently received MDS scheduling latency when performing Tost estimation.

[0063] S302, if Tpeer is less than the product of Tost and the preset coefficient, then the target data acquisition method is determined to be cross-client acquisition. If Tpeer is greater than or equal to the product of Tost and the preset coefficient, then the target data acquisition method is determined to be non-cross-client acquisition.

[0064] In step S203, the specific implementation of the target data acquisition method based on Tpeer and Tost is as follows: If Tpeer is less than the product of Tost and a preset coefficient, the target data acquisition method is determined to be cross-client acquisition; the preset coefficient is less than 1, used to compensate for the estimation error of Tpeer caused by network jitter. If Tpeer is greater than or equal to the product of Tost and the preset coefficient, the target data acquisition method is determined to be non-cross-client acquisition.

[0065] In this embodiment, a preset coefficient of less than 1 is used as a safety threshold to avoid misjudging scenarios where cross-client mode should not be used due to Tpeer estimation errors. Since Tpeer estimation relies on periodically collected bandwidth and RTT values, these collected values ​​inevitably contain measurement errors and instantaneous fluctuations. When Tpeer is close to Tost, measurement errors may lead to decisions that do not match the actual network conditions. Therefore, setting a preset coefficient of less than 1 is equivalent to requiring Tpeer not only to be less than Tost, but also to be significantly smaller than it to trigger cross-client mode acquisition, thereby avoiding erroneous decisions due to measurement errors and ensuring that read performance is not degraded.

[0066] Optionally, the preset coefficient ranges from 0.8 to 0.95. Within this range, cross-client acquisition can be triggered as much as possible without reducing read performance. Those skilled in the art can select specific coefficient values ​​within the above range based on the actual network environment, the confidence interval of the estimation accuracy, and the service's requirements for jitter sensitivity; no specific limitations are imposed here.

[0067] The above provides a detailed explanation of the specific estimation methods for Tpeer and Tost, as well as the data acquisition methods for decision-making objectives based on Tpeer and Tost.

[0068] As an example, during the process of obtaining target data in a cross-client manner, the client monitors the transmission status of cross-client data transmission. When an abnormal event is detected, the current transmission is interrupted and the client switches to obtaining target data from the object storage target.

[0069] Specifically, the abnormal event includes any of the following situations: (1) Data transmission connection with the target client times out. If the data transmission connection is not successfully established within a set time threshold (e.g., 200ms) after the client initiates a connection request to the target client, it is determined that the connection has timed out and an abnormal rollback is triggered.

[0070] (2) Data transmission interruption or packet loss rate exceeding the preset threshold. During cross-client data transmission, if a data transmission interruption is detected, or the packet loss rate exceeds the preset threshold (e.g., packet loss rate > 1%), the transmission quality is deemed unsatisfactory, triggering an abnormal rollback.

[0071] (3) The actual data transmission rate is lower than the preset percentage of the expected rate. During cross-client data transmission, if the actual transmission rate is lower than the preset percentage of the expected rate (e.g., 50% of the expected rate), the transmission efficiency is deemed not to meet the requirements, and an abnormal rollback is triggered.

[0072] (4) The checksum of the data received from the target client does not match the expected checksum. After receiving the data returned by the target client, the client calculates the checksum of the data and compares it with the checksum recorded in the interval cache descriptor. If the two do not match, it is determined that the data may have been erroneous or corrupted during transmission, triggering an abnormal rollback.

[0073] (5) The target client returns a cache invalidation notification. During cross-client data transmission, if the local cached data of the target client is invalidated due to file modification, expiration, or other reasons, the target client returns a cache invalidation notification to the current client. Upon receiving the notification, the current client triggers an abnormal rollback.

[0074] Once an abnormal fallback is triggered, the client immediately interrupts the data transmission connection with the target client and switches to retrieving the target data from the object storage target. This means it directly initiates a read request to the corresponding object storage target to continue retrieving the target data. This automatic fallback mechanism ensures that when cross-client data transmission anomalies occur, the client can promptly switch to the native read path, guaranteeing the successful completion of the read request and preventing business interruption or severe performance degradation due to cross-client data retrieval failures.

[0075] To illustrate the method provided in this application in more detail, the following will be combined with... Figure 4 The solution provided in this application will be described in more detail by way of specific embodiments.

[0076] In this embodiment, the distributed file system is Lustre as an example. The client cluster includes client A, client B, and client C, all of which mount the same Lustre file system and share object storage targets OST1 and OST2. MDS1 and MDS2 jointly provide metadata services, with the metadata of file F (FID 0x12345) managed by MDS2. Each client is configured with the cross-client cache collaborative access function provided in this embodiment.

[0077] Suppose client A receives a read request from an upper-layer application, which requests to read a data range of 4096 bytes starting from offset 1024 bytes in file F (FID is 0x12345).

[0078] Client A parses the read request and obtains the target file's FID as 0x12345, target offset as 1024, and target length as 4096. Client A first checks if the target data is already stored in its local cache. Since Client A has never read this file before, the target data is not in its local cache, resulting in a local cache miss.

[0079] Client A initiates a remote query process, retrieving range cache descriptions published by other clients. Since no client has previously cached this data, Client A fails to find a range cache description that meets the criteria, confirming that no valid remote data range exists.

[0080] Client A initiates a metadata query to MDS2 via the Lustre native I / O path to obtain the stripe layout information of file F. The stripe layout information returned by MDS2 indicates that the data of file F is distributed on OST1. Based on this layout information, client A establishes a connection with OST1 and reads target data starting at offset 1024 bytes and with a length of 4096 bytes from OST1. After a successful read, client A writes the target data to its local persistent client cache PCC.

[0081] After the data is written, client A establishes a corresponding interval cache descriptor, using a continuous data interval as the smallest management unit. This interval cache descriptor is represented as {FID:0x12345, OST:1, offset:1024, len:4096, valid:1, checksum:0xABCD}, where FID:0x12345 represents the file identifier to which the data belongs, OST:1 indicates that the data is stored on OST1, offset:1024 and len:4096 represent the starting offset and length of the continuous data interval, valid:1 indicates that it is currently valid, and checksum:0xABCD represents the checksum corresponding to this data interval. Client A then synchronizes this interval cache descriptor to other clients in the distributed file system.

[0082] Subsequently, as Figure 4 As shown, client B receives a read request from the upper-layer application, requesting the reading of target data starting at offset 1024 bytes and with a length of 4096 bytes from file F. After parsing the read request, client B first checks if the target data is already stored in its local cache. Since client B has not read this file before, the target data is not in its local cache, resulting in a local cache miss.

[0083] Client B enters the remote query process, retrieving range cache descriptions published by other clients. Client B obtains the range cache description {FID:0x12345, OST:1, offset:1024, len:4096,valid:1, checksum:0xABCD} published by Client A. The validity status information of this range cache description is valid, the file identifier FID:0x12345 matches the file identifier of the target data, and the indicated continuous data range (offset 1024 bytes, length 4096 bytes) completely covers the continuous data range corresponding to the target data. Client B identifies Client A, corresponding to this range cache description, as a candidate client, with a candidate client count of 1. Client A is then identified as the target client, and the remote data range corresponding to the range cache description published by Client A is determined as the valid remote data range.

[0084] Client B estimates the expected time Tpeer for acquiring target data across clients and the expected time Tost for reading target data from object storage. Client B periodically collects the current available bandwidth between itself and Client A, which is 100MB / s, and the network round-trip time (RTT) is 2ms. Client A periodically collects the queuing latency of its local read I / O operations, which is 3ms, and broadcasts it to other clients. Client B stores the most recently received queuing latency of Client A's read I / O operations. Based on the target data size of 4096 bytes, the current available bandwidth of 100MB / s, the RTT of 2ms, and the queuing latency of Client A's read I / O operations of 3ms, Client B determines Tpeer according to the formula Tpeer = DataSize / BW + RTT + Q_peer.

[0085] Client B periodically collects data on OST1: its current available bandwidth is 50MB / s, OST1's request queue latency is 7ms, and MDS2's scheduling latency is 1ms. Based on the target data size of 4096 bytes, OST1's current available bandwidth of 50MB / s, OST1's request queue latency of 7ms, and MDS2's scheduling latency of 1ms, Client B determines Tost using the formula Tost = DataSize / BW_ost + Q_ost + S_mds.

[0086] Client B determines the target data acquisition method based on Tpeer and Tost. Client B multiplies Tost by a preset coefficient (assuming the preset coefficient is 0.85). Since Tpeer is less than the product of Tost and 0.85, Client B determines to acquire the target data using a cross-client approach.

[0087] Client B establishes a data transmission connection with Client A and sends a cross-node read request to Client A. This request includes the file identifier FID: 0x12345, starting offset: 1024, length len: 4096, and the expected checksum 0xABCD. Upon receiving the request, Client A locates the corresponding data in its local PCC based on the description recorded in its local interval cache management module, reads the 4096 bytes of data, and returns it to Client B through the established connection. After receiving the data, Client B calculates the checksum of the data and compares it with the expected checksum 0xABCD. If the checksum passes, the data is delivered to the upper-layer application. Thus, Client B successfully obtains the target data.

[0088] During the aforementioned cross-node transmission process, assuming that client A suddenly fails and causes the network connection to be interrupted, after client B detects the data transmission interruption and the packet loss rate exceeds the preset threshold, it immediately interrupts the data transmission connection with client A and switches to obtaining the target data from the object storage target, that is, it directly sends a read request to OST1 to continue obtaining the target data, ensuring that the read request of the upper layer application can be completed smoothly.

[0089] The methods provided in the embodiments of this application have been described above. The apparatus provided in the embodiments of this application is described below.

[0090] See Figure 5 , Figure 5 This is a structural diagram of the device provided in an embodiment of this application. Figure 5 As shown, the device is applied to the client side of a distributed file system. The device includes: a creation module 501, a retrieval module 502, a decision module 503, and an acquisition module 504.

[0091] Create module 501, which is used to create a range cache description item locally and synchronize it to other clients after the client successfully reads file data through the distributed file system and caches it locally, with a continuous data range as the smallest management unit. The range cache description item does not contain file data, but is used to describe the cache information of file data on the client.

[0092] The retrieval module 502 is used to respond to a data read request. If the target data requested by the data read request is not found in the local cache, it retrieves the range cache description items published by other clients to determine whether there is a valid remote data range that completely covers the data read request.

[0093] The decision module 503 is used to estimate the expected time Tpeer for obtaining target data across clients and the expected time Tost for reading target data from object storage targets, respectively, if a valid remote data range exists; and to decide on the target data acquisition method based on Tpeer and Tost.

[0094] The acquisition module 504 is used to acquire the target data in a cross-client manner if the target data acquisition method is determined to be cross-client; otherwise, the target data is acquired directly from the object storage target if the target data acquisition method is determined to be non-cross-client.

[0095] As an example, when the decision module 503 executes the decision on the target data acquisition method based on Tpeer and Tost, it is specifically used to: if Tpeer is less than the product of Tost and a preset coefficient, then the target data acquisition method is determined to be cross-client acquisition, where the preset coefficient is less than 1 to compensate for the estimation error of Tpeer caused by network jitter; if Tpeer is greater than or equal to the product of Tost and the preset coefficient, then the target data acquisition method is determined to be non-cross-client acquisition.

[0096] As one example, Tpeer is determined based on the size of the target data, the current available bandwidth between the current client and the target client corresponding to the valid remote data range, the network round-trip time (RTT) between the current client and the target client, and the queuing delay of the target client's read I / O operations. Specifically, the size of the target data is positively correlated with Tpeer; the current available bandwidth is negatively correlated with Tpeer; the network round-trip time (RTT) is positively correlated with Tpeer; and the queuing delay of the target client's read I / O operations is positively correlated with Tpeer.

[0097] As one example, Tost is determined based on the size of the target data, the current available bandwidth of the object storage target, the request queue waiting latency of the object storage target, and the scheduling latency of the metadata server (MDS) of the distributed file system. Specifically, the size of the target data is positively correlated with Tost; the current available bandwidth of the object storage target is negatively correlated with Tost; the request queue waiting latency of the object storage target is positively correlated with Tost; and the scheduling latency of the MDS is positively correlated with Tost.

[0098] As an example, the interval cache descriptor includes at least: the identifier of the file to which the file data belongs, the continuous data interval of the file data on the object storage target of the distributed file system, and the validity status information of the file data within the data interval. The validity status information is valid when the interval cache descriptor is created. The creation module 501 is further configured to update the validity status information from valid to invalid and synchronize the updated interval cache descriptor to other clients when the file data is modified or the lifespan of the interval cache descriptor exceeds a preset lifespan.

[0099] As an example, when the retrieval module 502 performs the judgment of whether there is a valid remote data range that completely covers the data reading request, it is specifically used to: search for a range cache description item from the range cache description items published by other clients, where the valid status information is valid, the indicated file identifier is consistent with the file identifier to which the target data belongs, and the indicated continuous data range completely covers the continuous data range corresponding to the target data; and determine the remote data range corresponding to the found range cache description item as a valid remote data range.

[0100] This concludes the process. Figure 5 Structural description of the device shown.

[0101] See Figure 6 , Figure 6 This is a structural diagram of an electronic device provided in an embodiment of this application. Figure 6 As shown, the hardware structure may include: a processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; the processor is used to execute the machine-executable instructions to implement the method disclosed in the above example of this application.

[0102] Based on the same application concept as the above method, this application embodiment also provides a machine-readable storage medium storing a plurality of computer instructions, which, when executed by a processor, can implement the method disclosed in the above examples of this application.

[0103] For example, the aforementioned machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, etc. For instance, machine-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0104] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

[0105] This application can be implemented by executing several computer program code flows using an electronic device. The electronic device loads machine-executable instructions into a machine-readable storage medium and then uses its processor to read these machine-executable instructions into memory for execution. A hardware structure diagram of the electronic device, besides... Figure 5In addition to the processor, memory, network interface, and non-volatile memory shown, electronic devices may also include other hardware depending on their actual functions, which will not be elaborated further.

Claims

1. A performance-aware cross-client cache collaborative access method, characterized in that, This method is applied to the client side of a distributed file system, and the method includes: After a client successfully reads file data through the distributed file system and caches it locally, it establishes a range cache description item locally with a continuous data range as the smallest management unit and synchronizes it to other clients; the range cache description item does not contain the actual file, and the range cache description item is used to describe the cache information of the file data on the client; In response to a data read request, if the target data requested by the data read request is not found in the local cache, the range cache description item published by other clients is retrieved to determine whether there is a valid remote data range that completely covers the data read request. If the valid remote data range exists, estimate the expected time Tpeer for acquiring the target data across clients and the expected time Tost for reading the target data from the object storage target; decide on the target data acquisition method based on Tpeer and Tost. If the decision is made that the target data acquisition method is cross-client acquisition, then the target data shall be acquired in the cross-client manner; If the decision is made that the target data acquisition method is a non-cross-client acquisition method, then the target data is directly obtained from the object storage target.

2. The method according to claim 1, characterized in that, The method for acquiring decision target data based on Tpeer and Tost includes: If the Tpeer is less than the product of the Tost and the preset coefficient, then the target data acquisition method is determined to be cross-client acquisition; the preset coefficient is less than 1, which is used to compensate for the estimation error of the Tpeer caused by network jitter. If the Tpeer is greater than or equal to the product of the Tost and the preset coefficient, the target data acquisition method is determined to be a non-cross-client acquisition method.

3. The method according to claim 1, characterized in that, The Tpeer is determined based on the size of the target data, the current available bandwidth between the current client and the target client corresponding to the effective remote data range, the network round-trip time (RTT) between the current client and the target client, and the queuing delay of the target client's read input / output I / O operations; The size of the target data is positively correlated with the Tpeer. The current available bandwidth between this client and the target client corresponding to the effective remote data range is negatively correlated with the Tpeer. The network round-trip time (RTT) is positively correlated with the Tpeer. The queuing delay is positively correlated with the Tpeer.

4. The method according to claim 1, characterized in that, The Tost is determined based on the size of the target data, the current available bandwidth of the object storage target, the request queue waiting latency of the object storage target, and the scheduling latency of the metadata server MDS of the distributed file system. The size of the target data is positively correlated with the Tost. The current available bandwidth of the object storage target is negatively correlated with Tost; The request queue waiting latency is positively correlated with the Tost. The scheduling delay is positively correlated with the Tost.

5. The method according to claim 1, characterized in that, The interval cache descriptor includes at least: the identifier of the file to which the file data belongs, the continuous data interval of the file data on the object storage target of the distributed file system, and the valid status information of the file data on the data interval; wherein, the valid status information is valid when the interval cache descriptor is created; The method also includes: When the file data is modified or the lifespan of the interval cache description item exceeds the preset lifespan, the valid status information is updated from valid to invalid, and the updated interval cache description item is synchronized to other clients.

6. The method according to claim 5, characterized in that, The determination of whether there exists a valid remote data range that completely covers the data read request includes: From the interval cache description entries published by the other clients, find the interval cache description entries whose valid status information is valid, whose indicated file identifier is consistent with the file identifier to which the target data belongs, and whose indicated continuous data interval completely covers the continuous data interval corresponding to the target data. The remote data range corresponding to the found range cache descriptor is determined as the valid remote data range.

7. A performance-aware cross-client cache collaborative access device, characterized in that, This device is used on any client of a distributed file system, and the device includes: A module is created to establish a range cache description item locally and synchronize it to other clients after a client successfully reads file data through a distributed file system and caches it locally, using a continuous data range as the smallest management unit. The range cache description item does not contain the file data; instead, it describes the cache information of the file data on the client. The retrieval module is used to respond to a data read request. If the target data requested by the data read request is not found in the local cache, it retrieves the range cache description items published by other clients to determine whether there is a valid remote data range that completely covers the data read request. The decision module is used to estimate, if the valid remote data range exists, the expected time Tpeer for acquiring the target data across clients and the expected time Tost for reading the target data from the object storage target; and to decide on the target data acquisition method based on Tpeer and Tost. The acquisition module is used to acquire the target data in a cross-client manner if the target data acquisition method is determined to be cross-client; and to acquire the target data directly from the object storage target if the target data acquisition method is determined to be non-cross-client.

8. The apparatus according to claim 7, characterized in that, The decision-making module is specifically used to: When executing the Tpeer and Tost-based decision target data acquisition method, it is used to: If the Tpeer is less than the product of the Tost and the preset coefficient, then the target data acquisition method is determined to be cross-client acquisition; the preset coefficient is less than 1, which is used to compensate for the estimation error of the Tpeer caused by network jitter. If the Tpeer is greater than or equal to the product of the Tost and the preset coefficient, the target data acquisition method is determined to be a non-cross-client acquisition method. And / or, The Tpeer is determined based on the size of the target data, the current available bandwidth between the current client and the target client corresponding to the effective remote data range, the network round-trip time (RTT) between the current client and the target client, and the queuing delay of the target client's read input / output I / O operations; The size of the target data is positively correlated with the Tpeer. The current available bandwidth between this client and the target client corresponding to the effective remote data range is negatively correlated with the Tpeer. The network round-trip time (RTT) is positively correlated with the Tpeer. The queuing delay is positively correlated with the Tpeer; And / or, The Tost is determined based on the size of the target data, the current available bandwidth of the object storage target, the request queue waiting latency of the object storage target, and the scheduling latency of the metadata server MDS of the distributed file system. The size of the target data is positively correlated with the Tost. The current available bandwidth of the object storage target is negatively correlated with Tost; The request queue waiting latency is positively correlated with the Tost. The scheduling delay is positively correlated with the Tost; And / or, The interval cache descriptor includes at least: the identifier of the file to which the file data belongs, the continuous data interval of the file data on the object storage target of the distributed file system, and the valid status information of the file data on the data interval; wherein, the valid status information is valid when the interval cache descriptor is created; This method of creating modules is also used for: When the file data is modified or the lifespan of the interval cache description item exceeds the preset lifespan, the valid status information is updated from valid to invalid, and the updated interval cache description item is synchronized to other clients. And / or, The retrieval module, in performing the step of determining whether there exists a valid remote data range that completely covers the data reading request, is specifically used for: From the interval cache description entries published by the other clients, find the interval cache description entries whose valid status information is valid, whose indicated file identifier is consistent with the file identifier to which the target data belongs, and whose indicated continuous data interval completely covers the continuous data interval corresponding to the target data. The remote data range corresponding to the found range cache descriptor is determined as the valid remote data range.

9. An electronic device, characterized in that, The electronic device includes: Processor; and A machine-readable storage medium storing machine-executable instructions that, when executed by the processor, cause the processor to perform the steps of the method as described in any one of claims 1 to 6.

10. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores machine-executable instructions that, when executed by a processor, cause the processor to perform the steps of the method as described in any one of claims 1 to 6.

Images