A distributed storage time delay optimization method, device and medium

Abstract

The application discloses a distributed storage time delay optimization method and device and medium, and relates to the field of distributed systems. The method is applied to a distributed cluster, data sent by a client is received, a log and a transaction are constructed in a master OSD according to the data, the master OSD is selected in advance in all OSDs of a PG, each OSD contains a shared memory, the log and the transaction are stored in the shared memory of the master OSD, and other OSDs in the PG acquire the log and the transaction from the shared memory of the master OSD through RDMA. The above scheme realizes the shared memory between the OSD set of the PG in the distributed cluster, and uses RDMA to replace the traditional TCP / IP mode to perform the synchronization logic of the data between the master and the replica of the cluster, thereby effectively optimizing the network time delay of data synchronization and improving the data processing capacity of the whole cluster.

Description

Technical Field

[0001] This application relates to the field of distributed systems, and in particular to a method, apparatus and medium for optimizing distributed storage latency. Background Technology

[0002] In a distributed storage cluster, multiple redundancy strategies are employed to ensure data consistency. When a client transmits data with the cluster, it first needs to establish a network connection with the cluster's primary object storage device (OSD). When a client writes data to the cluster, it first constructs a log (PG_Log) and a transaction in the primary OSD, writes it to its local disk, and sends a message (containing PG_Log and Transaction) to the replicas (other OSDs). The primary OSD then waits until all replicas' I / O operations have been written to disk before returning a successful write response to the client via a registered callback function. When reading data, only the primary OSD of the OSD set containing the object provides the service; that is, only data is read from the primary OSD. Data transmission between primary and replicas is conducted through the cluster's cluster network. Typically, the cluster network is built using 10 Gigabit fiber optic cables and interconnects all nodes in the cluster via 10 Gigabit switches.

[0003] However, using callback functions for message synchronization is inefficient. Furthermore, considering that data transmission between the cluster's primary and replicas occurs over a 10 Gigabit network, significant data transmission latency can occur under heavy network load, impacting upper-layer applications. The callback mechanism further exacerbates this burden.

[0004] In view of the above, designing a distributed storage latency optimization method to improve the message synchronization efficiency of distributed clusters is an urgent problem to be solved by technical personnel in this field. Summary of the Invention

[0005] The purpose of this application is to provide a distributed storage latency optimization method, apparatus, and medium to solve the problem of poor efficiency and performance of message synchronization using callback functions in the distributed cluster message data synchronization process.

[0006] To address the aforementioned technical problems, this application provides a distributed storage latency optimization method, applied to a distributed cluster, comprising:

[0007] Receive data sent by the client;

[0008] Logs and transactions are constructed in the primary OSD based on the data; wherein the primary OSD is pre-selected from all OSDs of the PG, and each OSD contains shared memory.

[0009] The logs and transactions are stored in the shared memory of the master OSD so that other OSDs in the PG can retrieve the logs and transactions from the shared memory of the master OSD via RDMA.

[0010] Preferably, the process of selecting the master OSD from all the OSDs of the PG includes:

[0011] The CRUSH algorithm is used to generate a set containing a preset number of OSDs in the PG;

[0012] Obtain the first OSD in the set as the master OSD.

[0013] Preferably, the other OSDs obtaining the logs and transactions from the shared memory of the master OSD via RDMA includes:

[0014] The RDMA receives the LocalKey sent by the master OSD; wherein the LocalKey is pre-generated by the RDMA and represents the operation permission for the shared memory of the master OSD;

[0015] The logs and transactions in the shared memory of the main OSD are obtained based on the LocalKey.

[0016] Preferably, the process of generating the LocalKey includes:

[0017] The shared memory is registered to the RNIC via the RDMA, and the returned LocalKey is obtained.

[0018] Preferably, each data node in the distributed cluster is configured with a network interface card that supports RoCE and interconnected through a switch that supports RDMA.

[0019] Preferably, after the other OSDs retrieve the logs and transactions from the shared memory of the master OSD via RDMA, the method further includes:

[0020] The system returns a message indicating that the data was successfully written to disk to the client.

[0021] Preferably, the LocalKey includes the address range, length, and requested address range of the shared memory.

[0022] To address the aforementioned technical problems, this application also provides a distributed storage latency optimization device, applied to a distributed cluster, comprising:

[0023] The receiving module is used to receive data sent by the client;

[0024] A construction module is used to construct logs and transactions in the master OSD based on the data; wherein the master OSD is pre-selected from all OSDs of the PG, and each OSD contains shared memory.

[0025] A storage module is used to store the logs and transactions in the shared memory of the master OSD, so that other OSDs in the PG can retrieve the logs and transactions from the shared memory of the master OSD via RDMA.

[0026] To address the aforementioned technical problems, this application also provides a distributed storage latency optimization device, comprising:

[0027] Memory, used to store computer programs;

[0028] A processor is used to implement the steps of the distributed storage latency optimization method described above when executing the computer program.

[0029] To address the aforementioned technical problems, this application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the distributed storage latency optimization method described above.

[0030] The distributed storage latency optimization method provided in this application is applied to a distributed cluster. It involves receiving data sent by a client; constructing logs and transactions in the master OSD based on the data; wherein the master OSD is pre-selected from all OSDs of the PG, and each OSD contains shared memory; the logs and transactions are stored in the shared memory of the master OSD, so that other OSDs in the PG can retrieve the logs and transactions from the shared memory of the master OSD via RDMA. Therefore, the above scheme effectively optimizes the network latency of data synchronization, frees up central processing unit computing power, and improves the overall data processing capability of the cluster by implementing shared memory among the OSD sets of the PG in the distributed cluster and using RDMA to replace the traditional TCP / IP mode for data synchronization logic between cluster master replicas.

[0031] In addition, this application also provides a distributed storage latency optimization device and a computer-readable storage medium, which have the same effect as above. Attached Figure Description

[0032] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 A flowchart illustrating a distributed storage latency optimization method provided in this application embodiment;

[0034] Figure 2 A flowchart illustrating another distributed storage latency optimization method provided in this application embodiment;

[0035] Figure 3 A schematic diagram of a distributed storage latency optimization device provided in an embodiment of this application;

[0036] Figure 4 This is a schematic diagram of another distributed storage latency optimization device provided in an embodiment of this application. Detailed Implementation

[0037] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0038] The core of this application is to provide a method, apparatus, and medium for optimizing distributed storage latency.

[0039] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0040] Currently, when a client transmits data to a distributed cluster, it first needs to establish a network connection with the cluster's OSDs. When a client writes data to the cluster, it first constructs the PG_Log and Transaction in the primary OSD, writes them to the local disk, and sends a message (containing PG_Log and Transaction) to the replicas. The primary OSD then waits for all replicas' I / O to be written to disk before returning a successful data write to the client via a registered callback function. When reading data, only the primary OSD of the OSD set containing the object provides services; that is, only data is read from the primary OSD. Data transmission between primary and replicas is conducted through the cluster's Cluster network. Typically, Cluster networks are built using 10 Gigabit fiber optic cables and interconnected with 10 Gigabit switches across all nodes in the cluster. However, using a callback function message mechanism for message synchronization is inefficient. Furthermore, considering that data transmission between primary and replicas is via a 10 Gigabit network, significant data transmission latency may occur under high network load, impacting upper-layer applications; the callback mechanism further exacerbates this burden. Therefore, this application provides a distributed storage latency optimization method. Figure 1 This is a flowchart illustrating a distributed storage latency optimization method provided in an embodiment of this application. The method is applied to distributed clusters, such as... Figure 1 As shown, the method includes:

[0041] S10: Receive data sent by the client.

[0042] S11: Construct logs and transactions in the primary OSD based on the data; the primary OSD is pre-selected from all OSDs in the PG, and each OSD contains shared memory.

[0043] S12: Store logs and transactions in the shared memory of the primary OSD so that other OSDs in the PG can retrieve logs and transactions from the shared memory of the primary OSD via RDMA.

[0044] In a distributed cluster, there are tens of thousands of objects. Managing data directly at the object level is too expensive and inflexible. A Placement Group (PG) is a collection of objects that form a group and are placed on certain OSDs. Managing objects at the PG level has the following advantages: The number of PGs in the cluster is planned and strictly controllable, allowing for precise control over the resource consumption of individual OSDs and even the entire node, such as the Central Processing Unit (CPU), memory, and network bandwidth. Because the number of PGs in the cluster is much smaller than the number of objects, and the number of PGs and the identity of each PG are relatively fixed, data backup strategies, data synchronization, and migration at the PG level are easier and more flexible than managing data directly at the object level. Therefore, in this embodiment, after receiving data sent by the client, the distributed cluster constructs logs and transactions in the primary OSD under the PG.

[0045] Specifically, the log and transaction are PG_Log and Transaction. After receiving data from the client, the cluster constructs PG_Log and Transaction using PrimaryLogPG::execute_ctx->prepare_transaction(). It's important to note that PG_Log and Transaction are constructed within the primary OSD under the PG. The primary OSD is pre-selected from all OSDs in the PG. This embodiment does not restrict the selection method of the primary OSD; it depends on the specific implementation. In this application, primary replica data synchronization refers to the data synchronization between the primary OSD in the PG and other OSDs in the PG.

[0046] Furthermore, each OSD in the PG contains shared memory; that is, the primary OSD and other OSDs in the PG each have their own shared memory for data synchronization. Specifically, the primary OSD stores PG_Log and Transaction in its own shared memory so that other OSDs in the PG can retrieve PG_Log and Transaction from the primary OSD's shared memory. When the primary OSD receives a data retrieval request from another OSD, it calls PrimaryLogPG::issue_repop->submit_transaction() to distribute the data to the other OSD. In this embodiment, the specific process by which other OSDs retrieve data from the primary OSD's shared memory is not limited and depends on the specific implementation.

[0047] It is important to note that in this embodiment, other OSDs in the PG obtain PG_Log and Transaction from the shared memory of the main OSD via RDMA. Remote Direct Memory Access (RDMA) was developed to address the latency of server-side data processing during network transmission. RDMA directly transfers data to the computer's memory over the network, quickly moving data from one system to the memory of a remote system without affecting the operating system, thus minimizing the need for computer processing power. It eliminates the overhead of external memory copying and context switching, thereby freeing up memory bandwidth and CPU cycles to improve application system performance. The advantage of RDMA technology over traditional network architectures is its ability to quickly access the remote host's memory through zero-copy and kernel bypass techniques. Zero-copy means that applications can perform data transmission without involving the network protocol stack; kernel bypass means that applications can directly execute data transmission in user mode without context switching between kernel and user modes. Therefore, there are no frequent context switches between kernel and user modes during data transmission, and no CPU involvement is required.

[0048] In this embodiment, the distributed storage latency optimization method is applied to a distributed cluster. It receives data sent by a client and constructs logs and transactions in the master OSD based on this data. The master OSD is pre-selected from all OSDs in the PG, and each OSD contains shared memory. The logs and transactions are stored in the shared memory of the master OSD, so that other OSDs in the PG can retrieve the logs and transactions from the shared memory of the master OSD via RDMA. Therefore, the above scheme effectively optimizes network latency for data synchronization, frees up CPU computing power, and improves the overall data processing capability of the cluster by implementing shared memory among the OSD sets of the PG in the distributed cluster and using RDMA to replace the traditional TCP / IP mode for data synchronization logic between cluster master and replicas.

[0049] Based on the above embodiments:

[0050] As a preferred embodiment, the process of selecting the master OSD from all OSDs of the PG includes:

[0051] The CRUSH algorithm is used to generate a set containing a predetermined number of OSDs in the PG;

[0052] Get the first OSD in the set and use it as the master OSD.

[0053] In the above embodiments, the method for selecting the primary OSD is not limited and depends on the specific implementation. As a preferred embodiment, this embodiment uses the CRUSH algorithm to generate a set containing a preset number of OSDs in the PG; the first OSD in the set is then selected as the primary OSD.

[0054] CRUSH is a controlled-replication distributed hash algorithm, a method used in distributed clusters to control data distribution, efficiently and stably distributing data across ordinary structured clusters. It is a pseudo-random algorithm; under the same environment, similar inputs yield uncorrelated results, while identical inputs yield deterministic results. It only requires a cluster description map and some rules to derive a list of devices storing data from an integer input. Clients may execute the CRUSH algorithm when performing input / output operations.

[0055] The backend of the distributed cluster is an object storage system (RADOS), so all data is partitioned into objects of a specific size. Each object has an Objectid (oid), named according to the block_name_prefix of the image containing the data, followed by a sequentially increasing number. Each object can be assigned a corresponding pgid using (poolid, hash(oid) & mask). Based on the pgid, the client executes the CRUSH algorithm to obtain a set of OSDs (OSD1, OSD2, OSD3). These are then filtered, and OSDs meeting certain requirements (e.g., OSDs cannot be failed or overloaded) are selected based on their replication factor. After determining which OSDs the data will reside on, the client initiates an I / O request to the first OSD in the list (the primary OSD). This OSD becomes the primary OSD, and read / write requests are processed accordingly.

[0056] In this embodiment, a set of OSDs containing a preset number is generated using the CRUSH algorithm. The first OSD in the set is selected as the master OSD. The master OSD establishes a connection with the data sent by the client to facilitate data synchronization between the master and replica.

[0057] Based on the above embodiments:

[0058] As a preferred embodiment, other OSDs retrieve logs and transactions from the shared memory of the primary OSD via RDMA, including:

[0059] The LocalKey sent by the master OSD is received via RDMA; the LocalKey is pre-generated by RDMA and represents the operation permissions for the shared memory of the master OSD.

[0060] Retrieve logs and transactions in the shared memory of the primary OSD based on the LocalKey.

[0061] In the above embodiments, the specific process by which other OSDs obtain data from the shared memory of the main OSD is not limited, and depends on the specific implementation. As a preferred embodiment, the specific process in this embodiment is as follows: receiving the LocalKey sent by the main OSD via RDMA, and obtaining the logs and transactions in the shared memory of the main OSD based on the LocalKey.

[0062] After the primary OSD constructs the PG_Log and Transaction data, it sends the LocalKey to other replicas (i.e., other OSDs) via RDMA. The LocalKey represents the permission granted by RDMA to operate on the shared memory; therefore, only when another OSD possesses the primary OSD's LocalKey can it access the primary OSD's shared memory. Since only the LocalKey information is sent, it has the advantages of small data volume and high transmission efficiency. Once other replicas receive the primary OSD's shared memory LocalKey, they will proactively pull data from the primary OSD's shared memory to their local machines, completing data synchronization. This embodiment does not restrict the LocalKey generation process; it depends on the specific implementation.

[0063] In this embodiment, the LocalKey sent by the master OSD is received via RDMA; wherein, the LocalKey is pre-generated by RDMA and represents the operation permissions on the shared memory of the master OSD; the logs and transactions in the shared memory of the master OSD are obtained according to the LocalKey, thereby realizing the data synchronization of the master replica.

[0064] Based on the above embodiments:

[0065] As a preferred embodiment, the LocalKey generation process includes:

[0066] Register the shared memory with the RNIC via RDMA and obtain the returned LocalKey.

[0067] The generation process of LocalKey in the above embodiments is not limited and depends on the specific implementation. As a preferred embodiment, in a specific implementation, the shared memory of the created master OSD is registered to the RDMA RNIC through the RDMA's `struct ibv_mr*ibv_reg_mr()` interface, and the returned LocalKey for RDMA operations is obtained. Here, RNIC is a dedicated network interface card for RDMA.

[0068] In this embodiment, the shared memory is registered to the RNIC via RDMA, and the returned LocalKey is obtained, thus realizing the generation of the LocalKey so that other OSDs can access the shared memory of the master OSD through the LocalKey.

[0069] Based on the above embodiments:

[0070] In a preferred embodiment, each data node in the distributed cluster is configured with a network interface card that supports RoCE and interconnected through a switch that supports RDMA.

[0071] As can be seen from the above embodiments, since distributed storage latency optimization is applied to distributed clusters that support RDMA, each data node in the distributed cluster needs to be configured with a network card that supports RoCE and interconnected through a switch that supports RDMA.

[0072] RoCE (RDMA over Converged Ethernet) is an Ethernet-based RDMA technology that supports corresponding verbs interfaces. RoCE is a network protocol defined in the InfiniBand Trade Association (IBTA) standard, allowing the use of RDMA over Ethernet networks. In short, it can be seen as the application of RDMA technology in hyperconverged data centers, cloud, storage, and virtualization environments. To implement RoCE, a RoCE-enabled network interface card (NIC) or card driver needs to be installed, and RoCE-enabled NICs need to be interconnected with RDMA-enabled switches.

[0073] The advantages of RoCE include: low CPU utilization, accessing the memory of remote switches or servers without consuming CPU cycles on the remote server, thus fully utilizing available bandwidth and greater scalability; zero replication, sending and receiving data to and from remote buffers; high efficiency, as RoCE significantly improves network performance due to its improved latency and throughput; and cost savings, as RoCE allows for the processing of large amounts of data without purchasing new equipment or replacing Ethernet infrastructure, thereby greatly saving companies' capital expenditures.

[0074] In this embodiment, each data node of the distributed cluster is configured with a network card that supports RoCE, and interconnected through a switch that supports RDMA, thus realizing the construction of RDMA.

[0075] Figure 2 A flowchart illustrating another distributed storage latency optimization method provided in this application embodiment. Figure 2 As shown, after other OSDs retrieve logs and transactions from the shared memory of the master OSD via RDMA, the following steps are also included:

[0076] S13: Returns a message to the client indicating that the data was successfully written to disk.

[0077] As a preferred embodiment, after other OSDs retrieve logs and transactions from the shared memory of the primary OSD via RDMA, i.e. after the primary replica performs data synchronization, it indicates that the log has been successfully written to disk, and returns a message indicating successful data write to disk to the client to prompt the client to proceed to the next stage of operation.

[0078] Based on the above embodiments:

[0079] In a preferred embodiment, LocalKey includes the address range, length, and requested address range of the shared memory.

[0080] Understandably, the LocalKey, used for accessing shared memory on an OSD, needs to contain detailed information about the OSD, specifically the address range, length, and requested address range of the shared memory. This allows other OSDs to locate the shared memory of the primary OSD using the LocalKey and retrieve the data within it.

[0081] In the above embodiments, the distributed storage latency optimization method has been described in detail. This application also provides embodiments corresponding to the distributed storage latency optimization device. It should be noted that this application describes the embodiments of the device part from two perspectives: one is based on the functional modules, and the other is based on the hardware structure.

[0082] Figure 3 This is a schematic diagram of a distributed storage latency optimization device provided in an embodiment of this application. Figure 3 As shown, the device is applied to a distributed cluster and includes:

[0083] Receiver module 10 is used to receive data sent by the client;

[0084] Module 11 is used to construct logs and transactions in the master OSD based on the data; wherein the master OSD is pre-selected from all OSDs of the PG, and each OSD contains shared memory.

[0085] Storage module 12 is used to store logs and transactions in the shared memory of the master OSD so that other OSDs in the PG can retrieve logs and transactions from the shared memory of the master OSD via RDMA.

[0086] The distributed storage latency optimization device provided in this embodiment is applied to a distributed cluster and includes a receiving module, a construction module, and a storage module. It receives data sent by a client; constructs logs and transactions in the master OSD based on the data; wherein the master OSD is pre-selected from all OSDs of the PG, and each OSD contains shared memory; the logs and transactions are stored in the shared memory of the master OSD, so that other OSDs in the PG can retrieve the logs and transactions from the shared memory of the master OSD via RDMA. Therefore, the above solution effectively optimizes the network latency of data synchronization, frees up central processing unit computing power, and improves the data processing capability of the entire cluster by implementing shared memory among the OSD sets of the PG in the distributed cluster and using RDMA to replace the traditional TCP / IP mode for data synchronization logic between cluster master replicas.

[0087] Figure 4 A schematic diagram of another distributed storage latency optimization device provided in this application embodiment is shown below. Figure 4 As shown, the distributed storage latency optimization device includes:

[0088] Memory 20 is used to store computer programs;

[0089] The processor 21 is configured to implement the steps of the distributed storage latency optimization method mentioned in the above embodiments when executing a computer program.

[0090] The distributed storage latency optimization device provided in this embodiment may include, but is not limited to, smartphones, tablets, laptops, or desktop computers.

[0091] The processor 21 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 21 may be implemented using at least one of the following hardware forms: Digital Signal Processor (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 21 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 21 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 21 may also include an Artificial Intelligence (AI) processor, which handles computational operations related to machine learning.

[0092] The memory 20 may include one or more computer-readable storage media, which may be non-transitory. The memory 20 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 20 is used to store at least the following computer program 201, which, after being loaded and executed by the processor 21, is capable of implementing the relevant steps of the distributed storage latency optimization method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, and the storage method may be temporary or permanent storage. The operating system 202 may include Windows, Unix, Linux, etc. The data 203 may include, but is not limited to, the data involved in the distributed storage latency optimization method.

[0093] In some embodiments, the distributed storage latency optimization device may further include a display screen 22, an input / output interface 23, a communication interface 24, a power supply 25, and a communication bus 26.

[0094] Those skilled in the art will understand that Figure 4 The structure shown does not constitute a limitation on the distributed storage latency optimization device and may include more or fewer components than shown.

[0095] The distributed storage latency optimization device provided in this embodiment includes a memory and a processor. The processor executes a computer program to implement the steps of the distributed storage latency optimization method mentioned in the above embodiment. This involves receiving data sent by a client; constructing logs and transactions in the master OSD based on the data; wherein the master OSD is pre-selected from all OSDs in the PG, and each OSD contains shared memory; storing the logs and transactions in the shared memory of the master OSD, so that other OSDs in the PG can retrieve the logs and transactions from the shared memory of the master OSD via RDMA. Therefore, the above scheme effectively optimizes the network latency of data synchronization, releases the computing power of the central processing unit, and improves the data processing capability of the entire cluster by implementing shared memory among the OSD sets of the PG in the distributed cluster and using RDMA to replace the traditional TCP / IP mode for data synchronization logic between the cluster master and replicas.

[0096] Finally, this application also provides an embodiment corresponding to a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps described in the above method embodiments.

[0097] It is understood that if the methods in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0098] The computer-readable storage medium provided in this embodiment stores a computer program, which, when executed by a processor, implements the steps described in the above method embodiment. This involves receiving data sent by a client; constructing logs and transactions in the master OSD based on the data; wherein the master OSD is pre-selected from all OSDs of the PG, and each OSD contains shared memory; storing the logs and transactions in the shared memory of the master OSD, so that other OSDs in the PG can retrieve the logs and transactions from the shared memory of the master OSD via RDMA. Therefore, the above scheme effectively optimizes network latency for data synchronization, frees up central processing unit computing power, and improves the overall data processing capability of the cluster by implementing shared memory among the OSD sets of the PG in a distributed cluster and using RDMA to replace the traditional TCP / IP mode for data synchronization logic between cluster master replicas.

[0099] The foregoing provides a detailed description of a distributed storage latency optimization method, apparatus, and medium provided in this application. The various embodiments in the specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

[0100] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A distributed storage latency optimization method, characterized in that, Applied to distributed clusters, including: Receive data sent by the client; Logs and transactions are constructed in the primary OSD based on the data; wherein, the primary OSD is pre-selected from all OSDs of the PG, and each OSD contains shared memory; the PG is a collection of objects; The logs and transactions are stored in the shared memory of the master OSD so that other OSDs in the PG can retrieve the logs and transactions from the shared memory of the master OSD via RDMA; The other OSDs obtaining the logs and transactions from the shared memory of the main OSD via RDMA includes: The RDMA receives the LocalKey sent by the master OSD; wherein the LocalKey is pre-generated by the RDMA and represents the operation permission for the shared memory of the master OSD; after the master OSD has constructed the log and the transaction data, it sends the LocalKey to the other OSDs via RDMA; The logs and transactions in the shared memory of the main OSD are obtained based on the LocalKey; The process of generating the LocalKey includes: registering the shared memory to the RNIC through the RDMA, and obtaining the returned LocalKey; The LocalKey includes the address range, length, and requested address range of the shared memory.

2. The distributed storage latency optimization method according to claim 1, characterized in that, The process of selecting the primary OSD from all the OSDs of the PG includes: The CRUSH algorithm is used to generate a set containing a predetermined number of OSDs in the PG; the PG is a collection of objects. Obtain the first OSD in the set as the master OSD.

3. The distributed storage latency optimization method according to claim 1, characterized in that, Each data node in the distributed cluster is configured with a network interface card that supports RoCE and interconnected through a switch that supports RDMA.

4. The distributed storage latency optimization method according to claim 1, characterized in that, After the other OSDs retrieve the logs and transactions from the shared memory of the master OSD via RDMA, the process further includes: The system returns a message indicating that the data was successfully written to disk to the client.

5. A distributed storage latency optimization device, characterized in that, Applied to distributed clusters, including: The receiving module is used to receive data sent by the client; A construction module is used to construct logs and transactions in the master OSD based on the data; wherein, the master OSD is pre-selected from all OSDs of the PG, and each OSD contains shared memory; the PG is a collection of objects; A storage module is used to store the logs and transactions in the shared memory of the master OSD, so that other OSDs in the PG can retrieve the logs and transactions from the shared memory of the master OSD via RDMA; The other OSDs obtaining the logs and transactions from the shared memory of the main OSD via RDMA includes: The RDMA receives the LocalKey sent by the master OSD; wherein the LocalKey is pre-generated by the RDMA and represents the operation permission for the shared memory of the master OSD; after the master OSD has constructed the log and the transaction data, it sends the LocalKey to the other OSDs via RDMA; The logs and transactions in the shared memory of the main OSD are obtained based on the LocalKey; The process of generating the LocalKey includes: registering the shared memory to the RNIC through the RDMA, and obtaining the returned LocalKey; The LocalKey includes the address range, length, and requested address range of the shared memory.

6. A distributed storage latency optimization device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the distributed storage latency optimization method as described in any one of claims 1 to 4 when executing the computer program.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the distributed storage latency optimization method as described in any one of claims 1 to 4.

Images