Distributed storage method based on Raft algorithm and RDMA network

Abstract

The invention discloses a distributed storage method based on a Raft algorithm and an RDMA network. The method comprises the steps that a distributed storage cluster is set up, and all nodes in the distributed storage cluster are connected based on an RDMA adapter to form an RDMA network, so that data are directly transmitted among memories of all the nodes; in an initial state, all nodes are followers when being started, each follower is provided with a random election timeout, the followers can be converted into candidates after the election timeout, then the candidates are added with tentative numbers and send voting request messages, and the candidates with votes more than a first set threshold value are changed into leaders; a leader is used for receiving a request of a client, adding the request as a log entry to a log of the leader, and then synchronizing the request to other followers, so that the logs of all nodes are consistent. According to the invention, the CPU occupancy rate during data synchronization can be reduced from 100% to 5%, and the network delay can be reduced from ms level to 10us level.

Description

Technical Field

[0001] The present invention relates to the technical field of data storage, and in particular to a distributed storage method based on a Raft algorithm and an RDMA network. Background Art

[0002] In banks and other financial institutions, the main data storage method currently used is centralized storage. However, this method has many disadvantages, including single point failure risk, poor scalability, performance bottlenecks, high costs, and high requirements for the computer room environment. The following is an analysis of these disadvantages one by one:

[0003] 1. Single point failure risk:

[0004] Centralized storage concentrates all data in a single location. Once a failure occurs, the entire system may be unavailable. There is usually only one copy of the data, and multiple copies cannot be achieved for redundancy, resulting in low fault tolerance.

[0005] 2. Poor scalability

[0006] Centralized storage is difficult to adapt to large-scale storage needs and is difficult to expand. At the same time, adding storage resources or upgrading technology requires shutting down or significantly adjusting the existing system, affecting business continuity.

[0007] 3. Performance bottleneck

[0008] Since the network bandwidth is shared, the performance will drop significantly when multiple users or applications access it at the same time. In addition, the number of input and output operations (IOPS) of the centralized storage system is limited, making it difficult to handle high-concurrency tasks.

[0009] 4. High cost

[0010] Centralized storage devices usually use expensive dedicated hardware with high maintenance costs and high requirements for the computer room environment, such as large space, load-bearing and air-conditioning facilities, which increases operating costs.

[0011] 5. High requirements for the computer room environment

[0012] Centralized storage requires the computer room to have sufficient space, load-bearing capacity and an efficient air conditioning system. In addition, large centralized storage equipment consumes a lot of energy, which increases operating costs and may have an adverse impact on the environment.

[0013] 6. Poor data sharing

[0014] Some centralized storage methods (such as SAN storage) cannot effectively support multi-user file sharing, which affects collaboration efficiency, limits data sharing and collaboration between different departments or branches, and reduces data utilization and value.

[0015] 7. Recovery and backup are complicated

[0016] The backup process of centralized storage is complex and error-prone, requiring high skills of administrators. In addition, in the event of a major failure or disaster, the recovery capability of centralized storage is poor, which may result in the loss of important data.

[0017] 8. Difficult to manage and maintain

[0018] Centralized storage management involves complex configuration and monitoring, which increases the difficulty of management. Since all data is stored in a centralized location, any maintenance may require service suspension, affecting user experience.

[0019] In summary, the disadvantages of centralized storage are particularly prominent in areas such as banking that require high reliability and scalability. These disadvantages not only increase operating costs, but may also affect business continuity and data security at critical moments. Therefore, when considering data storage solutions, it is necessary to weigh the pros and cons of centralized storage and make the best choice based on specific business needs. Summary of the invention

[0020] The purpose of the present invention is to provide a distributed storage method based on Raft algorithm and RDMA network in view of the shortcomings of the prior art.

[0021] To achieve the above object, the present invention provides a distributed storage method based on the Raft algorithm and the RDMA network, comprising:

[0022] Building a distributed storage cluster, where each node in the distributed storage cluster is connected to form an RDMA network based on an RDMA adapter, so that data can be directly transmitted between the memories of each node without the intervention of the kernel of the operating system;

[0023] Log replication in the distributed storage cluster is managed based on the Raft algorithm to ensure data consistency between nodes. The details are as follows:

[0024] In the initial state, all nodes are followers at startup. Each follower sets a random election timeout. After the election timeout, the follower will become a candidate. Then the candidate increases the term number and sends a request for voting message. The candidate who obtains the first set number of votes above the threshold becomes the leader.

[0025] The leader receives requests from clients and appends the requests as log entries to its own log, which is then synchronized to other followers to make the logs of all nodes consistent.

[0026] Furthermore, the RDMA network adopts a bilateral mode to transmit data.

[0027] Furthermore, during the synchronization process, if the follower does not receive the leader's heartbeat or log replication request within the election timeout, it will become a candidate to participate in the election. During the election process, the candidate increases the current term number and then sends a voting request message to other nodes. The candidate who obtains votes above the first set threshold becomes the new leader.

[0028] Furthermore, when the leader synchronizes its log to other followers, it copies the new log entry and the index of the previous log entry to other followers. The followers check whether the previous log entry matches. If it matches, they append the new log entry and then send a confirmation message to the leader. When the leader receives a confirmation message sent by a follower above the second set threshold, it marks the log entry as submitted.

[0029] Furthermore, it also includes: periodically compressing logs through a snapshot mechanism to reduce storage and network transmission overhead.

[0030] Furthermore, the first set threshold and the second set threshold are both 50%.

[0031] Beneficial effects: The present invention combines the consensus algorithm with RDMA network technology, which can not only solve the problems faced by centralized storage such as single point failure, poor scalability, high pressure on data privacy and security protection, but also greatly improve the efficiency and performance of data storage; this combination solution makes the distributed storage system more adaptable to the strict requirements of data security, reliability and high performance in the banking and other fields, and provides strong support for the digital transformation of the financial industry; during data synchronization, the CPU occupancy rate is reduced from 100% to 5%, and the network delay is reduced from ms to 10us. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a schematic diagram of RDMA network interaction according to an embodiment of the present invention;

[0033] Figure 2 Schematic diagram of data transmission mode of RDMA network in an embodiment of the present invention;

[0034] Figure 3 It is a schematic diagram of the leader election process;

[0035] Figure 4 This is a schematic diagram of the log replication process. DETAILED DESCRIPTION

[0036] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. The embodiments are implemented based on the technical solution of the present invention. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0037] like Figures 1 to 4As shown, an embodiment of the present invention provides a distributed storage method based on the Raft algorithm and the RDMA network, including:

[0038] Build a distributed storage cluster. Each node in the distributed storage cluster is connected based on the RDMA adapter to form an RDMA network, so that data can be directly transferred between the memory of each node without the intervention of the operating system kernel. The principle of the RDMA (Remote Direct Memory Access) network is based on direct memory access technology, which allows data to be directly transferred between the memory of two computers without the intervention of the operating system kernel. Figure 1 As shown in the figure, this process is implemented through dedicated hardware on the network adapter (such as the RDMA engine), which bypasses the complex processing of the TCP / IP protocol stack and performs data transmission directly below the transport layer. The sender registers the data buffer in the memory of the RDMA adapter and sends the descriptor to the receiver. The receiver also registers the receive buffer and informs the adapter address. The data is then transmitted directly from the sender to the receiver through the RDMA adapter, achieving efficient and low-latency data transmission. RDMA networks are also divided into unilateral mode and bilateral mode. These two modes differ in the control of data transmission and the data transmission process. As Figure 2 As shown, in the unilateral mode, the sender has full control and can directly write data from its own memory to the receiver's memory without the receiver's participation, which is suitable for efficient scenarios such as batch data transmission. In the bilateral mode, both the sender and the receiver participate in the control of data transmission and transmit data through their respective buffers, which requires coordination and cooperation between the two parties. It is suitable for complex communication scenarios that require real-time interaction and confirmation. In view of the information security considerations within the industry, the present invention ultimately chooses the bilateral mode to transmit data in actual implementation.

[0039] Log replication in the distributed storage cluster is managed based on the Raft algorithm to ensure data consistency between nodes. The details are as follows:

[0040] In the Raft algorithm, each node in the cluster has one of three roles: leader, follower, and candidate. In the initial state, all nodes are followers at startup, and each follower sets a random election timeout. After the election timeout, all followers will become candidates, and then the candidate will increase the term number (Term) and send a request for voting message. The candidate who obtains more than the first set threshold number of votes becomes the leader. The above first set threshold is preferably 50%.

[0041] The leader receives the client's request and appends the client's request as a log entry to its own log, and then synchronizes it to other followers to make the logs of all nodes consistent. Followers do not send any requests themselves, but only respond to requests from leaders or candidates.

[0042] During the synchronization process, if the follower does not receive the leader's heartbeat or log replication request within the election timeout, it will become a candidate to participate in the election. During the election process, the candidate increases the current term number and then sends a voting request message to other nodes. The candidate who obtains votes above the set threshold becomes the new leader.

[0043] When the leader synchronizes its log to other followers, it copies the new log entry and the index of the previous log entry to other followers. The followers check whether the previous log entry matches. If it matches, they append the new log entry and then send a confirmation message to the leader. When the leader receives a confirmation message from a follower above the second set threshold, it marks the log entry as submitted. The above second set threshold is also preferably 50%. The log replication process is as follows: Figure 4 shown.

[0044] Finally, the Raft algorithm ensures the consistency of logs and the security of the system through multiple mechanisms, including election security, log matching, leader integrity, and state consistency, etc. The logs are compressed regularly through the snapshot mechanism to reduce storage and network transmission overhead.

[0045] In summary, the present invention effectively solves the problem of single point failure through collaboration and verification between distributed nodes. In a distributed storage system, multiple nodes jointly participate in the storage, verification and update of data. Even if some nodes fail, the system can maintain the stability of the overall operation and the integrity of the data through a consensus mechanism. This decentralized design not only enhances the reliability and fault tolerance of the system, but also greatly improves the security and privacy protection capabilities of the data. The introduction of RDMA network technology has brought a qualitative leap in the efficiency and performance of data storage. RDMA bypasses the operating system kernel and directly transmits data under the user state and network transmission layer, greatly reducing the delay of data transmission and the load of the CPU. In a distributed storage system, RDMA technology makes data exchange between nodes faster and more efficient, without the need for complex protocol processing and multiple data copies, thereby significantly improving the throughput and response speed of the system. After testing, the CPU occupancy rate dropped from 100% to 5% during data synchronization, and the network delay dropped from ms to 10us.

[0046] The above is only a preferred embodiment of the present invention. It should be noted that for ordinary technicians in this technical field, other parts not specifically described belong to the prior art or common knowledge. Several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered as the protection scope of the present invention.

Claims

1. A distributed storage method based on Raft algorithm and RDMA network, characterized in that: include: Building a distributed storage cluster, where each node in the distributed storage cluster is connected to form an RDMA network based on an RDMA adapter, so that data can be directly transmitted between the memories of each node without the intervention of the kernel of the operating system; Log replication in the distributed storage cluster is managed based on the Raft algorithm to ensure data consistency between nodes. The details are as follows: In the initial state, all nodes are followers at startup. Each follower sets a random election timeout. After the election timeout, the follower will become a candidate. Then the candidate increases the term number and sends a request for voting message. The candidate who obtains the first set number of votes above the threshold becomes the leader. The leader receives requests from clients and appends the requests as log entries to its own log, which is then synchronized to other followers to make the logs of all nodes consistent.

2. According to claim 1, a distributed storage method based on Raft algorithm and RDMA network is characterized in that: The RDMA network adopts a bilateral mode to transmit data.

3. According to claim 1, a distributed storage method based on Raft algorithm and RDMA network is characterized in that: During the synchronization process, if the follower does not receive the leader's heartbeat or log replication request within the election timeout, it will become a candidate to participate in the election. During the election process, the candidate increases the current term number and then sends a voting request message to other nodes. The candidate who obtains the first set threshold of votes becomes the new leader.

4. According to claim 1, a distributed storage method based on Raft algorithm and RDMA network is characterized in that: When the leader synchronizes its log to other followers, it copies the new log entry and the index of the previous log entry to other followers. The followers check whether the previous log entry matches. If it matches, they append the new log entry and then send a confirmation message to the leader. When the leader receives a confirmation message sent by a follower above the second set threshold, it marks the log entry as submitted.

5. According to claim 1, a distributed storage method based on Raft algorithm and RDMA network is characterized in that: Also includes: Logs are compressed periodically through a snapshot mechanism to reduce storage and network transmission overhead.

6. According to claim 4, a distributed storage method based on Raft algorithm and RDMA network is characterized in that: The first set threshold and the second set threshold are both 50%.

Images