A hyper-convergence performance optimization method, device and equipment

Abstract

The present application discloses a method, device and equipment for optimizing hyper-convergence performance, which includes: firstly, the hyper-convergence storage end provides the iser server end to perform large page memory and shared memory, so as to share the memory within the system, and share the memory address in multiple remote direct data access RDMA connections, then, the iser server end uses the memory address to perform direct registration, and when there is an IO request, sends the memory address content and the first key value of reading and writing required for IO to the remote node; and sends the memory address content and the second key value of reading and writing required for IO to the iser client end, and then, the remote node (or iser client end) sends the memory address content and the first key value (or second key value) of reading and writing to its own smart network card, so that the smart network card returns the data to the remote node (or iser client end) for distribution after copying the data at the corresponding memory location. Thus, the hyper-convergence performance can be reasonably optimized, the hyper-convergence delay can be reduced, and the CPU utilization rate can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device and equipment for optimizing hyper-convergence performance. Background Art

[0002] In the era of explosive growth of information, massive data is growing. Traditional storage is costly and inefficient and cannot meet the growth rate of user data. Efficient and intelligent distributed storage technology has emerged.

[0003] Distributed storage has the following characteristics: high performance, high reliability, high scalability, transparency, and autonomy. Distributed storage data storage must first be processed by sharding, and then the data storage location is calculated or found through a certain algorithm or metadata service. Since user data is divided into multiple data blocks, the loss of any data block can cause data unavailability, so distributed storage must consider a reasonable redundant storage model to provide multiple redundant storage copies for user data blocks to ensure data security and reliability. Hyper-convergence forwards copy data through the network internally and provides services externally through the network. Therefore, hyper-convergence cannot be separated from the network throughout the process; regarding network transmission, traditional network transmission requires CPU participation, but as the network speed increases, it will lead to a higher CPU usage rate. Therefore, how to reasonably optimize hyper-convergence performance to accelerate hyper-convergence storage performance and reduce hyper-convergence latency is a problem that needs to be solved urgently. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to provide a hyper-convergence performance optimization method, device and equipment, which can reasonably optimize the hyper-convergence performance, accelerate the hyper-convergence storage performance, reduce the hyper-convergence latency, and thus improve the CPU utilization.

[0005] In a first aspect, an embodiment of the present application provides a method for optimizing hyper-convergence performance, including:

[0006] The hyper-converged storage provides the ISER server with large page memory and shared memory to share the memory within the system and share the memory address among multiple remote direct data access (RDMA) connections.

[0007] The ISER server uses the memory address to register directly, and when there is an IO request, sends the memory address content that requires IO and the first key value for reading and writing to the remote node remote in the hyper-converged structure; and when there is an IO request, sends the memory address content that requires IO and the second key value for reading and writing to the ISER Client in the hyper-converged structure;

[0008] The remote node sends the memory address content and the first key value read and written to its own smart network card, so that the smart network card returns the data to the remote node for distribution after copying the data at the corresponding memory location;

[0009] The iser Client sends the memory address content and the second key value read and written to its own smart network card, so that the smart network card returns the data to the iser Client for distribution after copying the data at the corresponding memory location.

[0010] Optionally, the hyper-converged storage accesses the Client by using the iSER protocol; the iserserver uses large memory pages, lock-free queues, RoCE protocol and polling mechanism to implement hyper-converged full-link protocol offloading.

[0011] Optionally, the large page memory is used to avoid page fault interrupts; the lock-free queue is used to improve the concurrency capability of the hyper-convergence; the ISER protocol offloading and RoCE protocol offloading are used to reduce the latency of network transmission; and the polling mechanism is used to improve event perception.

[0012] Optionally, the iser client and iser server directly access memory through the smart network card; the iser client and iser server send IO through RDMA, and the iser server actively accesses the data address of the iser client and copies it to the locally registered large memory page address through the smart network card.

[0013] In a second aspect, the embodiment of the present application further provides a hyper-convergence performance optimization device, comprising:

[0014] The sharing unit is used to provide the ISER server with large page memory and shared memory on the hyper-converged storage side, so as to share the memory within the system and share the memory address among multiple remote direct data access (RDMA) connections.

[0015] The first sending unit is used for the ISER server to directly register with the memory address, and when there is an IO request, the memory address content requiring IO and the first key value for reading and writing are sent to the remote node remote in the hyper-converged structure; and when there is an IO request, the memory address content requiring IO and the second key value for reading and writing are sent to the ISER Client in the hyper-converged structure;

[0016] A second sending unit is used for the remote node to send the memory address content and the first key value read and written to its own smart network card, so that the smart network card returns the data to the remote node for sending after copying the data at the corresponding memory location;

[0017] The third sending unit is used for the ISER Client to send the memory address content and the second key value read and written to its own smart network card, so that the smart network card returns the data to the ISER Client for sending after copying the data at the corresponding memory location.

[0018] Optionally, the hyper-converged storage accesses the Client by using the iSER protocol; the iserserver uses large memory pages, lock-free queues, RoCE protocol and polling mechanism to implement hyper-converged full-link protocol offloading.

[0019] Optionally, the large page memory is used to avoid page fault interrupts; the lock-free queue is used to improve the concurrency capability of the hyper-convergence; the ISER protocol offloading and RoCE protocol offloading are used to reduce the latency of network transmission; and the polling mechanism is used to improve event perception.

[0020] Optionally, the iser client and iser server directly access memory through the smart network card; the iser client and iser server send IO through RDMA, and the iser server actively accesses the data address of the iser client and copies it to the locally registered large memory page address through the smart network card.

[0021] The embodiment of the present application also provides a hyper-convergence performance optimization device, including: a processor, a memory, and a system bus;

[0022] The processor and the memory are connected via the system bus;

[0023] The memory is used to store one or more programs, and the one or more programs include instructions, which, when executed by the processor, enable the processor to execute any one of the implementation methods of the above-mentioned hyper-convergence performance optimization method.

[0024] An embodiment of the present application also provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device executes any one of the implementation methods of the above-mentioned hyper-convergence performance optimization method.

[0025] The embodiment of the present application provides a hyper-convergence performance optimization method, device and equipment. First, the hyper-convergence storage end provides the ISER server end to perform large page memory and shared memory, so as to share the memory within the system and share the memory address in multiple remote direct data access RDMA connections. Then, the ISER server end uses the memory address for direct registration, and when there is an IO request, sends the memory address content and the first key value of the read and write required by IO to the remote node remote in the hyper-convergence structure; and when there is an IO request, sends the memory address content and the second key value of the read and write required by IO to the ISER Client end in the hyper-convergence structure. Then, the remote node sends the memory address content and the first key value of the read and write to its own smart network card, so that the smart network card returns the data to the remote node for distribution after copying the data at the corresponding memory location. At the same time, the ISER Client end also sends the memory address content and the second key value of the read and write to its own smart network card, so that the smart network card returns the data to the ISER Client end for distribution after copying the data at the corresponding memory location. This will enable reasonable optimization of hyper-convergence performance, accelerate hyper-convergence storage performance, reduce hyper-convergence latency, and thus improve CPU utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0027] Figure 1 A schematic diagram of a process flow of a hyper-convergence performance optimization method provided in an embodiment of the present application;

[0028] Figure 2 A schematic diagram of a physical architecture for hyper-convergence performance optimization provided in an embodiment of the present application;

[0029] Figure 3 An overall schematic diagram of hyper-convergence performance optimization provided in an embodiment of the present application;

[0030] Figure 4 A schematic diagram of the composition of a hyper-convergence performance optimization device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] At present, the copy data is forwarded internally through the network in the hyper-convergence, and the service is provided externally through the network. Therefore, the entire process of hyper-convergence cannot be separated from the network. Regarding network transmission, traditional network transmission requires CPU participation, but as the network speed increases, the CPU usage rate will be higher. Therefore, how to reasonably optimize the performance of hyper-convergence to accelerate the performance of hyper-convergence storage and reduce the latency of hyper-convergence is an urgent problem to be solved.

[0032] To solve the above defects, an embodiment of the present application provides a hyper-convergence performance optimization method. First, the hyper-convergence storage end provides the ISER server end to perform large page memory and shared memory, so as to share the memory within the system and share the memory address in multiple remote direct data access RDMA connections. Then, the ISER server end uses the memory address for direct registration, and when there is an IO request, the memory address content and the first key value of the read and write required by the IO are sent to the remote node remote in the hyper-convergence structure; and when there is an IO request, the memory address content and the second key value of the read and write required by the IO are sent to the ISER Client end in the hyper-convergence structure. Then, the remote node sends the memory address content and the first key value of the read and write to its own smart network card, so that the smart network card copies the data at the corresponding memory location and returns the data to the remote node for sending. At the same time, the ISER Client end also sends the memory address content and the second key value of the read and write to its own smart network card, so that the smart network card copies the data at the corresponding memory location and returns the data to the ISER Client end for sending. This will enable reasonable optimization of hyper-convergence performance, accelerate hyper-convergence storage performance, reduce hyper-convergence latency, and thus improve CPU utilization.

[0033] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0034] First embodiment

[0035] See also Figure 1 , is a flow chart of a hyper-convergence performance optimization method provided in this embodiment, the method comprising the following steps:

[0036] S101: The hyper-converged storage end provides an ISER server for large page memory and shared memory to share the system internally and share the memory address among multiple remote direct data access (RDMA) connections.

[0037] In this embodiment, whether the ISER protocol is unloaded or the RoCE protocol is unloaded, memory registration is actually performed through the smart network card. When data is read, written, sent or received, only the smart network card address is informed, and the smart network card actively copies the data. Therefore, the hyper-converged storage end provides the ISER server for large page memory and shared memory to share the system internally, and share the memory address in multiple remote direct data access RDMA connections. Large memory pages are used here for network card registration, that is, direct memory address access to node internal data is achieved through large page memory sharing, which can avoid page fault interrupts during use and affect the performance of hyper-converged Infrastructure (HCI).

[0038] S102: the iser server uses the memory address to perform direct registration, and when there is an IO request, sends the memory address content requiring IO and the first key value for reading and writing to the remote node remote in the hyper-converged structure; and when there is an IO request, sends the memory address content requiring IO and the second key value for reading and writing to the iserClient in the hyper-converged structure.

[0039] S103: The remote node sends the memory address content and the first key value read and written to its own smart network card, so that the smart network card returns the data to the remote node for distribution after copying the data at the corresponding memory location.

[0040] S104: the iser client sends the memory address content and the second key value read and written to its own smart network card, so that the smart network card copies the data at the corresponding memory location and then returns the data to the iser client for distribution.

[0041] In this embodiment, if Figure 2 As shown in the figure, when the Iser server makes an IO request, it informs the smart network card of its own memory address. After the remote smart network card prepares the address, the smart network cards copy the data directly to the memory address of the Iser server.

[0042] At the same time, after polling the IO request and data segment content, the Iser server slices the IO and forwards it through the RoCE protocol. A memory address can be registered multiple times in the network card, so the memory address of the Iser server is directly registered here. The address of the protocol forwarding message part uses the private memory address registered in the network card, and the data segment directly uses the data segment address received by the Iser server for copy location forwarding.

[0043] On this basis, after receiving the request, the node where the HCI replica is located directly performs a read write request, directly accesses the memory address registered remotely, copies the replica data to the local memory, and directly sends IO. In addition, after the replica data is written, a write request is used to notify that the replica data has been sent and the replica data has been written successfully. After the remote polling task is completed, the iser client is notified that the request processing is completed.

[0044] It can be seen that in the embodiment of the present application, the hyper-converged storage accesses the Client side by using the iSER protocol; the iser server side uses large memory pages, lock-free queues, RoCE protocol and polling mechanism to implement hyper-converged full-link protocol unloading, wherein large page memory is used to avoid page fault interrupts; lock-free queues are used to improve the concurrency capability of the hyper-convergence; iser protocol unloading and RoCE protocol unloading are used to reduce the latency of network transmission; the polling mechanism is used to improve event perception, thereby improving the CPU utilization of the system.

[0045] Specifically, the iser client and iser server directly access the memory through the smart network card, the iser client and iser server send IO through RDMA driver, the iser server actively accesses the data address of the iser client and copies it to the local registered large memory page address through the smart network card.

[0046] Distributed storage generally saves data through multiple copies to achieve data redundancy and avoid single point failures. At the same time, data is sliced ​​and saved. The ISER server receives the data and slices it. According to the location of different data slices, it informs the smart network card of the remote node to copy the data. After the remote node completes the processing, it writes the result back to the memory address of the ISER server through direct memory access. It directly uses remote memory operations to reduce the use of CPU operations such as send recv operations, reduce CPU participation, and reduce business waiting through polling memory non-blocking waiting.

[0047] In addition, in this embodiment, polling occupies a separate CPU for polling operations, promptly discovers the arrival of network requests and the setting of memory status bits, and directly processes non-time-consuming requests without CPU context switching, thereby achieving timely response and pro-NUMA perception to improve memory access speed.

[0048] Thus, for Figure 2 The hyper-converged performance optimized physical architecture shown in the figure can achieve, for example, Figure 3 The overall process of hyper-convergence performance optimization shown in the figure provides full-link protocol offloading, while ensuring zero copy of data in the IO path, accelerating IO access speed, reducing IO access latency, and improving storage performance in distributed storage virtualization scenarios.

[0049] In summary, the present embodiment provides a method for optimizing hyper-convergence performance. First, the hyper-convergence storage end provides the iser server end to perform large page memory and shared memory, so as to share the memory within the system and share the memory address in multiple remote direct data access RDMA connections. Then, the iser server end uses the memory address for direct registration, and when there is an IO request, sends the memory address content and the first key value of the read and write required by the IO to the remote node remote in the hyper-convergence structure; and when there is an IO request, sends the memory address content and the second key value of the read and write required by the IO to the iser client end in the hyper-convergence structure. Then, the remote node sends the memory address content and the first key value of the read and write to its own smart network card, so that the smart network card copies the data at the corresponding memory location and returns the data to the remote node for sending. At the same time, the iser client end also sends the memory address content and the second key value of the read and write to its own smart network card, so that the smart network card copies the data at the corresponding memory location and returns the data to the iser client end for sending. This will enable reasonable optimization of hyper-convergence performance, accelerate hyper-convergence storage performance, reduce hyper-convergence latency, and thus improve CPU utilization.

[0050] Second embodiment

[0051] This embodiment will introduce a hyper-convergence performance optimization device. For related content, please refer to the above method embodiment.

[0052] See also Figure 4 , is a schematic diagram of the composition of a hyper-convergence performance optimization device provided in this embodiment, the device comprising:

[0053] The sharing unit 401 is used for providing the ISER server with large page memory and shared memory to share the memory within the system and share the memory address among multiple remote direct data access (RDMA) connections.

[0054] The first sending unit 402 is used for the ISER server to directly register with the memory address, and when there is an IO request, send the memory address content that requires IO and the first key value of reading and writing to the remote node remote in the hyper-converged structure; and when there is an IO request, send the memory address content that requires IO and the second key value of reading and writing to the ISER Client in the hyper-converged structure;

[0055] The second sending unit 403 is used for the remote node to send the memory address content and the first key value read and written to its own smart network card, so that the smart network card returns the data to the remote node for sending after copying the data at the corresponding memory location;

[0056] The third sending unit 404 is used for the ISER Client to send the memory address content and the second key value read and written to its own smart network card, so that the smart network card returns the data to the ISER Client for sending after copying the data at the corresponding memory location.

[0057] In one implementation of this embodiment, the hyper-converged storage accesses the client by using the iSER protocol; the iser server uses large memory pages, lock-free queues, RoCE protocol and polling mechanism to implement hyper-converged full-link protocol unloading.

[0058] In one implementation of this embodiment, the large page memory is used to avoid page fault interrupts; the lock-free queue is used to improve the concurrency capability of the hyper-convergence; the ISER protocol offloading and RoCE protocol offloading are used to reduce the latency of network transmission; and the polling mechanism is used to improve event perception.

[0059] In one implementation of this embodiment, the ISER client and ISER server directly access memory through the smart network card; the IO sending of the ISER client and ISER server is driven by RDMA, and the ISER server actively accesses the data address of the ISER client and copies it to the locally registered large memory page address through the smart network card.

[0060] In summary, the present embodiment provides a hyper-converged performance optimization device. First, the hyper-converged storage end provides the iser server end to perform large page memory and shared memory, so as to share the memory within the system and share the memory address in multiple remote direct data access RDMA connections. Then, the iser server end uses the memory address for direct registration, and when there is an IO request, sends the memory address content and the first key value of the read and write required by the IO to the remote node remote in the hyper-converged structure; and when there is an IO request, sends the memory address content and the second key value of the read and write required by the IO to the iser client end in the hyper-converged structure. Then, the remote node sends the memory address content and the first key value of the read and write to its own smart network card, so that the smart network card copies the data at the corresponding memory location and returns the data to the remote node for sending. At the same time, the iser client end also sends the memory address content and the second key value of the read and write to its own smart network card, so that the smart network card copies the data at the corresponding memory location and returns the data to the iser client end for sending. This will enable reasonable optimization of hyper-convergence performance, accelerate hyper-convergence storage performance, reduce hyper-convergence latency, and thus improve CPU utilization.

[0061] Furthermore, the embodiment of the present application also provides a hyper-convergence performance optimization device, including: a processor, a memory, and a system bus;

[0062] The processor and the memory are connected via the system bus;

[0063] The memory is used to store one or more programs, and the one or more programs include instructions. When the instructions are executed by the processor, the processor executes any implementation method of the above-mentioned hyper-convergence performance optimization method.

[0064] Furthermore, an embodiment of the present application also provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device executes any one of the implementation methods of the above-mentioned hyper-convergence performance optimization method.

[0065] It can be known from the description of the above implementation mode that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., including several instructions for enabling a computer device (which can be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0066] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0067] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0068] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A hyper-convergence performance optimization method, characterized in that: include: The hyper-converged storage provides the ISER server with large page memory and shared memory to share the memory within the system and share the memory address among multiple remote direct data access (RDMA) connections. The ISER server uses the memory address to register directly, and when there is an IO request, sends the memory address content that requires IO and the first key value for reading and writing to the remote node remote in the hyper-converged structure; and when there is an IO request, sends the memory address content that requires IO and the second key value for reading and writing to the ISER Client in the hyper-converged structure; The remote node sends the memory address content and the first key value read and written to its own smart network card, so that the smart network card returns the data to the remote node for distribution after copying the data at the corresponding memory location; The ISER Client sends the memory address content and the second key value read and written to its own smart network card, so that the smart network card returns the data to the ISER Client for distribution after copying the data at the corresponding memory location.

2. The method according to claim 1, characterized in that The hyper-converged storage accesses the client by using the iSER protocol; the iser server uses large memory pages, lock-free queues, RoCE protocol and polling mechanism to implement hyper-converged full-link protocol unloading.

3. The method according to claim 2, characterized in that The large page memory is used to avoid page fault interrupts; the lock-free queue is used to improve the concurrency capability of the hyper-convergence; the ISER protocol offload and RoCE protocol offload are used to reduce the latency of network transmission; the polling mechanism is used to improve event perception.

4. The method according to claim 1, characterized in that The ISER client and ISER server directly access the memory through the smart network card; the IO sending of the ISER client and ISER server is driven by RDMA, and the ISER server actively accesses the data address of the ISER client and copies it to the locally registered large memory page address through the smart network card.

5. A hyper-convergence performance optimization device, characterized in that: include: The sharing unit is used to provide the ISER server with large page memory and shared memory on the hyper-converged storage side, so as to share the memory within the system and share the memory address among multiple remote direct data access (RDMA) connections. The first sending unit is used for the ISER server to directly register with the memory address, and when there is an IO request, the memory address content requiring IO and the first key value for reading and writing are sent to the remote node remote in the hyper-converged structure; and when there is an IO request, the memory address content requiring IO and the second key value for reading and writing are sent to the ISER Client in the hyper-converged structure; A second sending unit is used for the remote node to send the memory address content and the first key value read and written to its own smart network card, so that the smart network card returns the data to the remote node for sending after copying the data at the corresponding memory location; The third sending unit is used for the ISER Client to send the memory address content and the second key value read and written to its own smart network card, so that the smart network card returns the data to the ISER Client for sending after copying the data at the corresponding memory location.

6. The device according to claim 5, characterized in that The hyper-converged storage accesses the client by using the iSER protocol; the iser server uses large memory pages, lock-free queues, RoCE protocol and polling mechanism to implement hyper-converged full-link protocol unloading.

7. The device according to claim 6, characterized in that The large page memory is used to avoid page fault interrupts; the lock-free queue is used to improve the concurrency capability of the hyper-convergence; the ISER protocol offload and RoCE protocol offload are used to reduce the latency of network transmission; the polling mechanism is used to improve event perception.

8. The device according to claim 5, characterized in that The ISER client and ISER server directly access the memory through the smart network card; the IO sending of the ISER client and ISER server is driven by RDMA, and the ISER server actively accesses the data address of the ISER client and copies it to the locally registered large memory page address through the smart network card.

9. A hyper-convergence performance optimization device, characterized in that: include: Processor, memory, system bus; The processor and the memory are connected via the system bus; The memory is used to store one or more programs, wherein the one or more programs include instructions, and when the instructions are executed by the processor, the processor executes the method according to any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device executes the method according to any one of claims 1 to 4.

Images