Data transmission method, device, equipment and storage medium based on RDMA
Through traffic perception, dynamic switching of RDMA mode and queue-to-pooling management, the performance degradation of RDMA technology in high connection number and large data transmission is solved, and efficient and reliable data transmission is achieved, adapting to different traffic loads and reducing latency. Memory usage.
Patent Information
- Application Number
- CN202210682464.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-16
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2042-06-16
AI Technical Summary
The unilateral operation performance of RDMA technology at high connections is degraded, and the data transmission speed in UD mode is limited by MTU, especially when large data transmission is low.
Dynamically switch the data transmission mode of RDMA through traffic perception, use unilateral operation of RC mode to process large traffic loads and UD mode to process small traffic loads, realize dynamic adaptation of the mode, and pool queue pair management during switching to reduce the number of times Resource allocation and initialization time.
Improves RDMA data transmission efficiency, reduces performance degradation caused by connections, achieves good reliability, scalability, throughput and low latency, and reduces memory footprint.
Smart Images

Figure CN115174484B_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of computer technology, and specifically relates to a data transmission method, apparatus, device and storage medium based on RDMA. Background Art
[0002] RDMA (Remote Direct Memory Access) technology is widely used in storage devices. RDMA technology allows applications to directly read or write remote memory without kernel intervention and without memory copying, thereby accelerating upper-layer computing performance.
[0003] RDMA technology supports two data transmission modes: RC (Reliable Connection) mode and UD (Unreliable Datagram) mode. RDMA operations can be divided into: 1) unilateral operations (Write / Read / Atomic); 2) bilateral operations similar to traditional Ethernet (Send / Receive). RC mode supports unilateral operations, and UD mode supports bilateral operations.
[0004] However, in RC mode, the performance of unilateral operation will drop sharply if the number of connections is large. In UD mode, the size of data sent in bilateral operation is limited by MTU (Maximum Transmission Unit), so the data transmission speed is slow when the data volume is large. Summary of the invention
[0005] The present application proposes a data transmission method, apparatus, device and storage medium based on RDMA, which dynamically switches different data transmission modes through traffic perception, and realizes processing of traffic loads of different data volumes through unilateral operations in RC mode or bilateral operations in UD mode.
[0006] The first embodiment of the present application proposes a data transmission method based on RDMA, including:
[0007] Detect the data flow of the RDMA communication connection with the client;
[0008] According to the size of the data flow, the single-side operation or the double-side operation of RDMA is dynamically switched to perform data transmission with the client.
[0009] The second aspect of the present application provides a data transmission device based on RDMA, including:
[0010] A deep traffic detection module is used to detect the data traffic of the RDMA communication connection with the client;
[0011] The mode switching module is used to dynamically switch to use the unilateral operation or bilateral operation of RDMA to perform data transmission with the client according to the size of the data flow.
[0012] An embodiment of the third aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.
[0013] An embodiment of the fourth aspect of the present application provides a computer-readable storage medium on which a computer program is stored, and the program is executed by a processor to implement the method described in the first aspect above.
[0014] The technical solution provided in the embodiments of the present application has at least the following technical effects or advantages:
[0015] In an embodiment of the present application, different data transmission modes are dynamically switched through traffic perception, so that traffic loads of different data volumes can be processed separately through unilateral operations in RC mode or bilateral operations in UD mode. Based on the dynamic changes in data traffic, dynamic switching between RC mode and UD mode is performed so that the adopted data transmission mode is adapted to the size of the data traffic, improving the efficiency of data transmission based on RDMA and reducing the performance degradation caused by a large number of connections. The queue pairs in RC mode are also pooled and managed, and queue pairs are directly allocated to the client from the queue pair pool when switching to RC mode, reducing the time for resource allocation and initialization. After migrating from bilateral operations to unilateral operations, the original UD connection does not need to be destroyed. Therefore, the entire process is non-blocking, and the overhead of building an RC connection can be hidden without higher latency. The present application also implements reliable data transmission in UD mode by designing a software protocol. Therefore, the data transmission method of the present application has good reliability, scalability, high throughput, low latency, and occupies less memory.
[0016] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] By reading the detailed description of the preferred embodiment below, various other advantages and benefits will become clear to those of ordinary skill in the art. The accompanying drawings are only used for the purpose of illustrating the preferred embodiment and are not considered to be limitations of the present application. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings.
[0018] In the attached picture:
[0019] Figure 1 A flow chart of a data transmission method based on RDMA provided in an embodiment of the present application is shown;
[0020] Figure 2 A schematic diagram of data transmission based on RDMA provided in an embodiment of the present application is shown;
[0021] Figure 3 A comparison diagram showing the effects of various RDMA-based data transmission methods provided in an embodiment of the present application is shown;
[0022] Figure 4 A schematic diagram of the structure of a data transmission device based on RDMA provided in an embodiment of the present application is shown;
[0023] Figure 5 A schematic diagram of the structure of an electronic device provided by an embodiment of the present application is shown;
[0024] Figure 6 A schematic diagram of a storage medium provided in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0025] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.
[0026] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by technicians in the field to which this application belongs.
[0027] The following describes an RDMA-based data transmission method, apparatus, device, and storage medium proposed according to an embodiment of the present application in conjunction with the accompanying drawings.
[0028] An embodiment of the present application provides a data transmission method based on RDMA, which dynamically switches different data transmission modes through traffic perception, and realizes the processing of traffic loads of different data volumes through unilateral operation or bilateral operation of RDMA. The unilateral operation adopts the RC mode to carry the traffic load with a large amount of data, and the bilateral operation adopts the UD mode to bear the traffic load with a small amount of data. In this way, based on the dynamic changes in data traffic, dynamic switching between the RC mode and the UD mode is performed, so that the adopted data transmission mode is adapted to the size of the data traffic, thereby improving the data transmission efficiency based on RDMA and reducing the performance degradation caused by a large number of connections.
[0029] See also Figure 1 , the method specifically comprises the following steps:
[0030] Step 101: Detect the data flow of the RDMA communication connection with the client.
[0031] The execution subject of the embodiment of the present application can be any device that has an RDMA network card and can support RDMA technology. The device can be a server or a terminal. The embodiment of the present application is specifically described using a server as an example.
[0032] In actual applications, there may be one or more clients that need to perform RDMA data transmission with the server. These clients can be remote applications or remote user operation interfaces that access server program tasks, etc. The server establishes RDMA communication connections with all clients that need to perform data transmission. Among them, the RDMA communication connection between the server and the client refers to the communication connection between the RDMA network card of the server and the RDMA network card of the device where the client is located.
[0033] The server establishes an RDMA communication connection with each client and maintains the connection with each client in UD mode. After the server establishes an RDMA communication connection with the client, it first uses RDMA bilateral operations to transmit data with the client.
[0034] The bilateral operation of RDMA is a datagram-oriented data transmission mode. There is no need to establish a queue pair (QP) for each client. In the UD mode, data transmission between multiple clients can share at least one queue pair, so fewer queue pairs are required.
[0035] The server maintains an RDMA communication connection with each client in UD mode, and detects changes in data flow of the RDMA communication connection corresponding to each client in real time.
[0036] The embodiment of the present application dynamically switches between bilateral operation in UD mode and unilateral operation in RC mode based on the dynamic changes in the data traffic of the client. In UD mode, multiple clients can share the same queue pair, and fewer queue pairs are required. In RC mode, queue pairs cannot be shared, and a separate queue pair needs to be established for the client in order to reduce resource allocation and initial time when switching to unilateral operation and increase the speed of establishing RC connection. The embodiment of the present application also pre-establishes multiple queue pairs, and multiple queue pairs are formed into a queue pair pool. When the data transmission between the client is switched from bilateral operation to unilateral operation, a queue pair can be directly allocated to the client from the queue pair pool, thereby saving the time of creating a queue pair for the client.
[0037] Specifically, the server can determine the size of the storage space occupied by the queue pair pool according to the current memory occupancy. The current memory occupancy includes the occupancy of the SRAM (Static Random-Access Memory) in the RDMA network card and the occupancy of the DRAM (Dynamic Random Access Memory) of the server. In another implementation, a preset size can also be set as the size of the storage space occupied by the queue pair pool.
[0038] Maintain connections with multiple clients in UD mode. Based on the RDMA communication connection with the client, use RDMA bilateral operations to obtain the client's metadata, which includes the remote key and data storage address corresponding to the client. The data storage address is the storage address of the data in the client's memory, and the remote key is used to indicate the operation permissions required to operate the data at the data storage address in the client. The metadata can also include the client's device identifier, which can be the unique identifier of the RDMA network card on the device where the client is located.
[0039] After the server obtains the metadata corresponding to the client, it can create a queue pair corresponding to the client according to the metadata, and add the created queue pair to the queue pair pool.
[0040] For each client that establishes an RDMA communication connection with the server, the server obtains the metadata corresponding to the client in UD mode. The server also detects the current idle rate of the queue pairs in the queue pair pool in real time, which can be the ratio between the number of currently idle queue pairs in the queue pair pool and the maximum number of queue pairs that can be stored in the queue pair pool. If it is detected that the current idle rate is less than the preset ratio, a new queue pair is generated based on the obtained client metadata, and the new queue pair is added to the queue pair pool.
[0041] The preset ratio may be 10%, 15% or 20%, etc. In the process of replenishing the queue pair pool, the number of queue pairs in the queue pair pool may be replenished until it reaches the maximum number that the queue pair pool can store, or the queue pair pool may be replenished until the idle rate exceeds a certain ratio, such as replenishing until the idle rate exceeds 80% or 85%, etc.
[0042] The embodiment of the present application adopts bilateral operation in UD mode to obtain metadata of the client, pre-builds queue pairs required by the client in RC mode based on the metadata, and pools the pre-built multiple queue pairs and stores them in the queue pair pool, so that when the data transmission mode between the client and the client is switched from bilateral operation in UD mode to unilateral operation in RC mode, the queue pair can be directly allocated to the client from the queue pair pool, saving the time of storage resource allocation and initialization when switching to unilateral operation in RC mode, and improving the efficiency of data transmission in unilateral operation in RC mode.
[0043] In addition, the server also detects the usage of queue pairs in the queue pair pool, and replenishes the queue pair pool when the idle rate of queue pairs in the queue pair pool is lower than a certain value, ensuring that the queue pair pool can effectively support unilateral data transmission between the server and multiple clients in RC mode.
[0044] The server dynamically detects the change of the data flow of the client in the above manner and builds a queue pair pool, and then dynamically switches the data transmission mode between the client and the server based on the change of the data flow through the following step 102.
[0045] Step 102: According to the size of the data flow, dynamically switch to use the unilateral operation or bilateral operation of RDMA to perform data transmission with the client.
[0046] After the data flow corresponding to the client is detected through step 101, the detected data flow is compared with the preset threshold. If the data flow is greater than or equal to the preset threshold, the duration of the client's data flow greater than or equal to the preset threshold is recorded. If the duration exceeds the preset time, the data transmission mode with the client is switched from bilateral operation to unilateral operation, and the unilateral operation of RDMA is used to transmit data with the client.
[0047] The preset threshold value may be 500KB, 1MB, 2MB, 5MB, etc., and the preset duration may be 3ms, 5ms, 8ms, 10ms, etc. The embodiment of the present application does not limit the specific values of the preset threshold value and the preset duration. In actual applications, the specific values of the preset threshold value and the preset duration may be set according to requirements.
[0048] When the client's data flow exceeds the preset threshold for a continuous preset time, RC mode is used to perform data transmission with the client using unilateral operation. The maximum amount of data allowed to be transmitted during one transmission in RC mode can be GB-level data. Therefore, when the client's data flow is large, switching to RC mode for unilateral operation for data transmission can speed up the data transmission rate. RC mode can bypass the client and server kernels and does not require additional memory copies, which can save server computing resources and improve the data transmission efficiency between the client and the client.
[0049] When switching from bilateral operation in UD mode to unilateral operation in RC mode, the queue pair corresponding to the client is obtained from the pre-established queue pair pool. Specifically, if the metadata based on which the queue pair is pre-established includes the device identifier corresponding to the client, the queue pair corresponding to the device identifier is obtained from the queue pair pool according to the device identifier of the client, and the queue pair is allocated to the client.
[0050] If the metadata based on which the queue pair is pre-established does not include the device identifier corresponding to the client, the metadata of the client is currently obtained in UD mode, and the queue pair corresponding to the metadata is obtained from the queue pair pool according to the metadata, and the obtained queue pair is assigned to the client.
[0051] After allocating a queue pair to the client, the queue pair is used to store the data transmission request of the client, which may be a write request or a read request. The server processes the data transmission request stored in the queue pair using a unilateral operation of RDMA.
[0052] In the embodiment of the present application, the idle queue pairs in the RC mode are managed as a pool to reduce the resource allocation and initialization time of switching to the unilateral operation in the RC mode. After migrating from the bilateral operation in the UD mode to the unilateral operation in the RC mode, there is no need to destroy the original UD connection with the client. Therefore, the entire processing process is non-blocking, and the overhead of establishing the RC connection can be hidden without higher latency.
[0053] In other embodiments of the present application, if it is detected that the duration of the client's data flow being less than a preset threshold exceeds a preset time, the bilateral operation of RDMA is used to transmit data with the client. If the current data transmission between the client and the client uses the bilateral operation in the UD mode, the bilateral operation continues to be used. If the current data transmission between the client and the client uses the unilateral operation in the RC mode, the unilateral operation is switched to the bilateral operation in the UD mode.
[0054] When the data flow is small, bilateral operation in UD mode is used for data transmission, which will not affect the data transmission speed, but also reduce the number of RC connections and reduce the performance degradation caused by too many RC connections.
[0055] In other embodiments, when it is detected that the data flow of the client changes from being greater than a preset threshold to being less than a preset threshold and lasting for a preset time, the data transmission with the client is switched from a unilateral operation in the RC mode to a bilateral operation in the UD mode. Since multiple clients can share the same queue pair in the UD mode, the client needs a separate queue pair in the RC mode. Therefore, when switching from a unilateral operation in the RC mode to a bilateral operation in the UD mode, the state of the queue pair corresponding to the client in the RC mode can also be modified to an idle state, and the queue pair modified to an idle state is added to the queue pair pool.
[0056] That is, when switching from unilateral operation in RC mode to bilateral operation in UD mode, the queue pair of the client in RC mode is released and put back into the queue pair pool so that it can be allocated to the client again the next time the data traffic of the client changes to greater than the preset threshold.
[0057] In the process of using bilateral operation in UD mode to perform data transmission with the client, based on the RDMA communication connection with the client, a data transmission request from the client is received, and the data transmission request can be a send request (send) or a receive request (receive). The server stores the received data transmission request in a shared queue pair corresponding to the bilateral operation, and processes the data transmission request stored in the shared queue pair using the bilateral operation of RDMA.
[0058] In other embodiments of the present application, in order to achieve the reliability of data transmission in the UD mode, a software-level protocol is designed to enable bilateral operations in the UD mode to achieve reliable data delivery. Based on this protocol, in the mode of bilateral operation, data transmission requests all carry timestamps, which can ensure that the data transmission requests are carried out in an orderly manner during the transmission and processing process to avoid disorder. Moreover, after the server processes the data transmission request using bilateral operations, it transmits a confirmation message to the client. Through this confirmation message, the client confirms the completion of the data transmission request, thereby achieving the reliability of data transmission in the UD mode.
[0059] The data volume of the above confirmation message is zero, that is, the confirmation message does not contain data. The confirmation message is only used to notify the client that the data transmission request processing is completed to ensure reliable data transmission. The data volume of the confirmation message is zero, and the confirmation message can be sent to the client quickly, which improves the speed of confirming reliable data transmission in UD mode.
[0060] In other embodiments, the server registers at least one buffer, caches data received in the bilateral operation mode through the at least one buffer, and adjusts the number of buffers according to the dynamic change of the number of clients communicating using the bilateral operation. When the number of clients communicating using the bilateral operation is large, the number of buffers is increased. When the number of clients communicating using the bilateral operation is small, the number of buffers is reduced. Adjusting the number of buffers according to the change of the number of clients communicating using the bilateral operation can improve memory utilization.
[0061] In an embodiment of the present application, in order to enable the user to intuitively see the dynamic changes in the data traffic of each client, and subjectively select the data communication mode according to the dynamic changes in the data traffic. The embodiment of the present application also displays the dynamic changes in the data traffic of the client through a mode selection interface and displays a selection interface for different RDMA operation modes. Among them, the selection interface displayed in the mode selection interface may include an interface corresponding to unilateral operation in RC mode, an interface corresponding to bilateral operation in UD mode, and an interface corresponding to the method of dynamically switching between unilateral operation and bilateral operation according to data traffic changes provided in the embodiment of the present application.
[0062] The user can see the changes in the data flow of each client through the displayed mode selection interface. If the data flow of a client is small for a long time, the user can use the bilateral operation corresponding to the interface setting of the bilateral operation in the UD mode to perform the data transmission with the client using the bilateral operation. If the data flow of a client is large for a long time, the user can use the unilateral operation corresponding to the interface setting of the RC mode to perform the data transmission with the client using the unilateral operation. If the data flow of a client changes frequently and the data flow size is unstable, the user can use the interface for dynamically switching the data transmission mode based on the data flow size to set the dynamic switching of the data transmission mode based on the changes in the data flow of the client, use the bilateral operation in the UD mode when the data flow of the client is small, and switch to the unilateral operation in the RC mode when the data flow of the client is large.
[0063] The server displays the above mode selection interface and performs real-time detection on each selection interface in the mode selection interface. When the user operates a selection interface, the server receives a selection event triggered by the selection interface. At this time, the RDMA operation mode corresponding to the selection interface is used to transmit data with the corresponding client.
[0064] The above method can enable users to intuitively see the changes in the data flow of each client, and provide users with an interface for selecting a data transmission mode, so that users can independently select a data transmission mode based on the data flow of the client, providing a human-computer interaction method.
[0065] In order to facilitate understanding of the data transmission method provided in the embodiment of the present application, the following is a description with reference to the accompanying drawings. Figure 2 As shown in the figure, a light client refers to a client whose data flow is less than a preset threshold, and a heavy client refers to a client whose data flow is greater than a preset threshold. Figure 2 As shown, the server maintains communication connections with each client in UD mode. If the client is a lightweight client, the data transmission request sent by the client to the server may include a send request or a receive request, and the data transmission requests are stored in a queue pair shared by each client in UD mode. The server uses a buffer to cache data received or generated by data processing in UD mode. Figure 2 It can be seen that the data transmission between the queue pair in UD mode and each client is bidirectional, including the dotted arrow from the client to the queue pair in UD mode and the solid arrow from the queue pair in UD mode to the client. The solid arrow indicates that in UD mode, after the server processes the client's data transmission request, it will return a confirmation message to the client, thereby realizing reliable data transmission in UD mode.
[0066] like Figure 2 As shown, a queue pair pool is also constructed in the server, and the server allocates or releases the size of the storage space occupied by the queue pair pool based on the memory occupancy of the memory. The memory includes SRAM and DRAM.
[0067] The server obtains the metadata corresponding to the client from the client in UD mode, generates the corresponding queue pair in RC mode based on the metadata of the client, and stores the generated queue pair in the queue pair pool to implement pooled management of the queue pairs in RC mode. When it is detected in UD mode that the data traffic of a certain client continues for a preset time period greater than a preset threshold, the client is determined to be a heavy client, and the data transmission mode with the client is switched to a unilateral operation in RC mode. At this time, the queue pair corresponding to the client is obtained from the queue pair pool, and the queue pair is assigned to the heavy client. The data transmission request sent by the heavy client includes a read request or a write request, and these data transmission requests are stored in the queue pair in RC mode corresponding to the heavy client. When the data communication with the heavy client is terminated, or the heavy client is converted into a light client, the queue pair assigned to the heavy client will be initialized and put back into the queue pair pool again.
[0068] from Figure 2As can be seen from the heavy client in the lower right corner, the server has assigned a queue pair in RC mode to the heavy client and established an RC connection with the heavy client. However, the UD connection with the heavy client is still maintained, that is, the UD connection with the client is not disconnected during the switch from UD mode to RC mode, so that the data transmission channel with the client is not disconnected during the entire process, making the entire processing process a non-blocking process, so the overhead of building an RC connection can be hidden without higher latency. In addition, a queue pair in RC mode is created in advance, and the pre-created queue pair can be directly assigned to the heavy client during the switch, saving the time of creating the queue pair and improving the switching speed from UD mode to RC mode.
[0069] The above queue pair pool is elastic and can automatically allocate / release queue pairs based on the trade-off between memory usage and the idle rate of queue pairs. Therefore, the overhead of establishing an RC connection and registering the buffer accessible by RDMA can be reduced to nearly 700 milliseconds to 800 milliseconds, which is tolerable for most cases.
[0070] The data transmission method provided in the embodiment of the present application is applied to one of the classic load imbalance scenarios, such as 90% of the data traffic is related to 10% of the connections. This skewed workload is used to test the three modes of the present application: dynamic switching of data transmission mode, unilateral operation in RC mode, and bilateral operation in UD mode. Please note that in the related art, bilateral operation in UD mode requires at least two send / receive operations to achieve the same reliability as in RC mode. Figure 3 As shown, H-RDMA represents the dynamic switching method of the present application, and RC (uniform) represents the unilateral operation of the RC mode under normal workload scenarios. RC (skewed) represents the unilateral operation of the RC mode under the above-mentioned skewed load imbalance scenario. UD represents the bilateral operation under the UD mode. Compared with the unilateral operation under the RC mode, the dynamic switching method of the present application has matching scalability and similar performance. When the number of connections increases from 7 to 420, the performance of the present application decreases by only 16%, while the performance of the unilateral operation under the RC mode decreases by 87%. The throughput of the present application is 63% to 89% higher than that of the bilateral operation under the UD mode.
[0071] In addition to good reliability, scalability, throughput, and latency, the present application has a low memory footprint because fewer queue pairs are required, and also reduces the latency of data transmission because each data transmission request can be executed immediately without blocking.
[0072] In an embodiment of the present application, different data transmission modes are dynamically switched through traffic perception, so that traffic loads of different data volumes can be processed separately through unilateral operations in RC mode or bilateral operations in UD mode. Based on the dynamic changes in data traffic, dynamic switching between RC mode and UD mode is performed so that the adopted data transmission mode is adapted to the size of the data traffic, improving the efficiency of data transmission based on RDMA and reducing the performance degradation caused by a large number of connections. The queue pairs in RC mode are also pooled and managed, and queue pairs are directly allocated to the client from the queue pair pool when switching to RC mode, reducing the time for resource allocation and initialization. After migrating from bilateral operations to unilateral operations, the original UD connection does not need to be destroyed. Therefore, the entire process is non-blocking, and the overhead of building an RC connection can be hidden without higher latency. The present application also implements reliable data transmission in UD mode by designing a software protocol. Therefore, the data transmission method of the present application has good reliability, scalability, high throughput, low latency, and occupies less memory.
[0073] The embodiment of the present application also provides a data transmission device based on RDMA, which is used to execute the data transmission method based on RDMA provided in any of the above embodiments, such as Figure 4 As shown, the device comprises:
[0074] The flow detection module 201 is used to detect the data flow of the RDMA communication connection with the client;
[0075] The mode switching module 202 is used to dynamically switch to use the unilateral operation or bilateral operation of RDMA to perform data transmission with the client according to the size of the data flow.
[0076] The mode switching module 202 is used to use RDMA unilateral operation to transmit data to the client if the duration of the data flow rate being greater than or equal to the preset threshold exceeds the preset time; if the duration of the data flow rate being less than the preset threshold exceeds the preset time, then use RDMA bilateral operation to transmit data to the client.
[0077] The mode switching module 202 is used to obtain the queue pair corresponding to the client from the pre-established queue pair pool; store the data transmission request of the client through the determined queue pair; and process the data transmission request stored in the queue pair using the unilateral operation of RDMA.
[0078] The device also includes: a queue pair pool management module, which is used to obtain the metadata of the client based on the RDMA communication connection with the client by using the bilateral operation of RDMA, the metadata including the remote key and the data storage address; create a queue pair corresponding to the client according to the metadata; and add the created queue pair to the queue pair pool.
[0079] The queue pair pool management module is also used to change the state of the queue pair corresponding to the client to an idle state when the data transmission between the client and the client is switched to a bilateral operation; and add the queue pair changed to the idle state to the queue pair pool.
[0080] The queue pair pool management module is also used to determine the current idle rate of the queue pairs in the queue pair pool; if the idle rate is less than a preset ratio, a new queue pair is generated; and the new queue pair is added to the queue pair pool.
[0081] The mode switching module 202 is used to receive a data transmission request from the client based on the RDMA communication connection with the client; store the data transmission request in a shared queue pair corresponding to the bilateral operation; and process the data transmission request stored in the shared queue pair using the bilateral operation of RDMA.
[0082] In the mode of bilateral operation, the data transmission request carries a timestamp; the mode switching module 202 is also used to transmit a confirmation message to the client after processing the data transmission request using bilateral operation, and the data volume of the confirmation message is zero.
[0083] The device also includes: a connection establishment module, which is used to establish an RDMA communication connection with a client; and to perform data transmission with the client using a bilateral operation of RDMA.
[0084] The device also includes: a buffer management module, which is used to register at least one buffer, cache data received in the bilateral operation mode through the at least one buffer; and adjust the number of buffers according to the dynamic change of the number of clients communicating using the bilateral operation.
[0085] The device also includes: a display module, which is used to display the dynamic change of the data flow of the client through the mode selection interface and the selection interface of different RDMA operation modes;
[0086] The mode switching module 202 is further configured to, if a selection event triggered by a selection interface is received, adopt the RDMA operation mode corresponding to the selection interface to perform data transmission with the client.
[0087] The RDMA-based data transmission device provided in the above-mentioned embodiment of the present application and the RDMA-based data transmission method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.
[0088] The present application also provides an electronic device to perform the above RDMA-based data transmission method. Figure 5 , which shows a schematic diagram of an electronic device provided by some embodiments of the present application. Figure 5As shown, the electronic device 4 includes: a processor 400, a memory 401, a bus 402 and a communication interface 403, wherein the processor 400, the communication interface 403 and the memory 401 are connected via the bus 402; the memory 401 stores a computer program that can be run on the processor 400, and when the processor 400 runs the computer program, the RDMA-based data transmission method provided in any of the aforementioned embodiments of the present application is executed.
[0089] The memory 401 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the device network element and at least one other network element is realized through at least one communication interface 403 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.
[0090] The bus 402 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 401 is used to store programs, and the processor 400 executes the program after receiving an execution instruction. The RDMA-based data transmission method disclosed in any implementation of the aforementioned embodiment of the present application may be applied to the processor 400, or implemented by the processor 400.
[0091] The processor 400 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 400. The above processor 400 can be a general-purpose processor, including a processor (Central Processing Unit, referred to as CPU), a network processor (Network Processor, referred to as NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a readily available programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 401, and the processor 400 reads the information in the memory 401 and completes the steps of the above method in combination with its hardware.
[0092] The electronic device provided in the embodiment of the present application and the RDMA-based data transmission method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented therein.
[0093] The present application also provides a computer-readable storage medium corresponding to the RDMA-based data transmission method provided in the above embodiment. Figure 6 The computer-readable storage medium shown is a CD 30 on which a computer program (ie, a program product) is stored. When the computer program is run by a processor, the RDMA-based data transmission method provided by any of the aforementioned embodiments will be executed.
[0094] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.
[0095] The computer-readable storage medium provided in the above-mentioned embodiments of the present application and the RDMA-based data transmission method provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.
[0096] It should be noted that:
[0097] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known structures and technologies are not shown in detail so as not to obscure the understanding of this description.
[0098] Similarly, it should be understood that in order to streamline the present application and help understand one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, the various features of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be interpreted as reflecting the following schematic diagram: the claimed application requires more features than the features clearly stated in each claim. More specifically, as reflected in the claims below, the inventive aspects are less than all the features of the single embodiment disclosed above. Therefore, the claims following the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present application.
[0099] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present application and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.
[0100] The above is only a preferred specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.
Claims
1. A data transmission method based on RDMA, It is characterized in that The method comprises: Detect the data flow of the RDMA communication connection with the client; According to the size of the data flow, dynamically switch to use the unilateral operation or bilateral operation of RDMA to perform data transmission with the client; in the case of the unilateral operation, each of the multiple clients has a separate queue pair, and in the case of the bilateral operation, the multiple clients share the same queue pair; When the data transmission with the client is switched from the unilateral operation to the bilateral operation, the state of the queue pair corresponding to the client is modified to an idle state; and the queue pair modified to the idle state is added to the queue pair pool.
2. The method according to claim 1, It is characterized in that The dynamically switching the use of unilateral operation or bilateral operation of RDMA to perform data transmission with the client according to the size of the data flow includes: If the duration of the data flow being greater than or equal to the preset threshold exceeds a preset time length, a unilateral operation of RDMA is used to transmit data with the client; If the duration that the data flow rate is less than the preset threshold value exceeds a preset time length, a bilateral operation of RDMA is used to perform data transmission with the client.
3. The method according to claim 2, It is characterized in that The unilateral operation using RDMA to perform data transmission with the client includes: Obtaining a queue pair corresponding to the client from a pre-established queue pair pool; storing the data transmission request of the client by the determined queue pair; The data transmission request stored in the queue pair is processed by using a unilateral operation of RDMA.
4. The method according to claim 3, It is characterized in that Before determining the queue pair corresponding to the client from the pre-established queue pair pool, the method further includes: Based on the RDMA communication connection with the client, using the bilateral operation of RDMA to obtain the metadata of the client, the metadata including the remote key and the data storage address; Creating a queue pair corresponding to the client according to the metadata; The created queue pair is added to the queue pair pool.
5. The method according to any one of claims 3 to 4, It is characterized in that The method further comprises: Determining a current idle rate of queue pairs in the queue pair pool; If the idle rate is less than a preset ratio, generating a new queue pair; The new queue pair is added to the queue pair pool.
6. The method according to claim 2, It is characterized in that The bilateral operation using RDMA to perform data transmission with the client includes: Based on the RDMA communication connection with the client, receiving a data transmission request from the client; storing the data transmission request in a shared queue pair corresponding to the bilateral operation; The data transmission request stored in the shared queue pair is processed by using the bilateral operation of RDMA.
7. The method according to claim 6, It is characterized in that The method further comprises: In a mode adopting bilateral operation, the data transmission request carries a timestamp; After the data transmission request is processed by bilateral operation, a confirmation message is transmitted to the client, and the data volume of the confirmation message is zero.
8. The method according to any one of claims 1 to 4, It is characterized in that Before detecting the data flow of the RDMA communication connection with the client, the method further includes: Establish an RDMA communication connection with the client; The bilateral operation of RDMA is used to transfer data with the client.
9. The method according to any one of claims 1 to 4, It is characterized in that The method further comprises: registering at least one buffer, and caching data received in the bilateral operation mode by the at least one buffer; and, The number of buffers is adjusted according to the dynamic change of the number of clients communicating by using bilateral operation.
10. The method according to any one of claims 1 to 4, It is characterized in that The method further comprises: Displaying the dynamic change of the data flow of the client through the mode selection interface and displaying the selection interface of different RDMA operation modes; If a selection event triggered by the selection interface is received, the RDMA operation mode corresponding to the selection interface is adopted to perform data transmission with the client.
11. A data transmission device based on RDMA, It is characterized in that The device comprises: A traffic detection module is used to detect the data traffic of the RDMA communication connection with the client; A mode switching module, used for dynamically switching to use unilateral operation or bilateral operation of RDMA to perform data transmission with the client according to the size of the data flow; in the case of the unilateral operation, each of the multiple clients has a separate queue pair, and in the case of the bilateral operation, the multiple clients share the same queue pair; When the data transmission with the client is switched from the unilateral operation to the bilateral operation, the state of the queue pair corresponding to the client is modified to an idle state; and the queue pair modified to the idle state is added to the queue pair pool.
12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that The processor runs the computer program to implement the method described in any one of 1-10.
13. A computer-readable storage medium having a computer program stored thereon, It is characterized in that The program is executed by a processor to implement any one of the methods described in 1-10.
