RDMA network batch task processing method and device based on selective signal

Abstract

The invention discloses an RDMA network batch task processing method and device based on selective signals. The method comprises the steps that a driving program initializes a work queue; the application program issues work requests in batches; the driving program sets a work request type for each work request; the driving program submits a work request to the host adapter card; the host adapter card executes the work request and completes a data transmission task according to work request information; the host adapter card selectively reports a completion message; and the driving program processes a completion result of the batch tasks, and after a completion signal is received, state codes of completion queue elements in the completion queue are analyzed and subsequent processing is carried out. According to the RDMA network batch task processing method and device based on the selective signal, the completion signal is only reported to the key task through a selective signal mechanism, so that the completion state analysis and processing overhead caused by the fact that each task reports the completion signal to a driving program and an application program is avoided; therefore, the utilization rate of computing resources of the processor is optimized.

Description

Technical Field

[0001] The present invention belongs to the technical field of network communication, and in particular relates to a method and device for processing RDMA network batch tasks based on selective signals. Background Art

[0002] In Remote Direct Memory Access (RDMA) network communications, there is a widespread need to process batch tasks. For example, in a distributed storage system, data distribution and backup are typical batch task scenarios. File data is stored in shards on multiple nodes. When data needs to be distributed across nodes, it often involves multiple concurrent network transmission requests. In addition, in the collective communication scenario of distributed machine learning training, each training iteration requires parameter synchronization, involving a large number of concurrent data transmission requests. Batch network tasks not only require the system to have high throughput, but also need to be completed within a limited time to avoid affecting subsequent computing tasks.

[0003] In the above scenario of sending network tasks in batches, the existing network transmission model and batch task processing program implementation still have certain limitations. In the existing method, the completion status of each task needs to be reported to the driver by default, and the frequent triggering of completion signals causes excessive processor load. Summary of the invention

[0004] The present invention provides a method and device for processing RDMA network batch tasks based on selective signals to solve the above-mentioned technical problems, and specifically adopts the following technical solutions: A method for processing RDMA network batch tasks based on selective signals comprises the following steps: The driver initializes the work queue and configures the completion status reporting mode of the work queue to the selective signal mode; The application sends work requests in batches, packages multiple tasks to be sent and sends them to the driver; The driver sets a work request type for each work request, and the work request type is divided into whether a completion signal needs to be generated; The driver submits a work request to the host adapter card; The host adapter card executes the work request and completes the data transmission task according to the work request information; The host adapter selectively reports a completion signal, generates a completion queue element when the host adapter completes a work request that requires generation of a completion signal, and triggers a completion signal to the driver; The driver processes the completion results of the batch tasks, and after receiving the completion signal, parses the status code of the completion queue element in the completion queue and performs subsequent processing.

[0005] Furthermore, the driver specifies the number of work requests to be issued by writing a doorbell register.

[0006] Furthermore, the driver program implements a network task dispatching and status reporting mechanism through the cooperation of a work queue and a completion queue, wherein the work queue is used to store information of tasks to be executed, and the completion queue is used to report the completion status of the task.

[0007] Furthermore, the network driver sends the tasks in batches, and sends the same batch of network tasks in a single task processing flow.

[0008] Furthermore, the driver performs refined control on the work completion event reported by the host adapter card through a selective signal mechanism to optimize task batch processing.

[0009] Furthermore, the driver controls whether each work request needs to generate a completion signal through a flag bit.

[0010] Furthermore, the driver sets the completion signal flag bit to an invalid state for an intermediate work request of a batch task; and the driver sets the flag bit to a valid state for a last work request of a batch task.

[0011] Furthermore, the host adapter card selectively reports a completion signal, does not feed back a completion signal for an intermediate work request of a batch task, and reports a completion status of a last work request of the batch task according to a signal flag in the work request.

[0012] Furthermore, the host adapter card notifies the driver of the completion status of the work request through an interrupt or polling mechanism.

[0013] A selective signal-based RDMA network batch task processing device, used to implement the aforementioned selective signal-based RDMA network batch task processing method, comprising: a host memory, a host adapter card and a processor; The host memory stores the application program, driver program, data to be sent by the driver program, work queue and completion queue for processing network batch tasks; The host adapter card unloads the message parsing and sending, task processing and completion signal reporting functions of the RDMA network protocol; The processor is responsible for running the application, the driver, and the interaction logic between software and hardware.

[0014] The benefit of the present invention lies in the RDMA network batch task processing method and device based on selective signals, which only reports completion signals for key tasks through the selective signal mechanism, avoiding the completion status analysis and processing overhead brought to the driver and application by reporting completion signals for each task, thereby optimizing the utilization of processor computing resources.

[0015] The RDMA network batch task processing device based on selective signals provided by the present invention combines a selective signal mechanism, reduces the interaction frequency between the host adapter card and the processor, and the host adapter card can focus on efficiently processing work requests in the work queue, avoiding the hardware interruption and polling overhead caused by each task reporting a completion signal, and can improve the system throughput and task processing efficiency in high concurrency scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0017] Figure 1 A flowchart of a method for processing RDMA network batch tasks based on selective signals provided by the present invention.

[0018] Figure 2 Schematic diagram of a work request field of a work queue in an embodiment of the present invention.

[0019] Figure 3 The figure is a processing flow chart of selectively reporting a completion signal in an embodiment of the present invention.

[0020] Figure 4 Schematic diagram of the structure of selective reporting completion signal in an embodiment of the present invention.

[0021] Figure 5 A schematic diagram of an RDMA network batch task processing device based on selective signals provided by the present invention. DETAILED DESCRIPTION

[0022] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0023] like Figure 1As shown, the following first describes an RDMA network batch task processing method based on selective signals provided by the present invention through an embodiment, which specifically includes: Step S1: the driver initializes the work queue and configures the completion status reporting mode of the queue to be a selective signal mode, that is, the work queue selectively reports the completion status according to the flag bit of the work request.

[0024] The driver implements the network task dispatching and status reporting mechanism through the cooperation of the work queue and the completion queue. The work queue is the core data structure for the communication between the driver and the host adapter card. In the initialization phase, the driver allocates host memory resources for the work queue and the completion queue, where the work queue is used to store the task information to be executed, and the completion queue is used to report the completion status of the task. As a preferred embodiment, the allocated queue elements are aligned to the cache line boundary to optimize the access performance. In addition, in the initialization phase, the driver needs to set the head pointer and tail pointer of the queue to manage the read and write positions of the elements in the queue.

[0025] Step S2: The application sends work requests in batches, packages multiple tasks to be sent and sends them to the driver. The batch sending method can reduce the dispersion of requests, improve the operation efficiency of the queue, and send the same batch of network tasks in a single task processing flow.

[0026] Step S3: The driver sets the work request type, and the work request type is divided into whether a completion signal needs to be generated. Specifically, the driver controls whether each work request needs to generate a completion signal through a flag bit.

[0027] In this embodiment, the driver follows the following rules to set the work request type: for intermediate tasks, the completion signal flag is set to an invalid state to avoid generating unnecessary completion signals; for the last work request of a batch task, the flag is set to a valid state to ensure that the host adapter card generates a completion signal after the batch task is completed.

[0028] In addition, for each work request issued, the driver needs to fill in the specific information of the task. Specifically, the work request mainly contains the task identifier, data source address, target address, data length, operation code, flag bit and other fields, such as Figure 2 The flag field can be used to set whether the work request needs to report a completion signal.

[0029] Step S4: The driver submits a work request to the hardware and specifies the number of work requests to be issued by writing a doorbell register.

[0030] Step S5: the host adapter card executes the work request and completes the data transmission task according to the work request information.

[0031] Specifically, after receiving the doorbell register notification signal in step S4, the host adapter card takes out the work request from the head pointer position of the work queue, parses the work request field set in step S3, and performs the task of sending or receiving network data. Taking the sending operation as an example, the host adapter card first reads the specified source data from the host memory, and encapsulates the message according to the target address information in the work request, and then completes the data packet sending operation through the hardware network interface.

[0032] Step S6: the host adapter card selectively reports a completion signal. When the host adapter card completes a work request with a completion signal flag, it needs to generate a completion queue element and trigger a completion signal to the driver.

[0033] Specifically, the processing flow of the host adapter card selectively reporting the completion signal is as follows: Figure 3 When the host adapter completes a work request with a valid flag, a completion queue element is generated and a completion signal is triggered to the driver. As an optional implementation, in this embodiment, the hardware can notify the driver of the completion status of the work request through an interrupt or polling mechanism.

[0034] In this embodiment, the host adapter card selectively reports the queue structure of the completion signal as follows: Figure 4 As shown in the figure, the intermediate request of the batch task is only used to transmit data, and the completion status has no practical meaning for the application. For such intermediate tasks that do not need to immediately feedback the completion signal, the host adapter card only needs to report the completion status of the last work request of the batch task according to the signal flag in the work request, avoiding the completion status parsing and processing overhead brought to the driver and application by reporting the completion signal of each task, thereby optimizing the utilization of the processor computing resources.

[0035] Step S7: the driver processes the completion result of the batch task, and after receiving the completion signal, parses the status code in the completion queue element and performs subsequent processing.

[0036] Specifically, after the application receives the completion signal of the last task, if the batch of tasks is successfully completed, it will continue to send the subsequent batch tasks; otherwise, it will handle the error or resend the task according to the status code in the completion signal. After completing the work request, the driver also needs to release the work queue resources for subsequent use.

[0037] like Figure 5 As shown, based on the same inventive principle, an embodiment of the present invention also provides an RDMA network batch task processing device based on selective signals, which is used to implement the RDMA network batch task processing method based on selective signals of the above embodiment. Specifically, it includes a host memory, a host adapter card and a processor, and the specific functions of each component are as follows: The host memory is used to store applications and drivers that process network batch tasks, and cache the data to be sent by the driver, the work queue and the completion queue. The work queue stores the information of the work request to be executed, and the completion queue stores the status of the completed tasks and related metadata.

[0038] The host adapter card is the hardware core for batch task processing, which offloads the hardware logic of data packet parsing, encapsulation, and transmission and reception of the RDMA network protocol. This embodiment uses a programmable device as a specific implementation of the host channel adapter card, in which the direct memory access engine can directly read the task information and data in the work queue from the host memory without the intervention of the processor, thereby improving the data transmission efficiency.

[0039] In addition, the host adapter interacts with the driver through registers. When the driver sends a task request to the doorbell register, the host adapter begins to parse the corresponding work request and execute the task. For the execution result of each task, the host adapter generates a completion queue element according to the selective signal completion mechanism and notifies the driver of the completion status of the task through an interrupt or polling mechanism.

[0040] As the core computing unit of the device, the processor is responsible for running applications, drivers, and the interaction logic between software and hardware.

[0041] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any form, and any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A method for processing batch tasks in an RDMA network based on selective signals, characterized in that: The following steps are involved: The driver initializes the work queue and configures the completion status reporting mode of the work queue to the selective signal mode; The application sends work requests in batches, packages multiple tasks to be sent and sends them to the driver; The driver sets a work request type for each work request, and the work request type is divided into whether a completion signal needs to be generated; The driver submits a work request to the host adapter card; The host adapter card executes the work request and completes the data transmission task according to the work request information; The host adapter selectively reports a completion signal, generates a completion queue element when the host adapter completes a work request that requires generation of a completion signal, and triggers a completion signal to the driver; The driver processes the completion results of the batch tasks, and after receiving the completion signal, parses the status code of the completion queue element in the completion queue and performs subsequent processing.

2. According to the method for processing RDMA network batch tasks based on selective signals in claim 1, it is characterized in that: The driver specifies the number of work requests to be issued by writing a doorbell register.

3. According to the method of claim 1, the method is characterized in that: The driver performs a network task dispatching and status reporting mechanism through the cooperation of a work queue and a completion queue, wherein the work queue is used to store task information to be executed, and the completion queue is used to report the completion status of the task.

4. The RDMA network batch task processing method based on selective signal according to claim 1 is characterized in that: The network driver sends the tasks in batches, and sends the same batch of network tasks in a single task processing flow.

5. The RDMA network batch task processing method based on selective signal according to claim 4 is characterized in that: The driver performs refined control on the work completion event reported by the host adapter card through a selective signal mechanism to optimize task batch processing.

6. The RDMA network batch task processing method based on selective signal according to claim 5 is characterized in that: The driver controls whether each work request needs to generate a completion signal through a flag bit.

7. The RDMA network batch task processing method based on selective signal according to claim 6 is characterized in that: The driver sets the completion signal flag bit to an invalid state for an intermediate work request of a batch task; and sets the flag bit to a valid state for a last work request of a batch task.

8. The RDMA network batch task processing method based on selective signal according to claim 7 is characterized in that: The host adapter card selectively reports a completion signal, does not feed back a completion signal for an intermediate work request of a batch task, and reports the completion status of the last work request of the batch task according to the signal flag bit in the work request.

9. The RDMA network batch task processing method based on selective signal according to claim 1 is characterized in that: The host adapter card notifies the driver of the completion status of the work request through an interrupt or polling mechanism.

10. A RDMA network batch task processing device based on selective signals, used to implement the RDMA network batch task processing method based on selective signals according to any one of claims 1 to 9, characterized in that: include: Host memory, host adapter cards and processors; The host memory stores the application program, driver program, data to be sent by the driver program, work queue and completion queue for processing network batch tasks; The host adapter card unloads the message parsing and sending, task processing and completion signal reporting functions of the RDMA network protocol; The processor is responsible for running the application, the driver, and the interaction logic between software and hardware.

Images