Transfer management device, transfer management method, and transfer management program

Abstract

A host (10) of a physical server (100) comprises a DMA transfer driver (1) that performs DMA transfer from a host DRAM (12) to a device DRAM (51), and a plurality of transfer queues (30) that store requests for transferring data that is to be transferred by the DMA. The DMA transfer driver (1) creates one or more subsets from data in the host DRAM (12) specified by the request in each transfer queue (30), and sequentially transfers, to the device DRAM (51) by the DMA, the selected subset of each of the transfer queues (30).

Description

[Technical Field]

[0001] The present invention relates to a transfer management device, a transfer management method, and a transfer management program. [Background technology]

[0002] Methods of transferring data via a bus within a device are classified into PIO (Programmed Input / Output), in which the CPU relays data access commands, and DMA (Direct Memory Access), in which data is transferred directly between the device and memory without going through the CPU (Non-Patent Document 1). DMA hardware transfer enables high-speed and large-volume data transfer (Non-Patent Document 2). DMA often uses an application (APL) on a server as the transfer source and an accelerator or network device on the same server as the transfer destination.

[0003] The transfer engine selects each piece of data to be transferred that has been accepted as a request and performs a DMA transfer. The transfer engine performs arbitration to prevent the next transfer from being accepted while a certain piece of data is being transferred. This prevents data collisions even when multiple applications are using the same transfer destination at the same time. Non-Patent Document 3 describes the XDMA specification of Xilinx, an example of a transfer engine. The Xilinx driver has multiple channels of queues that store requests to be transferred, and takes out requests from each channel in a round-robin (even) manner and transfers them. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Takaoka Toko, "If you are told it is DMA compatible (1)", [online], [Retrieved November 26, 2021], Internet<URL:https: / / uquest.tktk.co.jp / embedded / learning / lecture15-1.html> [Non-patent document 2] EDN Japan, “What are the benefits of DMA?” [online], [Retrieved November 26, 2021], Internet<URL:https: / / ednjapan.com / edn / articles / 1608 / 18 / news015.html> [Non-patent document 3] XILINX, “DMA / Bridge Subsystem for PCI Express v4.1 Product Guide,” [online], [Retrieved November 26, 2021], Internet <URL:https: / / www.xilinx.com / support / documentation / ip_documentation / xdma / v4_1 / pg195-pcie-dma.pdf> Summary of the Invention [Problem to be solved by the invention]

[0005] FIG. 26 is a diagram showing the configuration of a conventional physical server 100z. Each of the threads 11z issues a request (shown as a shaded rectangle) to the host DRAM 12z for DMA transfer of data. The DMA transfer driver 1z reads each request and transfers them in order to the device DRAM 51z in the device 50z via a PCIe (Peripheral Component Interconnect Express) 59z. An IP core 52z, which is a field programmable gate array (FPGA) in the device 50z, reads and processes each request transferred to the device DRAM 51z.

[0006] FIG. 27 is a diagram showing the configuration of the DMA transfer driver 1z shown in FIG. The DMA transfer driver 1z has an accept queue 20z, a transfer queue 30z, and a transfer engine 40z. The accept queue 20z stores requests issued from each thread 11z as pointers to the storage destination of the data in the host DRAM 12z. The transfer queue 30z stores the pointers of the requests read from the accept queue 20z in order to notify the transfer engine 40z. The transfer engine 40z sequentially reads the requested data from the host DRAM 12z from the pointer in the transfer queue 30z, and transfers the data to the device DRAM 51z.

[0007] When multiple applications (threads 11z) perform DMA transfers to the same destination, the next request will wait while a request is being transferred. For example, the following procedure causes a transfer wait problem. (Step 1) The file transfer application A1 (first thread 11z) makes a request R1 for a file with a large data size to the transfer engine 40z. (Step 2) The transfer engine 40z starts transferring request R1 because there are currently no other requests to be transferred. (Step 3) The call application A2 (second thread 11z) requests the transfer engine 40z to issue a high-priority call data request R2 with a short turnaround time (TAT) requirement. (Step 4) The transfer engine 40z puts the transfer of request R2 on hold until the transfer of request R1 is completed. (Step 5) After completing the transfer of request R1, the transfer engine 40z completes the transfer of request R2, but fails to meet the TAT requirement due to the transfer wait.

[0008] We are also considering an extension that would allow file transfer application A1 and call application A2 to link together in advance and control the transfer order in advance. However, although this extension would allow high-priority requests to be sent first, it would require the creation of an application, which would increase development costs, so we would like to avoid adopting it.

[0009] Therefore, the main object of the present invention is to achieve a good balance between the transfer efficiency and delay requirements of the data to be transferred when transferring data from multiple transfer sources to the same transfer destination. [Means for solving the problem]

[0010] In order to solve the above problems, the transfer management device of the present invention has the following features. The present invention provides a transfer management device having a host unit having a host memory in which data from a transfer source is stored, and a device unit having a device memory to which the data is transferred, the host unit has a transfer control unit that performs DMA transfer from the host memory to the device memory, and a plurality of transfer queues that store requests indicating transfer requests that specify data to be DMA transferred, The plurality of transfer queues are assigned individual priorities and store requests of those priorities; The transfer control unit Multiple In the transfer queue each Creating one or more subsets of the data in said host memory specified by the request. In the process of creating a subset from the request data, the higher the priority of the request, the larger the size of the subset. and then select the Multiple In the transfer queue each The subset is transferred to the device memory by DMA. [Effects of the Invention]

[0011] According to the present invention, when data is transferred from a plurality of transfer sources to the same transfer destination, it is possible to achieve a good balance between the transfer efficiency of the data to be transferred and the delay requirement. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 2 is a configuration diagram of a physical server according to the present embodiment. [Figure 2] FIG. 2 is a configuration diagram of the DMA transfer driver of FIG. 1 according to the present embodiment. [Figure 3] This is a time chart showing a case where a high-priority request is kept waiting when the conventional DMA transfer driver shown in FIG. 26 is used. [Figure 4] 10 is a time chart showing a case where the waiting time of a high-priority request is reduced by dividing the data of the request when the DMA transfer driver of the present embodiment shown in FIG. 1 is used. [Figure 5] 5 is a time chart showing a case where the waiting time of a request with a higher priority is further reduced than in the case of FIG. 4 by changing the subset size of the request according to this embodiment. [Figure 6] 10 is a time chart according to the present embodiment when the subset size of the request is large. [Figure 7] 10 is a time chart according to the present embodiment when the subset size of the request is small. [Figure 8] FIG. 2 is a detailed configuration diagram of a DMA transfer driver according to the present embodiment. [Figure 9] 9 is a table showing the state of the transfer process of the DMA transfer driver of FIG. 8 at each time according to the present embodiment. [Figure 10] FIG. 2 is a diagram illustrating the configuration of a queue ID table according to the present embodiment. [Figure 11] FIG. 10 is a diagram illustrating a configuration of a transfer queue management table according to the present embodiment. [Figure 12] FIG. 10 is a diagram illustrating the configuration of a subset size table according to the present embodiment. [Figure 13] FIG. 10 is a diagram illustrating the configuration of a request counter table according to the present embodiment. [Figure 14] FIG. 10 is a diagram illustrating the configuration of a transfer engine table according to the present embodiment. [Figure 15] FIG. 2 is a diagram illustrating a configuration of a priority table according to the present embodiment. [Figure 16]FIG. 2 is a hardware configuration diagram of a physical server according to the present embodiment. [Figure 17] FIG. 10 is a sequence diagram illustrating a request transfer process of a physical server according to the present embodiment. [Figure 18] 17 is a sequence diagram showing a request transfer process of a physical server. [Figure 19] 10 is a flowchart showing a process in which a request is notified from an accepting queue to a transferring queue according to the present embodiment. [Figure 20] 10 is a flowchart illustrating a process for notifying a subset of requests from a transfer queue to a transfer engine according to an embodiment of the present invention. [Figure 21] 10 is a flowchart showing a process of DMA transfer of a subset from a transfer engine to a device according to the present embodiment. [Figure 22] 10 is a flowchart illustrating a first example of a process for changing a subset size of a request according to the present embodiment. [Figure 23] 23 is a time series graph of the subset sizes of high priority and low priority in the first example of FIG. 22. [Figure 24] 10 is a flowchart illustrating a second example of a process for changing the subset size of a request according to the present embodiment. [Figure 25] 25 is a time series graph of the subset sizes of high priority and low priority in the second example of FIG. 24. [Figure 26] FIG. 1 is a configuration diagram of a conventional physical server. [Figure 27] FIG. 27 is a diagram illustrating the configuration of the DMA transfer driver in FIG. 26. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

[0014] FIG. 1 is a diagram showing the configuration of a physical server 100. As shown in FIG. The physical server (transfer management device) 100 has a host (host section) 10 having a host DRAM (host memory) 12 in which the source data is stored, and a device 50 (device section) having a device DRAM (device memory) 51 to which the data is transferred. The thread 11 is, for example, a network application that uses an accelerator device 50 to speed up data processing, and operates in accordance with the vRAN (virtual radio access network) standard.

[0015] Each of the multiple threads 11 issues a request, which is a request for transferring data, to the host DRAM 12. The DMA transfer driver (transfer control unit) 1 reads the data of each request and transfers it in order to the device DRAM 51 in the device 50 via PCIe 59. There is a difference between the internal configuration of the DMA transfer driver 1 in FIG. 26 and FIG. 1 (see FIG. 8 for details). The IP core 52, which is an FPGA in the device 50, reads and processes the data of each request transferred to the device DRAM 51.

[0016] FIG. 2 is a diagram showing the configuration of the DMA transfer driver 1 shown in FIG. The DMA transfer driver 1 has an accept queue 20, a transfer queue 30, and a transfer engine 40. The DMA transfer driver 1 creates one or more subsets from the data in the host DRAM 12 specified by the request in each transfer queue 30, and DMA transfers the subsets of each transfer queue 30 selected in turn to the device DRAM 51. The accept queue 20 stores requests issued from each thread 11 as pointers in the host DRAM 12. The transfer queue 30 stores the pointers of requests read from the accept queue 20 in order to notify the transfer engine 40 of the requests.

[0017] 27 shows a configuration in which one transfer queue 30z is provided within one DMA transfer driver 1z, whereas in FIG. 2 a configuration in which multiple transfer queues 30 are provided within one DMA transfer driver 1z. Note that each transfer queue 30 is provided, for example, for each priority of a request, and one or more requests of the same priority are stored in the same transfer queue 30. Note that in this specification, requests with a higher priority number are given priority over other requests in the execution of DMA transfer processing.

[0018] Therefore, each queue in FIG. 2 is expanded from each queue in FIG. 27 as follows: The receiving queue 20 corresponds to a plurality of transfer queues 30, and can allocate a request to any one of the transfer queues 30 in response to an instruction from the transfer queue determination unit 21 in FIG. A plurality of transfer queues 30 are assigned individual priorities and store requests of those priorities. Each transfer queue 30 is expanded so that after enqueuing (inserting into the queue) one request, the request can be dequeued (removed from the queue) in a state in which the request is divided into a plurality of subsets. Therefore, the transfer queue 30 may have separate areas for storing requests before division and for storing the divided subsets.

[0019] The transfer engine 40 is expanded so that it can sequentially read and transfer subsets from a plurality of transfer queues 30 in response to instructions from the transfer queue selector 41 in FIG. In this specification, even if a request to be DMA transferred by the transfer engine 40 is not divided within the transfer queue 30 because it is originally small in size, it will still be referred to as a "subset" of the request (i.e., one divided request).

[0020] The transfer engine 40 reads the requests in the host DRAM 12 in order from the pointers in the transfer queue 30 and transfers the requests to the device DRAM 51 . Each transfer engine 40 will not accept the next request until it has completed transferring the request it has accepted. When a transfer engine 40 enters a ready state where it is not currently transferring, if there is only one transfer queue 30 containing a subset, it will extract the subset from that transfer queue 30 and transfer it.

[0021] On the other hand, if there are multiple transfer queues 30 storing subsets waiting for DMA transfer, the transfer engine 40 takes and transfers the subsets from the transfer queues 30 selected in order using a method such as round robin. The method for selecting the transfer queue 30 may be flexibly changed, such as round robin or weighted round robin, but a logic with a relatively light load is preferable. It is also possible to provide a plurality of transfer engines 40 within one DMA transfer driver 1. Each transfer engine 40 can perform transfer processing in parallel, as long as they do not simultaneously transfer requests to the same transfer destination.

[0022] The following describes the differences in effect between the conventional DMA transfer driver 1z shown in FIG. 26 and the DMA transfer driver 1 of this embodiment shown in FIG. 1, with reference to the time charts of FIGS. Fig. 3 is a time chart showing the case where high-priority requests R2 and R3 are made to wait when using the conventional DMA transfer driver 1z shown in Fig. 26. The horizontal axis of the time chart indicates the time slots (times t0 to t9), and the vertical axis indicates the requests stored in each component of the physical server 100z at each time.

[0023] 3, pointers are issued in the accept queue 20z in the order of three types of requests R1, R2, and R3. Request R1 has a large data size but low priority, while requests R2 and R3 have small data sizes but high priority (strict TAT requirements). At time t2, the pointers of requests R1, R2, and R3 are enqueued in this order from the accept queue 20z into the transfer queue 30z. Between times t3 and t5, the transfer engine 40z reads the request R1 from the transfer queue 30z and DMA-transfers it from the host DRAM 12z to the device DRAM 51z, causing the request R1 to be first stored in the IP core 52z (#1) after time t6.

[0024] At time t6, the transfer engine 40z reads the remaining requests R2 and R3 from the transfer queue 30z and transfers them by DMA. As a result, request R2 arrives at IP core 52z(#1) at time t6, and request R3 arrives at IP core 52z(#2) at time t7. However, due to the long waiting time caused by request R1 from time t3 to time t5, the delay requirements for high-priority requests R2 and R3 could not be met.

[0025] FIG. 4 is a time chart showing a case where the waiting time of high-priority requests R2 and R3 is reduced by dividing the request data when the DMA transfer driver 1 of this embodiment shown in FIG. 1 is used. At time t0, similar to time t1 in FIG. 3, pointers are issued in the receiving queue 20 in the order of three types of requests R1, R2, and R3 (the priority, size, and fill pattern of each request are the same as those in FIG. 3).

[0026] At time t1, the pointers of each request in the accept queue 20 are distributed to two transfer queues 30. The low-priority request R1 is distributed to the low-priority transfer queue 30 (#1), and the high-priority requests R2 and R3 are distributed to the high-priority transfer queue 30 (#2). At this point, the pointers of each request are enqueued in the transfer queue 30, so the size of the request is not directly shown in the time chart.

[0027] The requests in the transfer queue 30 are divided into multiple subsets of the same size before being dequeued. The high-priority requests R2 and R3 are each divided into two, and the low-priority request R1 is divided into 12. Between times t2 and t3, the transfer engine 40 reads the subsets from the two transfer queues 30 in order (round robin) and transfers them from the host DRAM 12 to the device DRAM 51 by DMA.

[0028] As a result, request R2 (=two subsets) arrives at IP core 52(#1) at time t3, and request R3 (=two subsets) arrives at IP core 52(#2) at time t4. On the other hand, the low-priority request R1 (= subset of 12) arrives at the IP core 52 (#1) at time t6, after the arrival of requests R2 and R3. As a result, requests R2 and R3, which entered the DMA transfer driver 1 later at time t0, are transferred between the divided portions of request R1, which entered earlier at time t0, thereby suppressing delays.

[0029] Fig. 5 is a time chart showing a case where the waiting time of a high-priority request is further reduced compared to the case of Fig. 4 by changing the subset size of the request. In Fig. 5, the DMA transfer driver 1 determines the size of the subset by referring to the priority of each request. Times t0 and t1 are the same in FIG. 4 and FIG. From the requests in the transfer queue 30, one or more subsets are generated according to the size corresponding to the priority of each request. Here, the DMA transfer driver 1 (transfer size determination unit 33 in FIG. 8) increases the size of the subset as the priority increases. As a result, the high-priority requests R2 and R3 each become one subset without being divided. The low-priority request R1 is divided into 12 subsets.

[0030] Between times t2 and t3, the transfer engine 40 reads the subsets from the two transfer queues 30 in order (round robin) and transfers them from the host DRAM 12 to the device DRAM 51 by DMA. As a result, request R2 (= one subset) arrives at IP core 52(#1) at time t2, and request R3 (= one subset) arrives at IP core 52(#2) at time t3. In other words, the high-priority requests R2 and R3 were able to complete their DMA transfers even earlier than in Figure 4.

[0031] Note that changing the subset size of the request itself affects the throughput of the request itself and the TAT of other requests. Figure 6 shows a time chart for when the subset size of the request R1 is large. As a result of setting the subset size of the request R1 large, the request R1 is divided into two subsets and transferred via DMA. As a result, the other request R2 starts its DMA transfer at time t3 and arrives at the IP core 52 (#2) at time t4. Therefore, the amount of data transferred at one time for the request R1 increases, and during that time the other request R2 cannot be transferred, making it less likely that the transfer will be interrupted (preempted). As a result, the throughput of the request R1 increases, and the TAT of the other request R2 worsens.

[0032] Figure 7 shows a time chart for when the subset size of the request R1 is small. As a result of setting the subset size of the request R1 small, the request R1 is divided into four subsets and DMA transferred. As a result, the other request R2 starts its DMA transfer at time t2 and arrives at the IP core 52 (#2) at time t3. Therefore, the amount of data transferred at one time for the request R1 is reduced, and the other request R2 can be transferred in between, making the transfer more likely to be interrupted. As a result, the throughput of the request R1 is reduced, and the TAT of the other request R2 is improved.

[0033] In other words, the subset size is a trade-off between throughput and TAT. Therefore, the DMA transfer driver 1 achieves optimization by flexibly changing the subset size. By changing the subset size, the frequency with which the transfer of the request is interrupted by transfer requests from other requests is adjusted (see Figures 22 to 25 for details).

[0034] FIG. 8 is a detailed configuration diagram of the DMA transfer driver 1. The DMA transfer driver 1 includes the reception queue 20, M transfer queues 30, and N transfer engines 40 described in FIG. 2, as well as a transfer queue determination unit 21, a priority determination unit 22, a request counter 31, a request division unit 32, a transfer size determination unit 33, a transfer queue selection unit 41, an integrated control unit 42, and an external controller IF 61. The arrows in FIG. 8 are as follows: · Thick solid arrow: Request path · Bold dashed arrows: Route of management signals (m-plane) Thin solid arrows: Path of control signal (c-plane)

[0035] The transfer queue determination unit 21 dequeues a request in the receiving queue 20 and determines which transfer queue 30 to enqueue it in. The request counter 31 counts the number of requests stored in each transfer queue 30. The priority determination unit 22 determines the priority of the request based on the contents of the request or information from the external controller IF 61.

[0036] The transfer size determination unit 33 determines the size of subsets when dividing a request based on the priority of the request and the communication requirements (TAT, throughput, etc.) required for the request. For example, it is desirable for the transfer size determination unit 33 to increase the size of the subset as the priority of the request increases. This allows more data to be transferred at once for requests with higher priority, thereby improving the throughput and TAT of the requests with higher priority. The request dividing unit 32 divides the request based on the size determined by the transfer size determining unit 33. The transfer queue selecting unit 41 determines from which transfer queue 30 the free transfer engine 40 will dequeue the subset.

[0037] The integrated control unit 42 controls the DMA transfer driver 1, such as setting setting information for the transfer size determination unit 33 (specifying the minimum and maximum sizes of the subset) and setting information for the transfer queue selection unit 41 (specifying the read count policy). A control signal from the external controller 60 is notified to the interior of the DMA transfer driver 1, such as the priority determination unit 22, via the external controller IF 61. This enables the external controller 60 to execute external cooperation, such as notifying the priority determination unit 22 of the priority of each request.

[0038] The DMA transfer driver 1 includes the request division unit 32 and the transfer queue selection unit 41, and as explained in Figure 4, divides each request into multiple subsets of a fixed size before transferring them via DMA. As a result, even though requests R2 and R3 arrive later than request R1, they are smaller in size and therefore arrive first as subsets, thereby meeting the TAT requirement. Furthermore, as Figure 4 does not refer to the priority of each request, the development costs associated with modifications to the DMA transfer driver 1 are unnecessary.

[0039] Furthermore, the DMA transfer driver 1 includes a priority determination unit 22 and a transfer size determination unit 33 in addition to a request division unit 32 and a transfer queue selection unit 41, and as explained in Fig. 5, each request is divided into one or more subsets according to priority before being DMA transferred. As a result, the DMA transfer of requests R2 and R3 can be completed even more quickly than in Fig. 4. However, some development costs are required to modify the DMA transfer driver 1 in order to make it recognize the priority of each request.

[0040] FIG. 9 is a table 70 showing the state of the transfer process of the DMA transfer driver 1 of FIG. 8 at each time. Table 70 shows the data stored in each of the components, namely, the accept queue 20, the transfer queue 30 (#1 for low priority, #2 for high priority), the transfer engine 40, and the device DRAM 51, for each of times T1 to T7. Below, the contents of table 70 will be explained along the time periods T1 to T7, to clarify the components of the DMA transfer driver 1 in FIG.

[0041] At time T1, request B (low priority) is accepted into the accepting queue 20. At this point, both of the two transfer queues 30 (#1, #2) are empty (no stored entries), and the transfer engine 40 is also empty. At time T2, the transfer queue determination unit 21 enqueues request B dequeued from the receiving queue 20 in the transfer queue 30 (#1) determined by referring to the priority of the request determined by the priority determination unit 22. At time T3, the transfer size determination unit 33 refers to the subset size table 73 in Fig. 12 to determine the division size and number of divisions (2 in this case) of request B in the transfer queue 30 (#1). The request division unit 32 divides request B into two subsets, requests B1 and B2.

[0042] At time T4, the transfer queue selector 41 dequeues request B1 from the selected transfer queue 30 (#1) by referring to the transfer engine table 75 in FIG. 14, and requests DMA transfer from the transfer engine 40. The transfer engine 40 DMA transfers request B1 to the device DRAM 51. During this DMA transfer, the accept queue 20 accepts request A (high priority). At time T5, the transfer queue determination unit 21 enqueues request A dequeued from the receiving queue 20 to the transfer queue 30 (#2) determined by referring to the priority of the request determined by the priority determination unit 22. The request division unit 32 does not divide request A in the transfer queue 30 (#2) because its original size is small, but instead makes it into one subset.

[0043] At time T6, the DMA transfer of request B1 is completed. The transfer queue selector 41 references the transfer engine table 75 and dequeues request A in transfer queue 30(#2) as the next request to be processed, rather than request B2 in transfer queue 30(#1). The transfer engine 40 DMA transfers request A to the device DRAM 51. At time T7, request A, which arrived at the accept queue 20 later than request B, was given priority over request B2 and was therefore able to meet the TAT.

[0044] Various data tables used by the DMA transfer driver 1 will be described below with reference to FIGS. FIG. 10 is a diagram showing the configuration of the queue ID table 71. In the queue ID table 71, transfer queue IDs are registered for each priority. The transfer queue determination unit 21 selects the transfer destination transfer queue 30 by referring to the queue ID table 71. The integrated control unit 42 writes entries in the queue ID table 71 in advance.

[0045] FIG. 11 is a diagram showing the configuration of the transfer queue management table 72. The transfer queue management table 72 registers the usage status for each transfer queue ID. The integrated control unit 42 has the transfer queue management table 72, and the transfer queue determination unit 21 refers to the transfer queue management table 72 to check whether the transfer queue 30 is available. The integrated control unit 42 updates the latest status of the transfer queue 30 as an entry in the transfer queue management table 72 as needed.

[0046] FIG. 12 is a diagram showing the configuration of the subset size table 73. The subset size table 73 registers the range of subset sizes (minimum size, maximum size) and the subset size for each time slot (slot 0, slot 1, . . . ) for each transfer queue ID. The request dividing unit 32 determines the size into which each request is divided by referring to the subset size table 73. The integrated control unit 42 pre-sets the range of subset sizes in the subset size table 73. The transfer size determination unit 33 writes the subset size for each time slot within the range of the subset size to the subset size table 73 based on the request counter table 74 (FIG. 13).

[0047] FIG. 13 is a diagram showing the configuration of the request counter table 74. The transfer queue determination unit 21 stores in the request counter table 74 the total number of requests that have arrived at the transfer queue 30 and the number of requests in each time slot (slot 0, slot 1, . . . ) for each transfer queue ID (for each priority). The transfer size determination unit 33 refers to the request counter table 74 to determine the size of the subset.

[0048] FIG. 14 is a diagram showing the configuration of the transfer engine table 75. A transfer engine table 75 is prepared for each transfer engine 40. In the transfer engine table 75, the number of times a subset is read and the size of the subset are associated with each transfer queue ID. These contents are set by the integrated control unit 42. The transfer queue selector 41 refers to the transfer engine table 75 to determine from which transfer queue 30 a specific transfer engine 40 should dequeue a subset. For example, in the transfer engine table 75 of FIG. 14, the read count is set to 1. Therefore, after reading one subset from transfer queue (0) this time, the transfer queue selector 41 will read one subset from transfer queue (1) next time. In addition, instead of the number of reads, the transfer queue selection unit 41 may switch to dequeuing a subset from another transfer queue 30 next time when the read subset size reaches the value registered in the transfer engine table 75.

[0049] FIG. 15 is a diagram showing the configuration of the priority table 76. In the priority table 76, for each thread ID of the thread 11, a base priority and a priority for each time slot (t1, t2, t3, . . . ) are registered. The priority determination unit 22 determines the priority of the request issued from each thread 11, as exemplified by the following (Method 1) to (Method 3). (Method 1) The thread 11 passes the priority to the DMA transfer driver 1 individually for each request as a parameter without referring to the priority table 76. An example of the API (Application Programming Interface) of the thread 11 for passing the parameter is implemented by a data write command such as "write(priority, *buf, size);". The API of this write command is an API that writes data of the size specified by size from the pointer position of *buf in the host DRAM 12 to the device DRAM 51 with the priority specified by priority.

[0050] (Method 2) A fixed priority is set for each request of a specific thread ID, even over time, by referring to the base priority in the priority table 76. For example, in the priority table 76 of Fig. 15, the priority is set to "0" for thread #1 and "2" for thread #2. (Method 3) By referring to the priority for each time slot (t1, t2, t3, ...) in the priority table 76, the priority of the time slot is set according to the timing information from the external controller 60 (the priority is dynamically changed as time passes). For example, in the priority table 76 of Fig. 15, when time slot = t2, the priority is determined to be "1" for thread #1 and "2" for thread #2.

[0051] FIG. 16 is a diagram showing the hardware configuration of the physical server 100. As shown in FIG. The physical server 100 is configured as a computer 900 having a CPU 901 , a RAM 902 , a ROM 903 , a HDD 904 , a communication I / F 905 , an input / output I / F 906 , and a media I / F 907 . The accelerator 905 is a device 50 (FIG. 1) that processes at least one of data from the communication I / F 908 and data from the RAM 902 at high speed. Note that the accelerator 905 may be of a type (look-aside type) that executes processing from the CPU 901 or RAM 902 and then returns the execution result to the CPU 901 or RAM 902. On the other hand, the accelerator 905 may be of a type (in-line type) that performs processing between the communication I / F 908 and the CPU 901 or RAM 902. The accelerator 905 is connected to an external device 915 via a communication I / F 908. The input / output I / F 906 is connected to an input / output device 916. The media I / F 907 reads and writes data from a recording medium 917. Furthermore, the CPU 901 controls each processing unit by executing a program (also called an application or an app for short) loaded into the RAM 902. This program can also be distributed via a communication line or recorded on a recording medium 917 such as a CD-ROM and distributed.

[0052] FIG. 17 is a sequence diagram showing the request transfer process of the physical server 100. As shown in FIG. The priority determination unit 22 reads the metadata (description information) of the request from the receiving queue 20 (S201), and notifies the transfer queue determination unit 21 of the priority information of the request extracted from the metadata (S202). The transfer queue determination unit 21 determines the storage destination of the request as transfer queue information (#0) based on the priority information of the request (S203). The transfer queue determination unit 21 enqueues the request dequeued from the receiving queue 20 into the transfer queue 30 (#0) (S204).

[0053] FIG. 18 is a sequence diagram showing the request transfer process of the physical server 100, following FIG. The transfer queue selection unit 41 checks whether a request is stored in the transfer queue 30(#0) (S211) and receives a notification of the stored request (S212). The transfer engine 40(#0) is in an idle state (not transferring data) and notifies the transfer queue selection unit 41 that it can accept a new request (S213). The transfer queue selection unit 41 requests the transfer queue 30(#0) to transfer the request notified in S212 to the transfer engine 40(#0) notified in S213 (S214). The request dividing unit 32 receives the request for which transfer was requested in S214 (S215), divides it into subsets, and notifies the transfer engine 40 (#0) of the subsets (S216).

[0054] FIG. 19 is a flowchart showing the process of notifying a request from the receiving queue 20 to the transfer queue 30. The transfer queue determination unit 21 determines whether a new request has been received from the thread 11 to the receiving queue 20 (S101). If the answer is Yes in S101, metadata (description information) of the request is sent from the receiving queue 20 to the priority determination unit 22 (S102). If the answer is No in S101, proceed to S103. The transfer queue determination unit 21 determines whether there is a request for which transfer queue information to be enqueued has been determined (S103). If the result in S103 is No, the process returns to S101. If the answer is Yes in S103, the transfer queue determination unit 21 determines whether or not there is a vacancy in the transfer queue 30 indicated by the determined transfer queue information (S104).If the answer is No in S104, the process returns to S101. If the answer is Yes in S104, the transfer queue determination unit 21 sends (enqueues) the request to the transfer queue 30 based on the transfer queue information (S105).

[0055] FIG. 20 is a flowchart showing the process by which the transfer queue 30 notifies the transfer engine 40 of a subset of requests. The transfer queue selection unit 41 selects a transfer queue 30 from which to dequeue the request. Then, the transfer queue selection unit 41 determines whether or not a notification that the request is available has been received from an available transfer engine 40 (S111). If the answer is No in S111, the process returns to S111. If the answer is Yes in S111, the request dividing unit 32 divides the requests in the transfer queue 30 into subsets. The size of the subsets is determined by the transfer size determination unit 33. Then, the transfer queue selection unit 41 transmits the divided subsets to the transfer engine 40 that was notified in S111 that it can accept the requests (S112). The transfer queue selection unit 41 also notifies the request counter 31 of the number of subsets that have been sent (the number of requests for each priority), and updates the request counter table 74 (S113).

[0056] FIG. 21 is a flowchart showing the process of DMA transfer of a subset from transfer engine 40 to device 50. The transfer engine 40 determines whether the previous subset is currently being transferred (S121). If the answer is Yes in S121, the process returns to S121. If the answer is No in S121, the transfer queue selection unit 41 determines whether or not the subset data to be transferred exists in any of the transfer queues 30 that are to be read (S122). If the answer is No in S122, the process returns to S121. If the answer is Yes in S122, the transfer queue selection unit 41 refers to the transfer engine table 75 and the previous transfer history to determine the transfer queue 30 that is the current read destination (S123).

[0057] The transfer queue selection unit 41 transmits a transfer request for the subset to the transfer queue 30 (S124) and receives the subset from the transfer queue 30 (S125). The transfer queue selection unit 41 transfers the subset of S125 to the transfer engine 40 (S126). As a result, the subset is DMA-transferred from the transfer engine 40 to the device 50.

[0058] Hereinafter, the process of dynamically changing the subset size of a request by the transfer size determination unit 33 will be described with reference to FIGS. FIG. 22 is a flowchart showing a first example of a process for changing the subset size of a request. In this first example, a case will be described in which requests with priority "2" (high priority) are usually generated infrequently, but bursts (large numbers) occur during random periods. In this case, the transfer size determination unit 33 controls the size of the subsets created from the request data so that during a period when no high priority "2" requests are received in the transfer queue 30, the size of the low priority "1" subset is enlarged, and during a period when high priority requests are received in the transfer queue 30, the size of the low priority subset is reduced. Below, the details of subset size control will be explained with reference to the flowchart.

[0059] The transfer size determination unit 33 extracts a count value obtained by counting the number of requests currently stored in each transfer queue 30 (S301). If the count value for priority "2" has increased compared to the previous count value (Yes in S302), that is, if a burst of requests occurs, the transfer size determination unit 33 resets the subset size of the transfer queue 30 corresponding to priority "1" to its initial value (S303). If the answer is No in S302 and the count value for priority "1" has increased by a certain amount compared to the previous count value (Yes in S304), the transfer size determination unit 33 increases the subset size of the transfer queue 30 corresponding to priority "1" (S305). This subset size increase process increases the size by a predetermined value (+n) within a range that does not exceed the maximum value.

[0060] FIG. 23 is a time series graph of the high-priority and low-priority subset sizes in the first example of FIG. The upper graph shows the count value c2(k) in time slot k with priority "2". The lower graph shows the subset size s1(k) for time slot k of priority "1." When throughput is prioritized, this s1(k) increases to the maximum subset size s1max for priority "1." On the other hand, when TAT of other priority "2" is prioritized, it decreases to the minimum subset size s1min for priority "1."

[0061] In period T11, priority "2" does not occur, so the subset size of priority "1" is increased to s1max, prioritizing throughput. In period T12, priority "2" occurs, so the subset size of priority "1" is decreased to s1min, prioritizing TAT of priority "2". In period T13, priority "2" does not occur, so the subset size of priority "1" is gradually increased to s1max, prioritizing throughput. In period T14, priority "2" occurs, so the subset size of priority "1" is decreased to s1min, prioritizing TAT of priority "2". During period T15, no priority "2" occurs, so the subset size of priority "1" is gradually increased up to s1max, thereby prioritizing throughput.

[0062] FIG. 24 is a flowchart showing a second example of the process of changing the subset size of a request. In this second example, a case will be described in which the priority "2" (high priority) increases and decreases at regular intervals. In this case, the transfer size determination unit 33 controls the size of the subsets created from the request data so that during a period when the number of high priority "2" requests entering the transfer queue 30 increases, the size of the low priority "1" subset is reduced, while during a period when the number of high priority requests entering the transfer queue 30 decreases, the size of the low priority subset is increased. Below, the details of subset size control will be explained with reference to the flowchart.

[0063] The transfer size determination unit 33 performs loop processing (S401 to S405) to increase the loop variable k by one (k++) from the initial value k=0 within a range less than the number of time slots (k<number of time slots). The transfer size determination unit 33 extracts the count value of the transfer queue 30 corresponding to the priority "2" of the time slot k (S402), and calculates s1(k) using (Equation 1) (S403).

[0064]

number

[0065] Then, the transfer size determination unit 33 updates the subset size in the time slot k with priority "1" to s1(k) calculated in S403 (S404).

[0066] FIG. 25 is a time series graph of the high-priority and low-priority subset sizes in the second example of FIG. 23, the upper graph shows c2(k) and the lower graph shows s1(k). The threshold c2P in the upper graph is a count threshold at which priority "2" should be processed with the highest priority. In the period T21, there are no requests with priority "2", so priority "1" has a maximum size s1max. During period T22, the number of requests with priority "2" increases, so the size of requests with priority "1" is reduced, thereby improving the TAT for priority "2". In the period T23, if the number of requests with priority "2" exceeds the threshold c2P, the number of requests with priority "1" is set to the minimum size s1min. During period T24, the number of requests with priority "2" decreases, so the size of requests with priority "1" is increased, thereby improving the throughput of requests with priority "1".

[0067] [effect] The present invention provides a physical server 100 having a host 10 having a host DRAM 12 in which source data is stored, and a device 50 having a device DRAM 51 to which the data is to be transferred, The host (10) has a DMA transfer driver (1) that performs DMA transfer from the host DRAM (12) to the device DRAM (51), and a plurality of transfer queues (30) that store requests indicating transfer requests for data that are the subject of DMA transfer; The DMA transfer driver 1 creates one or more subsets from the data of the host DRAM 12 designated by the request in each transfer queue 30, and transfers the subsets of each transfer queue 30 selected in order to the device DRAM 51 by DMA.

[0068] This reduces transfer delays for each request by dividing the request data before DMA transfer. For example, when multiple threads 11 request DMA transfers to the same device 50, even while a large-sized request from one thread 11 is being transferred, a request from another thread 11 that arrives later can be transferred. Therefore, fairness between the threads 11 is maintained, and a thread 11 with strict delay requirements can run on the same physical server 100 as other threads 11 (noisy neighbors). In this way, by providing equal opportunities for data transfer among threads 11 regardless of the size of the data requested from each thread 11, it is possible to achieve a good balance between the transfer efficiency and delay requirements of the data being transferred when transferring data from multiple sources to the same destination.

[0069] In the present invention, each transfer queue 30 is assigned an individual priority and stores requests of that priority. The DMA transfer driver 1 is characterized in that the size of the subset created from the data of the request increases as the priority of the request increases.

[0070] As a result, the higher the priority of a request, the larger the size of the subset, thereby increasing the efficiency of data transfer. Therefore, since the DMA transfer driver 1 has the versatility to handle the priority of each request, it is possible to reduce the development cost of implementing priority control (specifying the transfer order and required bandwidth) through direct cooperation between threads 11. Furthermore, threads 11 with different priorities can be accommodated on the same physical server 100, which reduces the cost of the chassis and power consumption compared to a system in which multiple physical servers 100 are operated for each priority.

[0071] In the present invention, each transfer queue 30 is assigned an individual priority and stores requests of that priority. The DMA transfer driver 1 controls the size of the subsets created from the request data so that the size of the low priority subset is enlarged during a period when no high priority requests enter the transfer queue 30, and the size of the low priority subset is reduced during a period when high priority requests enter the transfer queue 30.

[0072] This allows dynamic control of the subset size to balance the transfer requirements of both high-priority and low-priority requests, even if high-priority requests typically occur infrequently but occur in bursts at random periods.

[0073] In the present invention, each transfer queue 30 is assigned an individual priority and stores requests of that priority. The DMA transfer driver 1 controls the size of the subsets created from the request data so that the size of the low priority subset is reduced during a period when the number of high priority requests entering the transfer queue 30 is increasing, while the size of the low priority subset is increased during a period when the number of high priority requests entering the transfer queue 30 is decreasing.

[0074] This makes it possible to dynamically control the size of the subset so as to satisfy the transfer requirements of both high-priority and low-priority requests in a balanced manner when the number of high-priority requests increases or decreases periodically. [Explanation of symbols]

[0075] 1 DMA transfer driver (transfer control section) 10 Host (Host part) 11 threads 12 Host DRAM (host memory) 20 Reception Queue 21 Transfer queue determination unit 22 Priority determination section 30 transfer queue 31 Request Counter 32 Request Splitter 33 Transfer size determination unit 40 forwarding engine 41 Transfer queue selection section 42 Integrated control unit 50 Device (Device Section) 51 Device DRAM (Device Memory) 52 IP cores 59 PCIe 60 External Controller 61 External controller IF 71 Queue ID Table 72 Transfer queue management table 73 Subset Size Table 74 Request Counter Table 75 Transfer Engine Table 76 Priority Table 100 physical servers (transfer management devices)

Claims

1. A transfer management device having a host unit having a host memory in which data from a transfer source is stored, and a device unit having a device memory to which the data is transferred, the host unit has a transfer control unit that performs DMA transfer from the host memory to the device memory, and a plurality of transfer queues that store requests indicating transfer requests that specify data to be DMA transferred; The plurality of transfer queues are assigned individual priorities and store requests of those priorities; In the process of creating one or more subsets from the data of the host memory specified by each request in the plurality of transfer queues, the transfer control unit increases the size of the subset created from the data of the request as the priority of the request increases, and sequentially selects each subset in the plurality of transfer queues and DMA transfers them to the device memory. Transfer management device.

2. The transfer control unit controls the size of the subsets created from the request data so that the size of the low-priority subset is increased during a period when no high-priority requests are entering the transfer queue, and the size of the low-priority subset is reduced during a period when high-priority requests are entering the transfer queue. The transfer management device according to claim 1 .

3. The transfer control unit controls the size of the subsets created from the request data so that, during a period in which the number of high-priority requests entering the transfer queue increases, the size of the low-priority subset is reduced, and, on the other hand, during a period in which the number of high-priority requests entering the transfer queue decreases, the size of the low-priority subset is increased. The transfer management device according to claim 1 .

4. A transfer management method executed by a transfer management device having a host unit having a host memory in which data from a transfer source is stored and a device unit having a device memory to which the data is transferred, the host unit has a transfer control unit that performs DMA transfer from the host memory to the device memory, and a plurality of transfer queues that store requests indicating transfer requests that specify data to be DMA transferred; The plurality of transfer queues are assigned individual priorities and store requests of those priorities; In the process of creating one or more subsets from the data of the host memory specified by each request in the plurality of transfer queues, the transfer control unit increases the size of the subset created from the data of the request as the priority of the request increases, and sequentially selects each subset in the plurality of transfer queues and DMA transfers them to the device memory. Transfer management methods.

5. A transfer management program for causing a computer to function as the transfer management device according to any one of claims 1 to 3.

Images