Accelerator off-road device, accelerator off-road system, and accelerator off-road method

Abstract

Provided is an accelerator offloading device (100) that offloads a specific process of an APL (1) to an ACC (12), the device comprising a request-related processing unit (13) that executes predetermined processing required before offloading to the ACC (12) and notifies a request I / O unit (150) of an offloading request; the request I / O unit (150) which comprises a CPU core and which executes request processing for notifying the ACC (12) of the offloading request; a response I / O unit (160) which comprises a CPU core different from the above CPU core and which executes response processing for notifying a response-related processing unit (140) of the completion of computation by the ACC (12); and the response-related processing unit (140) that uses a computation result from the ACC (12) to execute computation described in the APL (1).

Description

Technical Field

[0001] The present invention relates to an accelerator offload device, an accelerator offload system, and an accelerator offload method.

Background Art

[0002] Depending on the type of processor, the workloads at which it excels are different. For a highly versatile CPU (Central Processing Unit), there are accelerators such as FPGA (Field Programmable Gate Array) / (in the following description, " / " represents "or") GPU (Graphics Processing Unit) / ASIC (Application Specific Integrated Circuit) that can perform operations on high-parallelism workloads that the CPU is not good at quickly and efficiently. By combining these heterogeneous processors and offloading the workloads that the CPU is not good at to the ACC for operation, the utilization of offload technology to improve the overall operation time and operation efficiency has been advancing.

[0003] Specific workloads for which ACC offloading is performed include, as representative examples, encoding / decoding processing (FEC: Forward Error Correction processing) in vRAN (virtual Radio Access Network), media processing of audio and video, and encryption / decryption processing.

[0004] FIG. 17 is a schematic diagram of a process of offloading a part of the process calculated by the CPU to an accelerator (ACC). As shown in FIG. 17, the accelerator system has an application (APL) 1 on hardware (HW) 10, OS, etc. 20, and user space 30.

[0005] The hardware 10 includes a CPU 11 and an accelerator (ACC) 12. ACC12 is a computing unit hardware that performs specific calculations at high speed based on input from CPU11. Specifically, accelerator 12 is a PLD (Programmable Logic Device) such as a GPU or FPGA.

[0006] As shown by the white arrow a in Figure 17, a portion of the processing of APL1 (workloads that CPU11 is not good at) is offloaded to ACC12, achieving performance and power efficiency that cannot be achieved by software (CPU processing) alone.

[0007] On the other hand, data transfer technologies within servers include New API (NAPI), DPDK (Data Plane Development Kit), and KBP (Kernel Busy Poll).

[0008] New API (NAPI) processes packets upon arrival via a hardware interrupt request followed by a software interrupt request (see Non-Patent Document 1).

[0009] DPDK implements packet processing functionality in user space where applications run, and immediately harvests packets upon arrival from user space using a polling model. Specifically, DPDK is a framework for controlling NICs (Network Interface Cards) in user space, a task traditionally handled by the Linux kernel. The biggest difference from the Linux kernel's processing is its polling-based receiving mechanism called PMD (Pull Mode Driver). Normally, the Linux kernel generates an interrupt upon data arrival at the NIC, triggering the execution of the receiving process. In contrast, PMD uses a dedicated thread to continuously check for data arrival and perform receiving processing. PMD enables high-speed packet processing by eliminating overhead such as context switching and interrupts. DPDK significantly improves packet processing performance and throughput, allowing more time for data plane application processing. However, DPDK exclusively uses computer resources such as the CPU and NIC.

[0010] Patent Document 1 describes a server-internal network delay control device (KBP). The KBP constantly monitors packet arrivals using a polling model within the kernel. This suppresses softIRQ and enables low-latency packet processing. [Prior art documents] [Patent Documents]

[0011] [Patent Document 1] International Publication No. 2021 / 130828 [Non-patent literature]

[0012] [Non-Patent Document 1] New API (NAPI), [online], [Searched March 11, 2002], Internet 〈 URL: http: / / http: / / lwn.net / 2002 / 0321 / a / napi-howto.php3〉 [Overview of the project] [Problems that the invention aims to solve]

[0013] Conventional ACC offload systems have two methods for obtaining calculation results from ACC: (1) interrupt method and (2) polling method. (1) The interrupt method is a method in which the application detects the completion of a calculation by an interrupt. (2) The polling method is a method in which the completion of a calculation by ACC is immediately detected by busy poll (constantly monitoring a buffer in which data is stored when a calculation is completed). The following will explain them in order.

[0014] Figure 18 is a diagram illustrating the interrupt method described in (1) above, and Figure 19 is a schematic diagram illustrating the time progression of the interrupt method in Figure 18. In explaining Figure 18, the same reference numerals are used for components identical to those in Figure 17. As shown in Figure 18, APL1 requests ACC12 to offload processing (see symbol b in Figure 18), and ACC12 responds by interrupting when the calculation is complete (see symbol c in Figure 18).

[0015] As shown in Figure 19, the application thread (CPU) requests that the currently executing process 1 be offloaded to ACC12 (see symbol b in Figure 19). While the application thread (CPU) is offloading to ACC, it can execute other processes (in this case, process 2). Because other processes can be executed while the process is offloaded to ACC, the CPU usage is efficient.

[0016] (1) In the interrupt method, APL1 detects the completion of the calculation by ACC via an interrupt (see symbol c in Figure 19). The application thread (CPU) needs to interrupt the process in progress (process 2) and save the intermediate data. In this case, the interrupt causes post-processing of process 1 (for example, forward error correction (FEC) processing) to be performed before continuing with the process in progress (continuation of process 2). Therefore, the increased processing time due to interrupt processing overhead becomes a problem.

[0017] Figure 20 is a diagram illustrating the polling method described in (2) above, and Figure 21 is a schematic diagram showing the time progression of the polling method in Figure 20. In explaining Figure 20, the same reference numerals are used for components identical to those in Figure 18. As shown in Figure 20, APL1 requests ACC12 to offload the processing (see symbol d in Figure 20). APL1 performs a busy poll (constantly monitoring the buffer where data is stored when ACC completes its calculations) (see symbol e in Figure 20). This allows APL1 to immediately detect the completion of the calculation by ACC12 (see symbol f in Figure 20).

[0018] As shown in Figure 21, the application thread (CPU) requests the ACC12 to offload the currently executing process 1 (see symbol d in Figure 21). While the ACC is offloading, the application thread (CPU) continues busy polling without executing any other processes until it receives the calculation result from ACC12. This results in wasted CPU resources during polling. However, the (2) polling method can quickly detect the completion of ACC processing through busy polling (see symbol f in Figure 21). After quickly detecting the completion of ACC processing, the post-processing of process 1 is executed before proceeding to the next process (process 2).

[0019] In summary, the advantages and disadvantages of (1) the interrupt method and (2) the polling method as methods for obtaining calculation results by ACC in ACC off-road are as follows: (1) Although the interrupt method has the advantage of high CPU usage efficiency, there is a problem that the processing time becomes long due to interrupt processing overhead. (2) Although the polling method can quickly detect the completion of the ACC operation by busy poll, there are problems of increased power consumption due to wasteful use of CPU resources during polling and deterioration of CPU resource efficiency.

[0020] In view of such a background, the present invention has been made, and the object of the present invention is to shorten the processing time and improve the CPU utilization efficiency.

Means for Solving the Problems

[0021] In order to solve the above-described problems, an accelerator offload device that offloads specific processing of an application program to an accelerator, which executes predetermined processing necessary until offloading to the accelerator and notifies the RequestI / O unit of a request for offloading, a Request-related processing unit, composed of CPU cores, which executes Request processing for notifying the accelerator of an offload request to the RequestI / O unit, and a ResponseI / O unit, composed of a CPU core different from the CPU core, which executes Response processing for notifying the Response-related processing unit of the completion of the accelerator's operation, and the Response-related processing unit, which uses the operation result of the accelerator to execute the operation described in the application program. An accelerator offload device is provided. Hardware interrupt Characterized by comprising:

Effects of the Invention

[0022] According to the present invention, it is possible to shorten the processing time and improve the CPU utilization efficiency.

Brief Description of the Drawings

[0023] [Figure 1] It is a schematic configuration diagram of an accelerator offload system according to a first embodiment of the present invention. [Figure 2] It is a time-lapse schematic diagram for explaining the outline of the operation of an accelerator offloading device of an accelerator offloading system according to the first embodiment of the present invention. [Figure 3] It is a flowchart showing a <preparation phase> by a management unit of an accelerator offloading device of an accelerator offloading system according to the first embodiment of the present invention. [Figure 4] It is a flowchart showing <Request side processing> of an accelerator offloading device of an accelerator offloading system according to the first embodiment of the present invention. [Figure 5] It is a flowchart showing <Response side processing> of an accelerator offloading device of an accelerator offloading system according to the first embodiment of the present invention. [Figure 6] It is a diagram for explaining an example of task allocation of a task scheduler of an accelerator offloading device of an accelerator offloading system according to the first embodiment of the present invention. [Figure 7] It is a schematic configuration diagram of a modification example of an accelerator offloading system according to the first embodiment of the present invention. [Figure 8] It is a time-lapse schematic diagram for explaining the outline of the operation of an accelerator offloading device of a modification example of an accelerator offloading system according to the first embodiment of the present invention. [Figure 9] It is a schematic configuration diagram of an accelerator offloading system according to the second embodiment of the present invention. [Figure 10] It is a time-lapse schematic diagram for explaining the outline of the operation of an accelerator offloading device of an accelerator offloading system according to the second embodiment of the present invention. [Figure 11] It is a flowchart showing a <preparation phase> by a management unit of an accelerator offloading device of an accelerator offloading system according to the second embodiment of the present invention. [Figure 12] It is a flowchart showing <Task scheduler processing / Request side processing> of an accelerator offloading device of an accelerator offloading system according to the second embodiment of the present invention. [Figure 13] It is a flowchart showing <Response side processing> of an accelerator offload device of an accelerator offload system according to a second embodiment of the present invention. [Figure 14] It is a hardware configuration diagram showing an example of a computer that realizes the functions of an accelerator offload device of an accelerator offload system according to a second embodiment of the present invention. [Figure 15] It is a diagram showing an example of applying an accelerator offload system to an interrupt model in a server virtualization environment with a general-purpose Linux kernel and a VM configuration. [Figure 16] It is a diagram showing an example of applying an accelerator offload system to an interrupt model in a server virtualization environment with a container configuration. [Figure 17] It is a schematic diagram of a process of offloading a part of a process calculated by a CPU to an accelerator (ACC). [Figure 18] It is a diagram for explaining an interrupt method. [Figure 19] It is a time lapse schematic diagram of the interrupt method of FIG. 18. [Figure 20] It is a diagram for explaining a polling method. [Figure 21] It is a time lapse schematic diagram of the polling method of FIG. 20.

Embodiment for Carrying Out the Invention

[0024] Hereinafter, an accelerator offload system and the like in an embodiment for carrying out the present invention (hereinafter referred to as "the present embodiment") will be described with reference to the drawings. (First Embodiment) [Overall Configuration] FIG. 1 is a schematic configuration diagram of an accelerator offload system according to the first embodiment of the present invention. The same reference numerals are assigned to the same components as in FIG. 17. As shown in Figure 1, the accelerator offload system 1000 includes hardware (HW) 10, an OS etc. 20, a high-speed data communication unit 40 which is high-speed data transfer middleware located on the user space 200, an accelerator offload device 100, and an APL1.

[0025] The hardware 10 comprises a CPU 11, an accelerator (ACC) 12, and a ring buffer 13. ACC12 is a computing unit hardware that performs specific calculations at high speed based on input from CPU11. Specifically, accelerator 12 is a PLD such as a GPU or FPGA. The ring buffer 13 is located on the hardware 10 and copies the target workload. The Request I / O unit 150 and the Response I / O unit 160 exchange data with the accelerator via the ring buffer 13.

[0026] Furthermore, application (APL)1 (an application program) is placed in user space200. APL1 is a program that is executed by the application thread (CPU) in user space200.

[0027] [Accelerator Off-Road Device 100] The accelerator offload device 100 comprises a management unit 110, a task scheduler 120, a Request-related processing unit 130, a Response-related processing unit 140, a Request I / O unit (CPU#n) 150, and a Response I / O unit (CPU#m) 160. The task scheduler 120 has a sleep control unit 121. The Request I / O unit (CPU#n) 150 has a sleep control unit 151. The Response I / O unit (CPU#m) 160 has a sleep control unit 161.

[0028] In the following explanation, "sleep" refers to the CPU executing an instruction with a low number of cycles, such as a pause instruction. Furthermore, CPU#n and CPU#m (where n and m are arbitrary natural numbers) indicate the use of different CPU cores.

[0029] The accelerator offload device 100 is located on user space 200. For example, the Request-related processing unit 130 and the Response-related processing unit 140 are implemented in APL1, while the Request I / O unit (CPU#n) 150 and the Response I / O unit (CPU#m) 160 are implemented in the high-speed data communication unit 40 (as a library of a high-speed data communication layer consisting of CUDA, OpenCL BBDEV API, etc.). The Request I / O unit (CPU#n) 150 and the Response I / O unit (CPU#m) 160 may be implemented by being included in the Request-related processing unit 130 and the Response-related processing unit 140, respectively, within APL1.

[0030] <Management Department 110> The management unit 110 manages a group of CPU cores consisting of multiple CPU cores. The management unit 110 determines which CPU core to use from the group of CPU cores, which is then selected by the Request-related processing unit 130, the Response-related processing unit 140, the Request I / O unit 150, or the Response I / O unit 160. The management unit 110 assigns one CPU core from the group of CPU cores to the Response I / O unit 160 as a function unit dedicated to Response.

[0031] The management unit 110 pre-allocates a group of CPU cores that each functional unit (Request-related processing unit 130, Response-related processing unit 140, Request I / O unit (CPU#n) 150, Response I / O unit (CPU#m) 160) may use. The operator may pre-determine how the CPU cores will be used so that other applications do not use the aforementioned group of CPU cores. The management unit 110 determines which CPU core each functional unit (Request-related processing unit 130, Response-related processing unit 140, Request I / O unit (CPU#n) 150, Response I / O unit (CPU#m) 160) will use from the group of CPU cores.

[0032] <Task Scheduler 120> The task scheduler 120 registers a task in the task queue of the Request-related processing unit 130 when a task requiring offloading to the accelerator occurs. The task scheduler 120 registers the task as one that uses a different CPU core than the one used by the Response I / O unit 160.

[0033] The task scheduler 120 includes a sleep control unit that puts threads running on the CPU to sleep if there are no tasks to perform calculations on the CPU.

[0034] • When a task requiring off-roading to ACC12 occurs When a task requiring offloading to ACC12 occurs, the task scheduler 120 registers the task in the task queue of the Request-related processing unit 130 (see Figure 6 below for an image of task distribution). At this time, the task scheduler 120 registers the task as one that uses a different CPU core than the one used by the Response I / O unit (CPU#m) 160. The sleep control unit 121 of the task scheduler 120 wakes up the Request-related processing unit 130 if it is sleeping. Furthermore, Task Scheduler 120 increases the CPU operating frequency of CPU core #n used by each processing unit if it has decreased, and transitions the CPU idle state from power-saving mode to non-power-saving mode if it is in power-saving mode.

[0035] • When an off-road request is notified to ACC12 The sleep control unit 121 of the task scheduler 120 puts the Request related processing unit 130 to sleep. At this time, for further power saving, the CPU operating frequency of the CPU core #n being used may be decreased, and the CPU idle state may be set to the power saving mode.

[0036] <Request related processing unit 130> The Request related processing unit 130 executes a series of processes (predetermined processes) necessary until the offloading of the accelerator. The series of processes necessary until the offloading of the accelerator will be described later. After the Request related processing unit 130 executes a series of processes necessary until the ACC offloading, it notifies the Request I / O unit (CPU#n) 150 of the request for ACC offloading. Note that when the CPU cores used by the sleep control unit 121 of the task scheduler 120 and the Request related processing unit 130 are different, the sleep control unit 121 of the task scheduler 120 may put the Request I / O unit 150 to sleep at this timing. The Request related processing unit 130 uses the calculation result of the ACC 12 to execute the calculation described in the APL1.

[0037] <Response related processing unit 140> The Response related processing unit 140 uses the calculation result of the ACC 12 to execute the calculation described in the application program.

[0038] The Response related processing unit 140 uses the calculation result of the ACC 12 to execute the calculation described in the APL1. The Response related processing unit 140 may perform the processing on the CPU core #m used by the Response I / O unit (CPU#m) 160, or may perform the processing on another core (FIG. 8 described later).

[0039] A series of processes necessary until the ACC offloading will be described. Take, for example, the case of offloading the FEC encoding process to the ACC12 in the vRAN vDU (virtual DU). · [Up Link Processing] In the Request-related processing unit 130, resource element mapping, equalization, IDFT (Inverse Discrete Fourier Transform), channel estimation, demodulation, and descrambling are performed. In the Response-related processing unit 140, frame processing (such as transmission processing of Ethernet frames) is performed.

[0040] · [Down Link Processing] In the Request-related processing unit 130, frame processing (such as reception processing of Ethernet frames) is performed. In the Response-related processing unit 140, scrambling, modulation, layer mapping, precoding, and resource element mapping are performed.

[0041] <Request I / O Unit (CPU#n) 150> The Request I / O Unit (CPU#n) 150 is composed of CPU cores and executes Request processing to notify the ACC12 of an offloading request.

[0042] The Request I / O Unit (CPU#n) 150 notifies the ACC12 of an offloading request. At this time, the Request I / O Unit (CPU#n) 150 copies the target workload via the ring buffer 13.

[0043] <Response I / O Unit (CPU#m) 160> The Response I / O Unit (CPU#m) 160 is composed of a CPU core different from the CPU core and executes Response processing to notify the Response-related processing unit 140 of the completion of the accelerator's operation.

[0044] The ResponseI / O unit (CPU#m) 160 wakes up upon receiving an interrupt. At this time, the task scheduler 120 increases the CPU operating frequency of CPU core #m used by the ResponseI / O unit 160 if it has decreased, and transitions the CPU idle state from power-saving mode to non-power-saving mode if it is in power-saving mode. The Response I / O unit (CPU#m) 160 notifies the Response-related processing unit 140 that the calculation of ACC12 is complete and transmits pointer information to the corresponding area of ​​the ring buffer 13 where the calculation result is stored.

[0045] [High-speed data communication unit 40] The high-speed data communication unit 40 is a high-speed data communication layer consisting of CUDA, OpenCL BBDEV API, etc. For example, the high-speed data communication unit 40 includes the CUDA Toolkit (registered trademark) for using NVIDIA GPUs and OpenCL (registered trademark) for calculations using heterogeneous processors. The BBDEV API (registered trademark) provides accelerator I / O functions for processing wireless access signals as a Development Kit (library). The high-speed data communication unit 40 can provide accelerator I / O functionality for processing wireless access signals to APL1 in user space 200 by incorporating accelerator I / O functions provided as libraries by the CUDA, OpenCL BBDEV API, etc. into APL1.

[0046] The operation of the accelerator off-road device 100 of the accelerator off-road system 1000 configured as described above will be explained below. (Explanation of the principle) (1) Reduce interrupt overhead The present invention separates CPU cores for Request processing and Response processing in order to avoid interrupt overhead caused by saving processing in an interrupt-based system. In this embodiment, the accelerator offload device 100 comprises a CPU core for Request processing (Request-related processing unit 130 and Request I / O unit 150) and a CPU core for Response processing (Response-related processing unit 140 and Response I / O unit 160). In other words, the accelerator offload device 100 provides (assigns) at least one of its multiple CPU cores as a CPU core that is a dedicated Response function unit (Response-related processing unit 140 and Response I / O unit 160). By providing a CPU core dedicated to response processing (a functional unit specifically for responses), the system maintains a state where there are no pending processes during interrupts, eliminating the need for save processing and thus achieving low latency.

[0047] (2) Power saving In order to suppress the increase in power consumption caused by providing a dedicated function unit for Response, this invention performs sleep control (including CPU operating frequency control and CPU idle state control) when there is no processing. When there are no power-saving processes, the system goes into sleep mode, and power consumption is reduced and power saving is achieved by controlling the CPU operating frequency and CPU idle state.

[0048] (3) Reduce the number of memory copies In data communication between APL1 and ACC12, a ring buffer 13 is provided that can be accessed from both the high-speed data communication layer and ACC. The I / O unit of the high-speed data communication unit 40 exchanges data with ACC12 via the ring buffer 13, thereby reducing the number of memory copies between APL1 and ACC12. By reducing unnecessary memory copies, high-speed data communication is achieved.

[0049] [Operation Overview of Accelerator Off-Road Device 100] As shown in Figure 1, APL1 requests the Request-related processing unit 130 of the accelerator offload device 100 to offload processing. After the Request-related processing unit 130 performs a series of processes necessary before ACC offloading, it notifies the Request I / O unit (CPU#n) 150 of the request for ACC offloading. The Request I / O unit (CPU#n) 150 notifies the ACC 12 of the offload request (see symbol aa in Figure 1). At this time, the Request I / O unit (CPU#n) 150 copies the target workload via the ring buffer 13.

[0050] ACC12 performs calculations on the offloaded workload and notifies the ResponseI / O unit (CPU#m) 160 via a hardware interrupt when the calculation is complete (see symbol bb in Figure 1), and stores the calculation result in the ring buffer 13. At this time, the destination of the hardware interrupt is specified as a CPU core (ResponseI / O unit (CPU#m) 160) that is different from the CPU core used by the Request-related processing unit 130 (RequestI / O unit (CPU#n) 150).

[0051] Figure 2 is a schematic timeline illustrating the operation of the accelerator off-road device 100 shown in Figure 1. As shown in Figure 2, the Request-related processing unit 130 and the Request I / O unit (CPU#n) 150 request the application thread (CPU) to offload process 1, which is currently being executed, to the ACC 12 (see symbol aa in Figure 1). While the application thread (CPU) is offloading to the ACC, it can execute other processes (in this case, process 2). Because other processes can be executed while the process is offloaded to the ACC, it enjoys the advantage of the conventional (1) interrupt method, which is good CPU utilization efficiency.

[0052] At this time, the Response-related processing unit 140 and the Response I / O unit (CPU#m) 160 have no processing to do (they are not involved in Request-related processing), so they perform sleep control (including CPU operating frequency control and CPU idle state control) while there is no processing. Power saving is achieved by performing sleep control (including CPU operating frequency control and CPU idle state control).

[0053] ACC12 performs calculations on the offloaded workload and notifies the ResponseI / O unit (CPU#m) 160 via a hardware interrupt when the calculation is complete (see symbol bb in Figure 2). In other words, the ResponseI / O unit (CPU#m) 160, which is a CPU core for response processing (a functional unit dedicated to responses), detects the completion of the calculation by ACC12 via an interrupt. The Response-related processing unit 140 and the ResponseI / O unit (CPU#m) 160 wake up via a hardware interrupt. Without interrupting the processing in progress (process 2) (because the Request-related processing unit 130 and the RequestI / O unit (CPU#n) 150 are exclusively processing it), the Response-related processing unit 140 and the ResponseI / O unit (CPU#m) 160 execute post-processing for process 1 via an interrupt, and then perform sleep control after the post-processing for process 1.

[0054] Here, the Response-related processing unit 140 and the Response I / O unit (CPU#m) 160 can immediately execute the post-processing for process 1 upon interruption because the Request-related processing unit 130 and the Request I / O unit (CPU#n) 150 are dedicated to terminating process 2. The Response-related processing unit 140 and the Response I / O unit (CPU#m) 160 perform sleep control until they are woken up by the next hardware interrupt.

[0055] Since there is no interruption to ongoing processing due to interrupts, and there is no need to save intermediate data (a problem with conventional interrupt methods), interrupt overhead can be suppressed by avoiding data saving processes.

[0056] At this time, the Request-related processing unit 130 and the Request I / O unit (CPU#n) 150 do not participate in Response-related processing (they are dedicated to the Response-related processing unit 140 and the Response I / O unit (CPU#m) 160), so the application thread (CPU) can request the next process (process 2) to be offloaded to ACC 12 (see the symbol cc in Figure 2). While the application thread (CPU) is offloading to ACC, it can execute other processes, but since there is no process to execute here, it performs sleep control. The Request-related processing unit 130 and the Request I / O unit (CPU#n) 150 achieve power saving by performing sleep control.

[0057] The reason there are no interruptions to the processing in progress described above is that by assigning Response-related processing to the Response-related processing unit 140 and the Response I / O unit (CPU#m) 160, and allocating those resources to the Request-related processing unit 130 and the Request I / O unit (CPU#n) 150, the efficiency of the Request-related processing unit 130 and the Request I / O unit (CPU#n) 150 is increased.

[0058] The Response I / O unit (CPU#m) 160 detects the completion of the calculation by ACC12 via an interrupt (see the symbol dd in Figure 2). The Response-related processing unit 140 and the Response I / O unit (CPU#m) 160 are woken up by a hardware interrupt. The Response-related processing unit 140 and the Response I / O unit (CPU#m) 160 then execute post-processing for process 2 via the interrupt.

[0059] [flowchart] The operation of the accelerator offload device 100 will be explained with reference to the flowcharts in Figures 3 to 5.

[0060] <Preparation Phase> Figure 3 is a flowchart showing the <preparation phase> by the control unit 110 of the accelerator off-road device 100. In step S1, the management unit 110 of the accelerator offloading device 100 (FIG. 1) preliminarily secures a CPU core group that each functional unit (Request-related processing unit 130, Response-related processing unit 140, Request I / O unit (CPU#n) 150, Response I / O unit (CPU#m) 160) may use. Here, the operator may determine in advance how to use the CPU cores so that other applications do not use the above CPU core group.

[0061] In step S2, the management unit 110 determines the CPU cores used by each functional unit (Request-related processing unit 130, Response-related processing unit 140, Request I / O unit (CPU#n) 150, Response I / O unit (CPU#m) 160) from the CPU core group and ends the processing of this flow.

[0062] An example of determining the CPU cores used by each functional unit from the CPU core group will be described. For example, it is determined that the Request-related processing unit 140 uses CPU#n-a to #n-1, the Request I / O unit 150 uses CPU#n, the Response I / O unit 160 uses CPU#m, the Response-related processing unit 140 uses CPU#m-b to #m-1, etc. The above a and b are constants. According to the processing load by the Request-related processing unit 140 and the Response-related processing unit 140, when the processing is heavy, a and b are increased to set a larger number of available CPU cores for the Request-related processing unit 130 and the Response-related processing unit 140.

[0063] <Request-side processing> FIGS. 4 and 5 are flowcharts showing the <actual processing phase> of the accelerator offloading device 100. The <actual processing phase> includes the <Request-side processing> in FIG. 4 and the <Response-side processing> in FIG. 5. FIG. 4 is a flowchart showing the <Request-side processing> of the accelerator offloading device 100. In step S11, when a task that needs to be offloaded to the ACC occurs, the task scheduler 120 (FIG. 1) of the accelerator offloading device 100 registers the task in the task queue of the Request related processing unit 130 (see FIG. 6 for the task allocation image). At this time, a CPU core different from the CPU core used by the Response I / O unit 160 is registered as the task to be used.

[0064] Here, when the Request related processing unit 130 (FIG. 1) is sleeping, the task scheduler 120 wakes it up. In addition, when the CPU operating frequency of CPU core #n used by each processing unit has decreased, the task scheduler 120 increases it, and when the CPU idle state is in the power saving mode, it transitions to the non - power saving mode.

[0065] In step S12, after the Request related processing unit 130 of the accelerator offloading device 100 executes a series of processes necessary until the ACC offloading, it notifies the Request I / O unit 150 of the ACC offloading request. Here, when the CPU cores used by the Request I / O unit 150 and the Request related processing unit 130 are different, the Request related processing unit 130 may be put to sleep at this timing.

[0066] In step S13, the Request I / O unit 150 (FIG. 1) of the accelerator offloading device 100 notifies the ACC12 of the offloading request (at this time, the target workload is copied via the ring buffer 13). Transition to <Request - side processing> (FIG. 5).

[0067] In step S14, the Request I / O unit (CPU#n) 150 determines whether there is a task in the task queue of the Request related processing unit 130.

[0068] If there is no task in the task queue of the Request-related processing unit 130 (S14: No), in step S15, the sleep control unit 151 of the Request I / O unit (CPU#n) 150 puts the Request-related processing unit 130 to sleep and ends the processing of this flow. At this time, for further power saving, the CPU operating frequency of the used CPU core #n may be decreased, and the CPU idle state may be set to the power saving mode.

[0069] If there is a task in the task queue of the Request-related processing unit 130 (S14: Yes), proceed to step S12.

[0070] <Response-side processing> FIG. 5 is a flowchart showing the <Response-side processing> of the accelerator offload device 100. In step S21, ACC12 (FIG. 1) calculates the offloaded workload, notifies the Response I / O unit 160 of the completion of the calculation by hardware interrupt, and stores the calculation result in the ring buffer 13. At this time, the raising destination of the hardware interrupt is specified as a CPU core different from the CPU core used by the Request-related processing unit 130.

[0071] If the processing performed by the Response-related processing unit 140 is heavy and a new ACC offload process is completed before the processing of the Response-related processing unit 140 ends, a hardware interrupt may be raised for a different CPU core, and the Request I / O unit 150 and the Request-related processing unit 130 may be processed in a multi-threaded manner. Also, when the interrupt frequency increases / decreases, scale-out / in may be performed by changing the interrupt destination CPU core.

[0072] In step S22, the sleep control unit 161 (Figure 1) of the ResponseI / O unit (CPU#m) 160 receives an interrupt and wakes up the ResponseI / O unit (CPU#m) 160. The sleep control unit 161 increases the CPU operating frequency of the CPU core #m used by the ResponseI / O unit (CPU#m) 160 if it has decreased, and transitions the CPU idle state from power-saving mode to non-power-saving mode if it is in power-saving mode.

[0073] In step S23, the Response I / O unit (CPU#m) 160 of the accelerator offload device 100 notifies the Response-related processing unit 140 that the ACC calculation is complete and transmits pointer information to the corresponding area of ​​the ring buffer 13 where the calculation result is stored. If the Request I / O unit (CPU#n) 150 and the Request-related processing unit 130 use different CPU cores, the Request I / O unit 150 may be put to sleep at this time.

[0074] In step S24, the Response-related processing unit 140 uses the calculation result of ACC12 to execute the calculation described in APL1. Here, the Response-related processing unit 140 may perform processing on the CPU core #m used by the Response I / O unit (CPU#m) 160, or it may perform processing on a different core (see Figure 8 below).

[0075] In step S25, if there are no other tasks to process, the sleep control unit 121 of the task scheduler 120 puts the Response I / O unit (CPU#m) 160 and the Response-related processing unit 140 to sleep, thereby terminating the processing of this flow.

[0076] The sleep control unit 121 may, in order to further reduce power consumption, lower the CPU operating frequency of the CPU core #m being used and / or set the CPU idle state to a power saving mode. Here, the sleep control of the ResponseI / O unit 160, the CPU operating frequency setting, and the CPU idle state setting may be performed during step S23 described above.

[0077] [Example of task assignment in Task Scheduler 120] This section explains an example of task assignment using Task Scheduler 120. Figure 6 illustrates an example of task assignment by the task scheduler 120. In the figure, the basket-shaped image next to the task scheduler 120 and the Request-related processing unit 130 schematically represents a task queue. This task queue schematically shows tasks to be offloaded to ACC12.

[0078] The task scheduler 120 distributes (schedules) tasks according to the availability of tasks in the task queue of the Request-related processing unit 130. At this time, round-robin may be used, or tasks may be distributed in order of the number of remaining tasks in the Request-related processing unit 130.

[0079] If tasks are registered simultaneously to multiple Request-related processing units 130, the multiple Request-related processing units 130 will complete their tasks simultaneously, and the processing will be transmitted to the Request I / O unit (CPU#n) 150 simultaneously. If there are multiple ACC offload processing units, the ACC offload processing will be completed simultaneously, and hardware interrupts will be triggered simultaneously to the Response I / O unit (CPU#m) 160. This could result in hardware interrupts occurring during processing by the Response I / O unit (CPU#m) 160. Therefore, it may be advisable to deliberately stagger the timing of task distribution from the task scheduler 120 to the Request-related processing units 130.

[0080] [Example of a case where post-response processing is heavy] This section explains an example of a case where post-response processing is computationally intensive. Figure 7 is a schematic diagram of a modified example of the accelerator offload system according to the first embodiment of the present invention. Figure 7 is a schematic diagram of the accelerator offload system that addresses the case where the response post-processing is heavy. Components identical to those in Figure 1 are denoted by the same reference numerals and their descriptions are omitted. As shown in Figure 7, the accelerator off-road system 1000A has an accelerator off-road device 100A. The accelerator offload device 100A comprises a management unit 110A, a task scheduler 120, a request-related processing unit 130, a response-related processing unit 140, a request I / O unit (CPU#n) 150, and a response I / O unit (CPU#m) 160. In addition to the functions of the management unit 110 of the accelerator offload device 100 shown in Figure 1 (see Figure 3), the management unit 110A has the function of assigning multiple CPU cores to the response-related processing unit 140.

[0081] The operation of the accelerator off-road device 100A of the accelerator off-road system 1000A, configured as described above, will be explained below. Figure 8 is a schematic timeline illustrating the operation of the accelerator off-road device 100A shown in Figure 7. As shown in Figure 8, if the post-processing by the Response-related processing unit 140 is heavy and takes a long time, a hardware interrupt may be received from ACC12 while the Response-related processing unit 140 is performing post-processing, potentially causing interrupt overhead such as memory saving (see symbols bb and dd in Figure 8). An example of a case where the post-processing by the Response-related processing unit 140 is heavy is when the Response I / O unit 160 and the Response-related processing unit 140 are using the same CPU core.

[0082] In the accelerator offload device 100A shown in Figure 7, the management unit 110A assigns multiple CPU cores to the response-related processing unit 140 during the <preparation phase> in Figure 3. In Figure 8, multiple CPU cores (CPU#m-1), (CPU#m-2) are assigned to the response-related processing unit 140. The Response I / O unit (CPU#m) 160 is dedicated to receiving (mediating) hardware interrupts from ACC12, and when the Response I / O unit (CPU#m) 160 receives a hardware interrupt (see symbol bb in Figure 8), it transfers the post-processing (post-processing of process 1) to the Response-related processing unit (CPU#m-1) 140 (see symbol dd in Figure 8). The Response-related processing unit (CPU#m-1) 140 exclusively executes the transferred post-processing (post-processing of process 1). Assuming that this post-processing (post-processing of process 1) requires a long time, the Response-related processing unit (CPU#m-1) 140 does not (cannot) execute any other processing.

[0083] At this time, the Request-related processing unit 130 and the Request I / O unit (CPU#n) 150 are not involved in Response-related processing (they are dedicated to the Response-related processing unit 140 and the Response I / O unit (CPU#m) 160), so the application thread (CPU) can request the next process (process 2) to be offloaded to ACC12 (see the symbol cc in Figure 8).

[0084] The Response I / O unit (CPU#m) 160 is dedicated to receiving (mediating) hardware interrupts from ACC12, and when the Response I / O unit (CPU#m) 160 receives a hardware interrupt (see symbol ee in Figure 8), it transfers the post-processing (post-processing of process 2) to the Response-related processing unit (CPU#m-2) 140 (see symbol ff in Figure 8). The Response-related processing unit (CPU#m-2) 140 exclusively executes the transferred post-processing (post-processing of process 2). At this time, the Response-related processing unit (CPU#m-1) 140 is exclusively executing (post-processing of process 1).

[0085] In Figure 8, an example is shown where post-processing is transferred from Response-related processing unit (CPU#m-1) 140 to Response-related processing unit (CPU#m-2) 140. However, it is also possible to transfer the post-processing to the Response-related processing unit 140 configured by load balancing using load balancing.

[0086] This allows the accelerator offload device 100A to avoid a situation where the Response I / O unit (CPU#m) 160, which receives the hardware interrupt, is performing other processing when a hardware interrupt occurs.

[0087] (Second Embodiment) In the first embodiment, a configuration was described in which a thread for request-related processing and a thread for response-related processing are executed on separate threads (separate CPU cores). A configuration in which request and response processing are handled by the same thread (CPU core) is also possible, and will be explained in the second embodiment.

[0088] Figure 9 is a schematic diagram of the accelerator off-road system according to a second embodiment of the present invention. Components identical to those in Figure 1 are denoted by the same reference numerals and their descriptions are omitted. As shown in Figure 9, the accelerator off-road system 1000B includes an accelerator off-road device 100B. The accelerator offload device 100B comprises a management unit 210, a task scheduler 220, a Request / Response related processing unit 230 (Request related processing unit 230, Response related processing unit 230), and a Request / Response I / O unit 250 (Request / Response I / O unit (CPU#n) 250, Request / Response I / O unit (CPU#n+1) 250). The Request / Response I / O unit (CPU#n) and the Request / Response I / O unit (CPU#n+1) have the same configuration, and the same number is assigned to both Request / Response I / O unit (CPU#n)250 and Request / Response I / O unit (CPU#n+1)250. The task scheduler 220 has a sleep control unit 221. The Request / Response I / O unit (CPU#n) and Request / Response I / O unit (CPU#n+1) 250 each have a sleep control unit 251.

[0089] The management unit 210 pre-allocates a group of CPU cores that each functional unit (Request / Response related processing unit 230, Request / Response I / O unit (CPU#n) 250, Request / Response I / O unit (CPU#n+1) 250) may use. The operator may pre-determine how the CPU cores will be used so that other applications do not use the aforementioned group of CPU cores. The management unit 210 determines which CPU core each functional unit (Request / Response related processing unit 230, Request / Response I / O unit (CPU#n) 250, Request / Response I / O unit (CPU#n+1) 250) will use.

[0090] When a task requiring offloading to ACC12 occurs, the task scheduler 220 registers the task in the task queue of the Request / Response related processing unit 230 (see Figure 6 for an image of task distribution). The task scheduler 220 distributes and assigns tasks to multiple CPU cores, taking into account the timing of receiving calculation results from ACC12. To maintain a state where there are no pending processes when a hardware interrupt is received for receiving processing results from ACC12, the task scheduler 220 registers the Request / Response I / O unit (CPU#n+1) 250 as a task that uses a different CPU core than the one used by the Request / Response I / O unit (CPU#n) 250. The sleep control unit 221 of the task scheduler 220 wakes up the Request-related processing unit 130 if it is sleeping.

[0091] The Request / Response related processing unit 230 executes a series of processes necessary before ACC offloading, and then notifies the Request / Response I / O unit (CPU#n) 250 and Request / Response I / O unit (CPU#n+1) 250 of the request for ACC offloading. The Request / Response related processing unit 230 uses the calculation result of ACC12 to execute the calculation described in APL1.

[0092] Request / Response I / O unit (CPU#n)250 and Request / Response I / O unit (CPU#n+1)250 notify ACC12 of the offload request. At this time, Request / Response I / O unit (CPU#n)250 copies the target workload via ring buffer 13.

[0093] Request / Response I / O unit (CPU#n)250 and Request / Response I / O unit (CPU#n+1)250 wake up upon receiving an interrupt. At this time, the task scheduler 220 increases the CPU operating frequency of CPU core #m used by Request / Response I / O unit (CPU#n+1)250 if it has decreased, and transitions the CPU idle state from power-saving mode to non-power-saving mode if it is in power-saving mode. The Request / Response I / O unit (CPU#n) 250 and the Request / Response I / O unit (CPU#n+1) 250 notify the Request / Response related processing unit 230 that the calculation of ACC12 is complete and transmit the pointer information of the corresponding area in the ring buffer 13 where the calculation result is stored.

[0094] The operation of the accelerator off-road device 100B of the accelerator off-road system 1000B, configured as described above, will be explained below. The task scheduler 220 shown in Figure 9 distributes and assigns tasks to multiple CPU cores, taking into account the timing of receiving calculation results from ACC12. For example, the task scheduler 220 assigns process 1 to CPU#n and process 2 to CPU#n+1, which is a different CPU core from CPU#n. The task scheduler 220 distributes tasks across multiple CPU cores to maintain a state where there are no pending processes when a hardware interrupt is received from ACC for receiving processing results. In this case, process 1 is handled by the Request / Response I / O unit (CPU#n) 250, and process 2 is handled by the Request / Response I / O unit (CPU#n+1) 250.

[0095] As shown in Figure 9, APL1 requests the Request / Response related processing unit 230 of the accelerator offload device 100B to offload processing (process 1). After the Request / Response related processing unit 230 performs a series of processes necessary before ACC offloading, it notifies the Request / Response I / O unit (CPU#n) 250 of the request for ACC offloading. The Request / Response I / O unit (CPU#n) 250 notifies ACC12 of the offload request (see symbol gg in Figure 9). At this time, the Request / Response I / O unit (CPU#n) 250 copies the target workload via the ring buffer 13.

[0096] ACC12 performs calculations on the offloaded workload, notifies the Request / Response I / O unit (CPU#n) 250 via a hardware interrupt when the calculation is complete (see code hh in Figure 9), and stores the calculation result in the ring buffer 13.

[0097] The task scheduler 220 distributes and assigns tasks to multiple CPU cores, taking into account the timing of receiving calculation results from ACC12. The task scheduler 120 registers the Request / Response I / O unit (CPU#n+1) 250 as a task to maintain a state where there are no pending processes when a hardware interrupt occurs upon receiving processing results from ACC12.

[0098] APL1 requests the Request / Response related processing unit 230 of the accelerator offload device 100B to offload the next process (process 2). After the Request / Response related processing unit 230 executes a series of processes necessary before ACC offloading, it notifies the Request / Response I / O unit (CPU#n+1) 250 of the request for ACC offloading. The Request / Response I / O unit (CPU#n+1) 250 notifies ACC12 of the offload request (see symbol ii in Figure 9). At this time, the Request / Response I / O unit (CPU#n+1) 250 copies the target workload via the ring buffer 13.

[0099] ACC12 performs calculations on the offloaded workload, notifies the Request / Response I / O unit (CPU#n+1) 250 via a hardware interrupt when the calculation is complete (see the symbol jj in Figure 9), and stores the calculation result in the ring buffer 13.

[0100] Figure 10 is a schematic timeline illustrating the operation of the accelerator off-road device 100B shown in Figure 9. Task Scheduler 220 registers the task as one that uses the Request / Response I / O unit (CPU#n) 250. As shown in Figure 10, the Request-related processing unit 230 and the Request / Response I / O unit (CPU#n) 250 request the ACC 12 to offload process 1, which is currently being executed by the application thread (CPU) (see the symbol gg in Figure 10). While the application thread (CPU) is offloading to ACC, it can execute other processes (in this case, process 2).

[0101] At this time, the Request-related processing unit 230 and the Request / Response I / O unit (CPU#n) 250 have no processing to do (they are not involved in Request-related processing), so they perform sleep control (including CPU operating frequency control and CPU idle state control) while there is no processing to do.

[0102] ACC12 computes the offloaded workload and notifies the Request / Response I / O unit (CPU#n) 250 via a hardware interrupt that the computation is complete (see symbol hh in Figure 10). The Request-related processing unit 230 and the Request / Response I / O unit (CPU#n) 250 are woken up by the hardware interrupt. The Request-related processing unit 230 and the Request / Response I / O unit (CPU#n) 250 execute post-processing for process 1 upon interruption, and after the post-processing for process 1, they perform the next process (process 3).

[0103] Task Scheduler 220 registers the Request / Response I / O unit (CPU#n+1) 250, which is a different CPU core from CPU#n, as a task to maintain a state where there are no pending processes when a hardware interrupt is received after receiving the processing result from ACC12.

[0104] APL1 requests the Request-related processing unit 230 of the accelerator offload device 100B to offload the next process (process 2) (see symbol ii in Figure 10). The application thread (CPU) can execute other processes (in this case, process 2) while ACC offloading is in progress. Process 2 will be handled by the Request / Response I / O unit (CPU#n+1) 250.

[0105] At this time, the Request-related processing unit 230 and the Request / Response I / O unit (CPU#n+1) 250 have no processing to do (they are not involved in Request-related processing), so they perform sleep control (including CPU operating frequency control and CPU idle state control) while there is no processing to do.

[0106] ACC12 computes the offloaded workload and notifies the Request / Response I / O unit (CPU#n+1) 250 via a hardware interrupt that the computation is complete (see symbol jj in Figure 10). The Request-related processing unit 230 and the Request / Response I / O unit (CPU#n) 250 wake up via a hardware interrupt. The Request-related processing unit 230 and the Request / Response I / O unit (CPU#n) 250 execute post-processing for process 2 via the interrupt, and then perform sleep control after the post-processing for process 2.

[0107] In the first embodiment, the thread that performs Request-related processing and the thread that performs Response-related processing are executed on separate threads (separate CPU cores). In this case, in order to maintain a state where there are no processes in progress when a hardware interrupt is received to receive processing results from ACC, the task scheduler 120 (Figure 1) distributes and assigns tasks to multiple CPU cores, taking into account the timing of receiving calculation results from ACC. CPU#n must wait until the result of process 1 is returned from ACC, and during this time the CPU cannot be used for other processes 2, 3, ... Therefore, CPU resource efficiency may deteriorate. On the other hand, since a state of sleep can be reliably created when a Response is received, there is no conflict with other tasks when receiving a Response, and scheduling processing to avoid conflicts is not required, which has the advantage of simplifying the implementation on the application side.

[0108] The accelerator offload device 100B of this embodiment includes a task scheduler 220 that distributes and assigns tasks to multiple CPU cores, taking into account the timing of receiving calculation results from ACC12, and a Request / Response related processing unit 230 and a Request / Response I / O unit (CPU#n) 250, Request / Response I / O unit (CPU#n+1) 250 that handle Request and Response processing in the same thread (CPU core).

[0109] This prevents a deterioration in CPU resource efficiency caused by the CPU becoming unavailable for the other processes mentioned above.

[0110] [flowchart] The operation of the accelerator offload device 100B will be explained with reference to the flowcharts in Figures 11 to 13.

[0111] <Preparation Phase> Figure 11 is a flowchart showing the <preparation phase> by the control unit 210 of the accelerator off-road device 100B. In step S31, the management unit 210 (Fig. 9) of the accelerator offloading device 100B preliminarily secures a group of CPU cores that each functional unit (Request / Response related processing unit 230, Request / Response I / O unit (CPU#n) 250, Request / Response I / O unit (CPU#n + 1) 250) may use. Here, the operator may determine in advance how to use the CPU cores so that other applications do not use the above CPU core group.

[0112] In step S32, the management unit 210 determines the CPU cores used by the Request / Response related processing unit 230, Request / Response I / O unit (CPU#n) 250, and Request / Response I / O unit (CPU#n + 1) from the group of CPU cores, and ends the processing of this flow.

[0113] An example of determining the CPU cores used by each functional unit from the group of CPU cores will be described. For example, it is determined that the Request / Response related processing unit 230 uses CPU#n - a to #n - 1, the Request / Response I / O unit (CPU#n) 250 uses CPU#n, the Request / Response I / O unit (CPU#n + 1) 250 uses CPU#n - 1, etc. The above a is a constant. Depending on the weight of the processing, when the processing is heavy, a is increased to set a larger number of CPU cores available for the Request / Response related processing unit 230.

[0114] <Task Scheduler Processing / Request Side Processing> Figs. 12 and 13 are flowcharts showing the <Actual Processing Phase> of the accelerator offloading device 100B. The <Actual Processing Phase> includes the <Task Scheduler Processing / Request Side Processing> in Fig. 12 and the <Response Side Processing> in Fig. 13. Fig. 12 is a flowchart showing the <Task Scheduler Processing / Request Side Processing> of the accelerator offloading device 100B. In step S41, the task scheduler 220 (Figure 9) of the accelerator offload device 100B registers a task in the task queue of the Request / Response related processing unit 230 if a task requiring offloading to the ACC occurs (see Figure 6 for an image of task distribution). The task scheduler 220 distributes and assigns tasks to multiple CPU cores, taking into account the timing of receiving calculation results from ACC12.

[0115] Here, the task scheduler 220 wakes up the Request / Response related processing unit 230 (Figure 9) if it is sleeping. Furthermore, Task Scheduler 220 increases the CPU operating frequency of the CPU cores used by each processing unit if it has decreased, and transitions the CPU idle state from power-saving mode to non-power-saving mode if it is in power-saving mode.

[0116] In step S42, the Request / Response related processing unit 230 of the accelerator offload device 100B performs a series of processes necessary before ACC offload, and then notifies the Request / Response related processing unit 230 of the request for ACC offload. Here, if the Request / Response related processing unit 230 uses a different CPU core than the Request / Response I / O unit (CPU#n) 250 and the Request / Response I / O unit (CPU#n+1) 250, the Request / Response related processing unit 230 may be put to sleep at this time. Furthermore, to further save power, the CPU operating frequency of the CPU core being used may be lowered, and / or the CPU idle state may be set to a power-saving mode.

[0117] In step S43, the Request / Response I / O unit (CPU#n) 250 and the Request / Response I / O unit (CPU#n + 1) 250 of the accelerator offloading device 100B notify the offloading request to ACC12 (at this time, copy the target workload via the ring buffer 13). Transition to <Request side processing> (Fig. 13).

[0118] In step S44, the sleep control units 251 of the Request / Response I / O unit (CPU#n) 250 and the Request / Response I / O unit (CPU#n + 1) 250 put the Request / Response related processing unit 230 to sleep and end the processing of this flow. At this time, for further power saving, the CPU operating frequency of the used CPU core may be lowered and / or the CPU idle state may be set to the power saving mode.

[0119] <Response side processing> Fig. 13 is a flowchart showing the <Response side processing> of the accelerator offloading device 100B. In step S51, ACC12 (Fig. 9) calculates the offloaded workload, notifies the Request / Response I / O unit 250 of the completion of the calculation via hardware interrupt, and stores the calculation result in the ring buffer 13.

[0120] In step S52, the sleep control units 251 of the Request / Response I / O unit (CPU#n) 250 and the Request / Response I / O unit (CPU#n + 1) 250 wake up the Request / Response I / O unit (CPU#n) 250 and the Request / Response I / O unit (CPU#n + 1) 250 if they are sleeping. Also, the sleep control unit 251 raises the CPU operating frequency of the used CPU core if it has been lowered, and transitions the CPU idle state from the power saving mode to the non-power saving mode if it is in the power saving mode.

[0121] In step S53, the Request / Response I / O unit (CPU#n) 250 and the Request / Response I / O unit (CPU#n+1) 250 notify the Request / Response related processing unit 230 that the calculation of ACC12 is complete and transmit the pointer information of the corresponding area in the ring buffer 13 where the calculation result is stored. In this case, if the Request / Response related processing unit 230 and the Request / Response I / O unit (CPU#n) 250 and Request / Response I / O unit (CPU#n+1) 250 use different CPU cores, the Request / Response I / O unit (CPU#n) 250 and Request / Response I / O unit (CPU#n+1) 250 may be put to sleep. Furthermore, to further reduce power consumption, you may lower the CPU operating frequency of the CPU cores being used and / or set the CPU idle state to a power-saving mode.

[0122] In step S54, the sleep control unit 221 of the task scheduler 220 wakes up the Request / Response related processing unit 230 if it is sleeping. The sleep control unit 221 also increases the CPU operating frequency of the CPU core being used if it has decreased, and transitions the CPU idle state from power saving mode to non-power saving mode if it is in power saving mode.

[0123] In step S55, the Request / Response related processing unit 230 uses the calculation result of ACC12 to perform the calculation described in APL1.

[0124] In step S56, if there are no other tasks to process, the sleep control unit 221 of the task scheduler 220 puts the Request / Response related processing unit 230 to sleep and terminates the processing of this flow. Furthermore, the sleep control unit 221 may lower the CPU operating frequency of the CPU core being used and / or set the CPU idle state to a power saving mode in order to further reduce power consumption.

[0125] [Task Scheduler extensions] This section discusses the extended features of the Task Scheduler. Task schedulers 120 and 220 (Figures 1 and 9) may, in either the first or second embodiment, assign tasks in a manner that maximizes the sleep time of each CPU core so that each CPU core can sleep for as long as possible. In other words, CPUs often have a CPU idle state control function such as C-state, and can transition to a power-saving state by controlling voltage, frequency, etc., through hardware control. This CPU idle state control function can transition to a deeper sleep state if the sleep time is longer. Therefore, by optimizing the assignment to maximize the sleep time, a greater power saving effect can be expected.

[0126] Furthermore, to prevent a thread from being in the middle of other processes when receiving calculation results from ACC12, the timing of receiving calculation results from ACC12 can be inferred from past performance using machine learning, and scheduling can be done using the inference results. For example, the FEC processing time in vRAN varies depending on the data size and error rate. Therefore, by learning this, it is possible to estimate the time from sending a request to ACC to receiving a response.

[0127] Task schedulers 120 and 220 (Figures 1 and 9) may, in either the first or second embodiment, scale out when the number of tasks increases, by scheduling to increase the number of CPU cores used. Conversely, they may scale in when the number of tasks decreases, by scheduling to decrease the number of CPU cores used.

[0128] [Hardware configuration] The accelerator off-road devices 100, 100A, and 100B according to each of the above embodiments are implemented by a computer 900 having a configuration such as that shown in Figure 14. Figure 14 is a hardware configuration diagram showing an example of a computer 900 that implements the functions of accelerator offload devices 100, 100A, and 100B. Computer 900 includes a CPU 901, RAM 902, ROM 903, HDD 904, accelerator 905, input / output interface (I / F) 906, media interface (I / F) 907, and communication interface (I / F) 908. Accelerator 905 corresponds to accelerator (ACC) 12 in Figures 1, 7, and 9.

[0129] Accelerator 905 is an accelerator (device) 12 (Figures 1, 7, and 9) that processes at high speed data from either the communication interface 908 or the RAM 902. Note that accelerator 905 may be of a type that returns the execution result to the CPU 901 or RAM 902 after processing from the CPU 901 or RAM 902 (look-aside type). Alternatively, accelerator 905 may be of an in-line type that acts as an intermediary between the communication interface 908 and the CPU 901 or RAM 902 to perform processing.

[0130] The accelerator 905 is connected to the external device 915 via the communication interface 908. The input / output interface 906 is connected to the input / output device 916. The media interface 907 reads and writes data to the recording medium 917.

[0131] The CPU 901 operates based on a program stored in the ROM 903 or HDD 904, and controls the respective parts of the accelerator offload devices 100, 100A, and 100B shown in Figures 1, 7, and 9 by executing the program (also called an application or app) loaded into the RAM 902. This program can also be distributed via a communication line or by recording it on a recording medium 917 such as a CD-ROM. ROM903 stores boot programs executed by CPU901 when the computer 900 starts up, as well as programs that depend on the computer 900's hardware.

[0132] The CPU 901 controls the input / output device 916, which consists of an input unit such as a mouse or keyboard, and an output unit such as a display or printer, via the input / output interface 906. The CPU 901 acquires data from the input / output device 916 via the input / output interface 906 and outputs generated data to the input / output device 916. In addition to the CPU 901, a GPU (Graphics Processing Unit) or the like may also be used as a processor.

[0133] HDD904 stores programs executed by CPU901 and data used by those programs. Communication I / F908 receives data from other devices via a communication network (e.g., NW (Network)) and outputs it to CPU901, and also transmits data generated by CPU901 to other devices via the communication network.

[0134] The media I / F 907 reads the program or data stored in the recording medium 917 and outputs it to the CPU 901 via the RAM 902. The CPU 901 loads the program related to the target process from the recording medium 917 onto the RAM 902 via the media I / F 907 and executes the loaded program. The recording medium 917 is an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable Disk), a magneto-optical recording medium such as a MO (Magneto Optical disk), a magnetic recording medium, a conductor memory tape medium, or a semiconductor memory, etc.

[0135] For example, when the computer 900 functions as the accelerator offload devices 100, 100A, 100B configured as one device according to this embodiment, the CPU 901 of the computer 900 realizes the functions of the accelerator offload devices 100, 100A, 100B by executing the program loaded onto the RAM 902. Also, the data in the RAM 902 is stored in the HDD 904. The CPU 901 reads and executes the program related to the target process from the recording medium 917. In addition, the CPU 901 may read the program related to the target process from another device via the communication network.

[0136] [Application Example] The accelerator offload devices 100, 100A, 100B only need to be arranged on the user space 200, and the OS is not limited. Also, it is not limited to being in a server virtualization environment. Therefore, the accelerator offload systems 1000 to 1000B can be applied to each configuration shown in FIGS. 15 and 16.

[0137] <Example of Application to VM Configuration> FIG. 15 is a diagram showing an example in which the accelerator offload system 1000C is applied to an interrupt model in a server virtualization environment of a general-purpose Linux kernel (registered trademark) and a VM configuration. The same components as those in FIGS. 1, 7, and 9 are denoted by the same reference numerals. As shown in Figure 15, the accelerator offload system 1000C includes hardware (HW) 10, Host OS 50, hypervisor 60, VM 70, Guest OS 80, high-speed data communication unit 40 which is high-speed data transfer middleware located on user space 200, and accelerator offload device 100. Note that accelerator offload device 100 is an example, and may be accelerator offload devices 100A, 100B (Figures 7 and 9).

[0138] In detail, the server comprises a Host OS 50 on which virtual machines and external processes formed outside the virtual machines can run, a hypervisor 60, a VM 70 with a virtual interface 71, and a Guest OS 80 running within the virtual machine. The Host OS 50 has a ring buffer 13 managed by the kernel in the server's memory space.

[0139] In the accelerator offload system 1000C, the accelerator offload device 100 is located on user space 200. Therefore, like DPDK, the accelerator offload device 100 can bypass the kernel and access the ring-structured buffer. In other words, the accelerator offload device 100 does not use the ring buffer (ring buffer 13) or pole list (not shown) within the kernel. The accelerator offload device 100 can bypass the kernel and access a ring-structured buffer (ring buffer 13), enabling it to instantly detect packet arrivals (i.e., it uses a polling model rather than an interrupt model).

[0140] As shown in Figure 15, the Request-related processing unit 130 and Request I / O unit (CPU#n) 150 of the accelerator offload device 100 request ACC12 to offload process 1, which is being executed by the application thread (CPU), via the virtual IF71 of VM70 (see code kk in Figure 15). ACC12 performs calculations on the offloaded workload and notifies the virtual IF71 of VM70 of the completion of the calculations via a hardware interrupt (see code ll in Figure 15). The virtual IF71 of VM70 notifies the Response I / O unit (CPU#m) 160 via a hardware interrupt. In other words, the Response I / O unit (CPU#m) 160, which is a CPU core for response processing (a functional unit dedicated to responses), detects the completion of the calculations by ACC12 via an interrupt.

[0141] By doing so, in a VM virtual server configuration system, the notification interrupt for ACC offload results can be applied to the APL1 located on the guest OS, regardless of whether it is the Host OS50 or Guest OS80.

[0142] <Example of application to container configuration> Figure 16 shows an example of applying the accelerator offload system 1000D to the interrupt model in a containerized server virtualization environment. The same reference numerals are used for components identical to those in Figure 15. As shown in Figure 16, the accelerator offload system 1000D includes a Guest OS 50 and a container configuration in which the OS is replaced by a container 90. The container 90 has a virtual IF 91. The accelerator offload device 100 is located on the user space 200.

[0143] As shown in FIG. 16, the Request-related processing unit 130 and the Request I / O unit (CPU#n) 150 of the accelerator offloading device 100 offload the process 1 being executed by the application thread (CPU) to the ACC 12 via the virtual IF 91 of the container 90 (see reference numeral mm in FIG. 16). The ACC 12 calculates the offloaded workload and notifies the virtual IF 91 of the container 90 of the completion of the calculation by hardware interrupt (see reference numeral nn in FIG. 16). The virtual IF 91 of the container 90 notifies the Response I / O unit (CPU#m) 160 by hardware interrupt. That is, the completion of the calculation by the ACC 12 is detected by the Response I / O unit (CPU#m) 160, which is a CPU core for Response processing (a function unit dedicated to Response) by interrupt.

[0144] By doing so, in a container-configured system, it is also possible to apply the notification interrupt of the ACC offloading result when notifying the APL 1 arranged on the container.

[0145] <Application Example to Pair Metal Configuration (Non-Virtualized Configuration)> The present invention can be applied to a non-virtualized configuration system such as a bare metal configuration (FIGS. 1, 7, 9).

[0146] <CPU Pinning When Using Hyper Threading> When using the Hyper Threading technology that logically creates a plurality of CPU cores from one physical CPU core, for the logical core CPU cores #n1 and #n2 created on the physical CPU core #n, the Request-related processing unit 130 (FIG. 1) and the Request I / O unit 150 (FIG. 1) are assigned to the CPU core #n1, and the Response-related processing unit 140 (FIG. 1) and the Response I / O unit 160 (FIG. 1) are assigned to the CPU core #n2, so that the cache hit rate may be improved.

[0147] Since CPU cores #n1 and #n2 use the same physical CPU core, they may share either the L1 or L2 cache. In this case, by assigning the Request and Response processes to use the same physical core, it becomes possible to increase the cache hit rate when there is data that is used in common by both the Request and Response processes.

[0148] [effect] As described above, the accelerator offload device 100 offloads specific processing of an application program (APL1) to an accelerator (ACC12), and comprises: a Request-related processing unit 130 that executes predetermined processing necessary before offloading to the accelerator and notifies the Request I / O unit 150 of the offload request; a Request I / O unit 150 composed of a CPU core that executes Request processing to notify the accelerator of the offload request; a Response I / O unit 160 composed of a CPU core separate from the CPU core that executes Response processing to notify the Response-related processing unit 140 of the completion of the accelerator's calculation; and a Response-related processing unit 140 that uses the accelerator's calculation results to execute calculations described in the application program.

[0149] By doing this, when offloading workloads that the CPU is not good at to ACC, the overhead caused by hardware interrupts when receiving ACC calculation results can be suppressed, thereby achieving reduced processing time (lower latency) and improved CPU utilization efficiency.

[0150] The accelerator offload device 100 is equipped with a management unit 110 that manages a group of CPU cores consisting of multiple CPU cores, and the management unit 110 determines which CPU core to use from among the group of CPU cores, with the Request-related processing unit 130, Response-related processing unit 140, Request I / O unit 150, or Response I / O unit 160 being the one to use.

[0151] By doing so, the Request-related processing unit 130, the Response-related processing unit 140, the Request I / O unit 150, or the Response I / O unit 160 can determine which CPU core to use from among the CPU core group, thereby improving CPU utilization efficiency.

[0152] In the accelerator offload device 100, the management unit 110 is characterized by assigning one CPU core to the Response I / O unit 160 as a function unit dedicated to Response from among the CPU core group.

[0153] By doing so, CPU utilization efficiency can be improved by allocating at least one CPU core to the Response I / O unit 160 as a dedicated Response function from among the CPU core group.

[0154] The accelerator offload device 100 has a task scheduler 120 that registers a task to the task queue of the Request-related processing unit 130 when a task requiring offloading to the accelerator (ACC12) occurs, and the task scheduler 120 is characterized in that it registers the task as one that uses a CPU core different from the one used by the Response I / O unit 160.

[0155] This approach prevents a deterioration in CPU resource efficiency caused by the CPU becoming unavailable for other processes. Furthermore, it avoids situations where the Response I / O unit (CPU#m) 160, which receives hardware interrupts, is performing other tasks when post-Response processing is heavy or when a hardware interrupt occurs, thereby shortening processing time and improving CPU utilization efficiency.

[0156] The accelerator offload device 100 is characterized in that the task scheduler 120 includes a sleep control unit that puts threads running on the CPU to sleep when there are no tasks to perform calculations on the CPU.

[0157] By doing this, significant power savings can be achieved by putting the system to sleep when there is no processing to be done. Furthermore, even greater power savings can be achieved by controlling the CPU operating frequency and CPU idle state.

[0158] The accelerator offload system 1000 is an accelerator offload system that includes an accelerator offload device for offloading specific processing of an application program to an accelerator, the accelerator offload device 100 being located on user space 200. Hardware 10 having an accelerator (ACC12) is equipped with a ring buffer 13 for copying the target workload, and the Request I / O unit 150 and Response I / O unit 160 exchange data with the accelerator via the ring buffer 13.

[0159] By doing so, in data communication between APL1 and ACC12, the number of unnecessary memory copies is reduced via the ring buffer 13, thereby achieving further reductions in processing time (lower latency) and improved CPU utilization efficiency.

[0160] Furthermore, among the processes described in each of the above embodiments, all or part of the processes described as being performed automatically may be performed manually, or all or part of the processes described as being performed manually may be performed automatically by known methods. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above documents and drawings may be changed at will unless otherwise specified. Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions.

[0161] Furthermore, each of the above configurations, functions, processing units, and processing means may be implemented in hardware, either partially or entirely, by designing them as integrated circuits, for example. Alternatively, each of the above configurations and functions may be implemented in software that allows the processor to interpret and execute programs that implement each function. Information such as programs, tables, and files that implement each function can be stored in memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC (Integrated Circuit) card, an SD (Secure Digital) card, or an optical disc. [Explanation of Symbols]

[0162] 1. APL (Application Program) 10 Hardware 11 CPU11 12 ACC (Accelerator) 13 ring buffer 40 High-speed data communication section 100, 100A, 100B Accelerator Off-Road Device 110,210 Management Department 120,220 Task Scheduler 130 Request-related processing unit 140 Response-related processing unit 150 Request I / O section (CPU#n) 160 Response I / O section (CPU#m) 121,151,161,221,251 Sleep Control Unit 200 user space 230 Request / Response related processing unit (Request related processing unit, Response related processing unit) 250 Request / Response I / O section (CPU#n), Request / Response I / O section (CPU#n+1) (Request I / O section, Response I / O section) 1000~1000D Accelerator Off-Road System

Claims

1. An accelerator offload device that offloads specific processing of an application program to an accelerator, A Request-related processing unit that performs predetermined processing necessary before the accelerator is offloaded and notifies the Request I / O unit of the request for offloading, The Request I / O unit, which consists of a CPU core, executes Request processing to notify the accelerator of an offload request, A Response I / O unit, which is composed of a CPU core separate from the aforementioned CPU core, and which performs a Response process that notifies the Response-related processing unit of the completion of the accelerator's calculations via a hardware interrupt, The system comprises a Response-related processing unit that uses the calculation results of the accelerator to perform calculations described in the application program. An accelerator off-road device characterized by the following.

2. It includes a management unit that manages a group of CPU cores consisting of multiple CPU cores, The management unit determines which CPU core to be used by the Request-related processing unit, the Response-related processing unit, the Request I / O unit, or the Response I / O unit from among the group of CPU cores. The accelerator off-road device according to feature 1.

3. The management unit assigns one of the CPU cores from the group of CPU cores to the Response I / O unit as a function unit dedicated to Response. The accelerator off-road device according to feature 2.

4. The system includes a task scheduler that registers tasks to the task queue of the Request-related processing unit when tasks requiring offloading to the accelerator occur. The task scheduler registers the task as one that uses a different CPU core than the one used by the Response I / O unit. The accelerator off-road device according to feature 1.

5. The task scheduler includes a sleep control unit that puts threads running on the CPU to sleep if there are no tasks to perform calculations on the CPU. The accelerator off-road device according to feature 4.

6. An accelerator offload system comprising an accelerator offload device that offloads specific processing of an application program to an accelerator, The aforementioned accelerator offload device is located in user space. A Request-related processing unit that performs predetermined processing necessary before the accelerator is offloaded and notifies the Request I / O unit of the request for offloading, The Request I / O unit, which consists of a CPU core, executes Request processing to notify the accelerator of an offload request, A Response I / O unit, which is composed of a CPU core separate from the aforementioned CPU core, and which performs a Response process that notifies the Response-related processing unit of the completion of the accelerator's calculations via a hardware interrupt, The system comprises a Response-related processing unit that uses the calculation results of the accelerator to perform calculations described in the application program. An accelerator off-road system characterized by the following.

7. The hardware having the accelerator includes a ring buffer for copying the target workload, The Request I / O unit and the Response I / O unit exchange data with the accelerator via the ring buffer. The accelerator off-road system according to feature 6.

8. An accelerator offloading method for an accelerator offloading device that offloads specific processing of an application program to an accelerator, The aforementioned accelerator off-road device is The steps include: executing the necessary processes for the accelerator to go off-road and notifying the request for off-road operation; The steps include: executing a Request process using a CPU core to notify the accelerator of an offload request; The steps include: executing a response process, which notifies the completion of the accelerator's calculation via a hardware interrupt, using a CPU core other than the CPU core; The steps of executing the calculation described in the application program using the calculation result of the accelerator are performed. An accelerator off-road method characterized by the following.

Images