Accelerator offloading apparatus and accelerator offloading method

Abstract

This accelerator offloading apparatus (100) comprises: a traffic collection unit (140) that collects traffic information; a device mimicking unit (110) that conceals switching of connection destinations such that it appears that an application is communicating with the accelerator; an offload destination determination unit (130) that determines an offload destination on the basis of the collected traffic information and provides offload destination information for the application; an offload destination connection unit (120) that connects the device mimicking unit (110) and the accelerator at the offload destination in accordance with the provided offload destination information; and a device control unit (150) that, in accordance with indications from the offload destination determination unit (130), handles the electric power of devices including the accelerator.

Description

Accelerator offloading device and accelerator offloading method

[0001] The present invention relates to an accelerator offload device and an accelerator offload method.

[0002] Depending on the type of processor, different workloads are suited to different tasks (high processing power). While general-purpose central processing units (CPUs) are capable of quickly and efficiently processing highly parallel workloads that CPUs are not good at (low processing power), accelerators (hereinafter referred to as ACCs) such as field programmable gate arrays (FPGAs), graphics processing units (GPUs), and application-specific integrated circuits (ASICs) are available. By combining these heterogeneous processors and offloading workloads that CPUs are not good at (low processing power) to the ACCs, offloading technology is being increasingly utilized to improve overall processing time and efficiency.

[0003] In a virtual radio access network (vRAN) or the like, when the performance of a CPU alone is insufficient to meet requirements, some processing is offloaded to an accelerator capable of high-speed calculations, such as an FPGA or a GPU. Typical examples of specific workloads for which ACC offloading is performed include encoding / decoding processing (FEC: Forward Error Correction processing) in a vRAN, media processing of audio and video, and encryption / decryption processing.

[0004] Additionally, there are data transfer technologies within a server such as New API (NAPI), DPDK (Data Plane Development Kit), and KBP (Kernel Busy Poll). KBP constantly monitors packet arrivals using a polling model within the kernel, thereby suppressing softIRQs and achieving low-latency packet processing.

[0005] When a packet arrives, the New API (NAPI) processes the packet using a software interrupt request after issuing a hardware interrupt request.

[0006] DPDK implements packet processing functions in the user space where applications run, and immediately harvests packets when they arrive from the user space using a polling model (see Non-Patent Document 1). Specifically, DPDK is a framework for controlling NICs (Network Interface Cards), a function traditionally performed by the Linux kernel (registered trademark), in user space. The biggest difference from the processing in the Linux kernel is the inclusion of a polling-based reception mechanism called PMD (Pull Mode Driver). Typically, in the Linux kernel, an interrupt occurs when data arrives at the NIC, triggering the execution of reception processing. On the other hand, in the PMD, a dedicated thread continuously checks data arrival and performs reception processing. High-speed packet processing is possible by eliminating overhead such as context switches and interrupts. DPDK significantly improves packet processing performance and throughput, allowing more time to be allocated to data plane application processing. However, DPDK exclusively uses computer resources such as the CPU (Central Processing Unit) and NIC.

[0007] Next, the DPDK system will be described. [DPDK System Configuration] Fig. 27 is a diagram showing the configuration of a DPDK system that controls HW 10 having accelerators 11 and 21. The DPDK system has HW 10, packet processing APIs (Application Programming Interfaces) 14 and 24, and applications (APL) 1 and APL 2.

[0008] APL1 and APL2 are packet processes that are performed prior to the execution of APL.

[0009] The packet processing APIs 14 and 24 are APIs used when offloading packet processing to a NIC or accelerator. The packet processing APIs 14 and 24 are high-speed data transfer middleware and DPDK located in user space. DPDK realizes packet processing functions in the user space where APL1 and APL2 run, and enables packet transfer delays to be reduced by immediately reaping packets when they arrive from the user space using a polling model. That is, DPDK retrieves packets by polling (a CPU busy polls the queue) (referring to the contents of the packets stored in the buffer, processing the packets, and deleting the corresponding queue entry from the buffer in consideration of the next processing), so there is no waiting and delays are small.

[0010] The HW 10 communicates with the APL 1 and the APL 2 to transmit and receive data. In the following description, the data flow in which the APL 1 and the APL 2 receive packets from the HW 10 is referred to as Rx-side reception, and the data flow in which the APL 1 and the APL 2 transmit packets to the HW 10 is referred to as Tx-side transmission.

[0011] The HW 10 includes accelerators (ACC) 11 and 21. The HW 10 may also include a NIC (physical NIC) for connecting to a communication network. The accelerators 11 and 21 are computational unit hardware that performs specific calculations at high speed based on input from a CPU. Specifically, the accelerator 11 is a programmable logic device (PLD) such as a GPU or FPGA. In FIG. 27 , the accelerators 11 and 21 include multiple intellectual property cores (IP cores) 12 and 22, and device queues 13 and 23, which are physical queues consisting of an Rx queue (waiting queue) and a Tx queue that hold data in a first-in, first-out list structure. The IP cores 12 and 22 are design information for reusable circuit components that constitute semiconductors such as FPGAs, ICs, and LSIs.

[0012] Part of the processing of APL1 and APL2 is offloaded to the accelerators 11 and 21, achieving performance and power efficiency that cannot be achieved by software (CPU processing) alone. Cases in which the accelerator 11 described above is applied are envisioned, such as in large-scale server clusters, such as data centers that implement NFV (Network Functions Virtualization) and SDN (Software Defined Network).

[0013] Existing applications (APL1, APL2) that transfer data to an accelerator in poll mode, such as DPDK, operate by being fixedly linked to device queues 13 and 23 used by the application at the time of initialization (see the dashed-line box in FIG. 27). The application thread (hereinafter referred to as the application thread) performs transmission and reception processing via a ring buffer (not shown) corresponding to the accelerator 11. Here, the application thread is a polling thread.

[0014] BBDEV API (connection information on page 5), [online], [searched June 6, 2022], Internet <http: / / fast.dpdk.org / events / slides / DPDK-2017-09-BBdev.pdf>

[0015] 27 transmits requests to accelerators 11 and 21 through a packet processing API (Application Programming Interface) and receives the requests after the processing is completed in poll mode or interrupt mode. The accelerator resources to be transmitted and received by the threads are fixedly allocated in units of device queues 13 and 23 when APL1 and APL2 are initialized (the device queue 13 of APL1 and the accelerator 11, and the device queue 23 of APL2 and the accelerator 21 are fixedly allocated).

[0016] However, in a system like vRAN where traffic volume fluctuates significantly throughout the day, preparing multiple accelerators to handle peak traffic can result in excessive resources when traffic is low, such as during the night. The accelerators do not necessarily consume power in proportion to the incoming traffic, resulting in increased power consumption.

[0017] The problems of the conventional technology will be described with reference to Figures 28 and 29. Figure 28 is a diagram showing the traffic volume (upper diagram of Figure 28) and ACC power (lower diagram of Figure 28) when the traffic volume fluctuates greatly throughout the day. Figure 29 is a diagram showing the daytime resources and nighttime resources of Figure 28. The solid line blocks in Figure 29 indicate the ACC maximum allowable traffic, and the shaded areas in Figure 29 indicate the ACC traffic.

[0018] If multiple accelerators are provided to accommodate peak traffic, resources are appropriate during the daytime, indicated by arrow a in Fig. 28, as shown in the upper diagram of Fig. 29, but resources become excessive during the nighttime, indicated by arrow b in Fig. 28, as shown in the lower diagram of Fig. 29. Furthermore, accelerators consume a certain amount of power while they are being configured (formed), and this power has almost no effect on the accelerator's usage. Therefore, when traffic is low, such as during the nighttime, there is an issue of excessive resources, resulting in increased power consumption (see arrow c in Fig. 28).

[0019] The present invention has been made in light of this background, and aims to reduce the power consumption of an accelerator without causing degradation in application performance due to the reduction in accelerator power consumption, while eliminating the need to add a function to support power saving in the application.

[0020] In order to solve the above-mentioned problems, an accelerator offload device has a plurality of accelerators and offloads specific processing of an application to the accelerators, and is characterized by comprising: a traffic collection unit that collects current traffic information and / or predicted traffic information of the processing to be offloaded; a device simulation unit that conceals connection destination switching from the perspective of the application so that it appears that the application is communicating with an accelerator; an offload destination determination unit that determines an offload destination based on the traffic information collected by the traffic collection unit and provides offload destination information for the application; an offload destination connection unit that connects the device simulation unit to the offload destination accelerator in accordance with the offload destination information provided by the offload destination determination unit; and a device control unit that performs power operation to turn on or off the power of a device including the accelerator in accordance with instructions from the offload destination determination unit.

[0021] According to the present invention, it is possible to reduce the power consumption of an accelerator without causing a degradation in the performance of an application due to the reduction in the power consumption of the accelerator, while eliminating the need to add a function to support power saving in the application.

[0022] 1 is a schematic configuration diagram of a power-saving accelerator management system according to an embodiment of the present invention. 2 is a diagram showing a device mapping table of an accelerator offload device according to an embodiment of the present invention. 3 is a diagram showing an application management table of an accelerator offload device according to an embodiment of the present invention. 4 is a diagram showing a task table of an accelerator offload device according to an embodiment of the present invention. 5 is a diagram showing a traffic table of an accelerator offload device according to an embodiment of the present invention. 6 is a diagram showing a device management table of an accelerator offload device according to an embodiment of the present invention. 7 is a diagram showing an operation explanatory diagram of dynamic change of an offload destination accelerator and accelerator sharing at the changed destination (feature <1>) of an accelerator offload device according to an embodiment of the present invention. 8 is a diagram explaining the operation of <excessive resources> and <insufficient resources> in the power-saving accelerator management system of FIG. 7. 9 is a control sequence explaining ACC scale-in in feature <1> of a power-saving accelerator management system according to an embodiment of the present invention. 10 is a diagram showing a flowchart of ACC scale-in of an offload destination determination unit in the control sequence of FIG. 9. 11 is a diagram showing a schematic diagram of ACC scale-in when the scale-in target ACC in the power-saving accelerator management system according to an embodiment of the present invention is ACC2. 12 is a control sequence of <ACC scale-in derivation in feature <1>> of a power-saving accelerator management system according to an embodiment of the present invention. 14 is a control sequence illustrating ACC scale-out in feature <1> of the power-saving accelerator management system according to an embodiment of the present invention. FIG. 15 is a diagram illustrating a flowchart of ACC scale-out of the offload destination determination unit in the control sequence of FIG. 13. FIG. 16 is a diagram illustrating ACC scale-out when the scale-out target ACC of the power-saving accelerator management system according to an embodiment of the present invention is ACC1. FIG. 17 is a control sequence of <ACC scale-out derivation in feature <1>> of the power-saving accelerator management system according to an embodiment of the present invention. FIG. 18 is an operational explanatory diagram illustrating reconfiguration of the offload destination (feature <2>) taking into account optimal placement of the power-saving accelerator management system according to an embodiment of the present invention.24 is a diagram illustrating "power-off ACC redetermination" and "task load balancing" in feature <2> of the power-saving accelerator management system according to an embodiment of the present invention. FIG. 25 is a control sequence illustrating the reconfiguration of the offload destination taking into account optimal placement in feature <2> of the power-saving accelerator management system according to an embodiment of the present invention. FIG. 26 is a flowchart illustrating the reconfiguration of the offload destination taking into account optimal placement in the offload destination determination unit in the control sequence of FIG. 19. FIG. 27 is a schematic diagram illustrating the reconfiguration of the offload destination taking into account optimal placement in feature <2> of the power-saving accelerator management system according to an embodiment of the present invention. FIG. 28 is an operational explanatory diagram illustrating a scale trigger (feature <3>) based on traffic and performance information in the power-saving accelerator management system according to an embodiment of the present invention. FIG. 29 is a diagram illustrating ACC scale-in / scale-out taking into account "excess resources" and "insufficient resources" in feature <3> of the power-saving accelerator management system according to an embodiment of the present invention. FIG. 29 is a control sequence illustrating the scale trigger based on traffic and performance information in feature <3> of the power-saving accelerator management system according to an embodiment of the present invention. FIG. 29 is a flowchart illustrating a scale trigger based on traffic and performance information A in the configuration change trigger unit in the control sequence of FIG. 24. 27 is a hardware configuration diagram showing an example of a computer that realizes the functions of an accelerator offload device according to an embodiment of the present invention. 28 is a diagram showing the configuration of a DPDK system that controls HW equipped with an accelerator. 29 is a diagram showing traffic volume and ACC power when the traffic volume fluctuates greatly within a day. 29 is a diagram showing a schematic diagram of daytime resources and nighttime resources in FIG. 28.

[0023] A power-saving accelerator management system and the like according to an embodiment of the present invention (hereinafter referred to as the "present embodiment") will be described below with reference to the drawings. [Overall Configuration] FIG. 1 is a schematic diagram of a power-saving accelerator management system according to an embodiment of the present invention. Components identical to those in FIG. 27 are designated by the same reference numerals. In FIG. 1, outline arrows represent data flows, and thin arrows represent control signals. As shown in FIG. 1, the power-saving accelerator management system 1000 includes an APL 30 having multiple APLs 1 and 2 and threads 15 and 25, an accelerator offload device 100, and accelerators (ACCs) 11 and 21 having multiple IP cores 12 and 22 and device queues 13 and 23. The accelerator offload device 100 is disposed between the APL 30 and the accelerators 11 and 21 installed in the hardware. The APL 30 directly communicates with the offload device 100, so it can see the presence of the offload device 100, but it cannot see the physical devices 11 and 21.

[0024] [Accelerator Offload Device 100] The accelerator offload device 100 includes a device simulation unit 110, an offload destination connection unit 120, an offload destination determination unit 130, a traffic collection unit 140, a device control unit 150, an offload destination reconfiguration unit 160, a performance acquisition unit 170, and a configuration change trigger unit 180.

[0025] Here, the device simulation unit 110, offload destination connection unit 120, offload destination determination unit 130, traffic collection unit 140, and device control unit 150 constitute a functional unit 101 for dynamically changing the offload destination accelerator and sharing the accelerator at the changed destination (Feature <1>). The offload destination reconfiguration unit 160 constitutes an offload destination reconfiguration unit 102 that takes optimal placement into consideration (Feature <2>). The performance acquisition unit 170 and configuration change trigger unit 180 constitute a scale trigger unit 103 based on traffic and performance information (Feature <3>).

[0026] <Device Simulation Unit 110> The device simulation unit 110 conceals the switching of the connection destination from the perspective of the application, so that it appears as if APL1, 2 (application) is communicating with the accelerator.

[0027] <Offload Destination Connection Unit 120> The offload destination connection unit 120 connects the device simulation unit 110 and the offload destination accelerator in accordance with the offload destination information provided by the offload destination determination unit 130 (in accordance with the offload destination determined by the offload destination determination unit 130).

[0028] <Offload Destination Determination Unit 130> The offload destination determination unit 130 determines an offload destination based on traffic information collected by the traffic collection unit 140 and provides offload destination information for an application. The offload destination determination unit 130 also instructs the device control unit 150 to control the power supply of the ACC. When traffic decreases, the offload destination determination unit 130 switches the offload destination of a task on one accelerator to the other accelerator, and the device control unit 150 performs accelerator scale-in, which turns off the power supply of the accelerator.

[0029] <Traffic Collector 140> The traffic collector 140 collects current traffic information and / or predicted traffic forecast information (see FIG. 22 below) for the offload target process, and provides the collected information to the offload destination determiner 130.

[0030] <Device control unit 150> The device control unit 150 performs power operations to turn on or off the power of devices including accelerators in accordance with instructions from the offload destination determination unit 130. The device control unit 150 performs power operations to turn on the power of surplus accelerators, and the offload destination determination unit 130 performs accelerator scale-out, switching the offload destination of some of the tasks on an accelerator with performance concerns to that accelerator.

[0031] <Offload Destination Reconfiguration Unit 160> The offload destination reconfiguration unit 160 recalculates the offload destination based on the traffic information collected by the traffic collection unit 140, and provides the offload destination reconfiguration information to the offload destination determination unit 130. Specifically, the offload destination reconfiguration unit 160 initializes the current offload destinations of APLs 1 and 2 based on the traffic information obtained from the traffic collection unit 140, determines the offload destinations that should exist from scratch, and provides the offload destination determination unit 130 with the offload destination reconfiguration information.

[0032] Here, the offload destination reconfiguration unit 160 may be provided independently in an external server. By providing the offload destination reconfiguration unit 160 independently in an external server, the offload destination reconfiguration unit 160 can be applied to a RAN Intelligent Controller (RIC) in a RAN, etc.

[0033] In the case of accelerator scale-in, if an accelerator that consumes more power than an accelerator that has been powered off due to accelerator scale-in can be powered off, the offload destination reconfiguration unit 160 provides offload destination reconfiguration information to the offload destination determination unit 130 to cause the offload destination to be modified to that accelerator. Upon receiving this information, the offload destination determination unit 130 modifies the offload destination of the task based on the offload destination reconfiguration information, and the device control unit 150 powers off the accelerator with the larger power.

[0034] In the case of task load balancing, the offload destination reconfiguration unit 160 recalculates the offload destination based on the traffic information collected by the traffic collection unit 140, and if task load balancing is possible for each of the minimum required ACCs, provides offload destination reconfiguration information to the offload destination determination unit 130 to correct the offload destination of the task. Upon receiving this information, the offload destination determination unit 130 corrects the offload destination of the task based on the offload destination reconfiguration information.

[0035] <Performance Acquisition Unit 170 > The performance acquisition unit 170 acquires performance information including application delay and throughput (performance information for each of APL1 and APL2), and notifies the configuration change trigger unit 180 of this information.

[0036] <Configuration Change Trigger Unit 180> The configuration change trigger unit 180 determines whether to issue an accelerator scale-in or accelerator scale-out command based on the traffic information collected by the traffic collection unit 140 and the performance information acquired by the performance acquisition unit 170, and triggers the offload destination determination unit 130 to issue an accelerator scale-in or accelerator scale-out command, or triggers the offload destination reconfiguration unit 160 to issue an offload destination reconfiguration command.

[0037] The configuration change trigger unit 180 determines whether to scale out based on the performance of each of APL1 and APL2. If scale out is not necessary, the configuration change trigger unit 180 requests traffic information from the traffic collection unit 140.

[0038] Here, the performance acquisition unit 170 and the configuration change trigger unit 180 may be provided independently in an external server. By providing the performance acquisition unit 170 and the configuration change trigger unit 180 independently in an external server, they can be applied to RIC in a RAN, etc.

[0039] [Tables] The following describes the tables possessed by each functional unit of the accelerator offload device 100. Fig. 2 is a diagram showing a device mapping table 201 that is referenced by the offload destination determination unit 130 and written by the offload destination connection unit 120. The device mapping table 201 associates application IDs with device IDs.

[0040] 3 is a diagram showing the application management table 202 that is referenced by the configuration change trigger unit 180 and written by the performance acquisition unit 170. The application management table 202 stores a performance value, a performance threshold, and an operating status for each APL ID. The performance threshold and performance value are, for example, delay times. The operating status is active / down, etc.

[0041] 4 is a diagram showing the task table 203, which is referenced by the offload destination determination unit 130 and the offload destination reconfiguration unit 160 and written by the traffic collection unit 140. The task table 203 stores traffic and traffic forecast values ​​for each APL ID.

[0042] 5 is a diagram showing a traffic table 204 that is referenced by the offload destination determination unit 130, the offload destination reconfiguration unit 160, and the configuration change trigger unit 180 and written by the traffic collection unit 140. The traffic table 204 stores traffic, traffic forecast values, traffic lower limit values, and traffic upper limit values ​​for each device ID.

[0043] 6 is a diagram showing the device management table 205, which is referenced by the offload destination determination unit 130 and the offload destination reconfiguration unit 160 and written by the device control unit 150. The device management table 205 stores the capacity, free space, and power status for each device ID. The power status indicates whether the accelerator is ON / OFF / degenerated, etc.

[0044] The operation of the energy-saving accelerator management system configured as described above is explained below. (Features and Operational Overview) Feature <1>: Dynamic change of offload destination accelerator and sharing of accelerator at changed destination <Requirement 1: Transparency> is met by switching the offload destination of tasks without changing the application. <Requirement 2: Energy saving> is met by performing power operations such as turning off the power of accelerators that no longer have offload tasks. <Requirement 3: Performance> is met by performing power operations such as turning on the power of accelerators when there are performance concerns and adding them to the offload destination.

[0045] Feature <2>: Reconfiguration of offload destinations with consideration for optimal placement Based on the current traffic volume, the system calculates which accelerator each application's tasks should be offloaded to in order to reduce power consumption while still maintaining performance, and determines the offload destination, thereby simultaneously satisfying <Requirement 2: Power saving> and <Requirement 3: Performance>.

[0046] Feature <3>: Scale trigger based on traffic and performance information By triggering a change in the offload destination in a timely manner according to traffic and performance values, <Requirement 2: Energy efficiency> and <Requirement 3: Performance> are simultaneously met.

[0047] [Dynamic change of offload destination accelerator and accelerator sharing at the changed destination (Feature <1>)] Dynamic change of offload destination accelerator and accelerator sharing at the changed destination (Feature <1>) will be described. FIG. 7 is an explanatory diagram of the operation of a power-saving accelerator management system to explain dynamic change of offload destination accelerator and accelerator sharing at the changed destination (Feature <1>). The same components as in FIG. 1 are assigned the same reference numerals. In the following description, functional units of the relevant operations are indicated by bold frames. Feature <1> is realized by the functional unit 101 shown in the dashed line in FIG. 7. Specifically, it is as follows.

[0048] <Operation in Feature <1> of Device Simulation Unit 110> The device simulation unit 110 conceals the change in connection destination by making it appear to APL1 and 2 that they are communicating with an accelerator. The device simulation unit 110 connects to APL1 and 2 using an interface equivalent to an existing interface and provides virtual queues (not shown) instead of the device's physical queues 13 and 23. The device simulation unit 110 corresponds to the device's physical queue and connects the device to APL1 and 2 by having application threads 15 and 25 perform packet processing using the virtual queues instead of the physical queues. The virtual queues are queues that are shown to application threads 15 and 25 instead of the physical queues (device queues) 13 and 23.

[0049] We do not want APL1 and APL2 to be aware of whether the actual specific device is accelerator 11 or accelerator 21, but want to hide the specific device behind it by having them communicate with an abstract accelerator. The device simulation unit 110 connects APL1 and APL2 with an interface equivalent to that of an existing accelerator, and provides a virtual queue instead of the device's physical queues 13 and 23. This makes it appear as if APL1 and APL2 are always communicating with the accelerator.

[0050] <Operation of Offload Destination Connection Unit 120 in Feature <1>> The offload destination connection unit 120 connects the device simulation unit 110 to the accelerator of the offload destination in accordance with the offload destination determined by the offload destination determination unit 130 .

[0051] <Operation of Traffic Collector 140 in Feature <1>> The traffic collector 140 collects the current traffic of the ACC resource and provides it to the offload destination determiner 130 (see arrow aa in FIG. 7).

[0052] <Operation of offload destination determination unit 130 in feature <1>> The offload destination determination unit 130 determines an offload destination based on traffic information from the traffic collection unit 140, and provides offload destination information for the application to the offload destination connection unit 120 (see arrow bb in FIG. 7 ). The offload destination determination unit 130 instructs the device control unit 150 to control the power supply of the ACC (see arrow cc in FIG. 7 ).

[0053] <Operation of Device Control Unit 150 in Feature <1>> The device control unit 150 performs power operations such as turning on or off the power of devices in accordance with instructions from the offload destination determination unit 130 .

[0054] Fig. 8 is a diagram illustrating the operation of <excessive resources> and <insufficient resources> in the power-saving accelerator management system of Fig. 7. The device simulation unit 110 and the offload destination connection unit 120 switch the offload destination of a task without changing the application, thereby satisfying <Requirement 1: transparency>.

[0055] 8, the thread of APL1, whose offload destination is the device queue 13, sends a request to the accelerator 11 (see arrow dd in FIG. 7), and receives the request after the processing is completed in poll mode or interrupt mode. Also, the thread of APL1, whose offload destination is the device queue 23 (see arrow ee in FIG. 7), sends a request to the accelerator 21, and receives the request after the processing is completed in poll mode or interrupt mode.

[0056] In the case of <excessive resources>, the offload destination determination unit 130 determines ACC scale-in, and the offload destination connection unit 120 switches the connection (see arrow ff in FIG. 7 ). The device control unit 150 also performs ACC power control (here, powering off the accelerator 21). As a result, the power-saving accelerator management system satisfies <Requirement 2: power saving>.

[0057] In the case of <insufficient resources>, the device control unit 150 performs ACC power control, the offload destination determination unit 130 determines ACC scale-out, and the offload destination connection unit 120 switches the connection. As a result, the power-saving accelerator management system satisfies <Requirement 3: Performance>.

[0058] Incidentally, conventionally, as shown in FIG. 27, accelerator resources to be transmitted and received by threads were fixedly allocated in units of device queues 13, 23, etc. at the time of initialization of APL1 and APL2 (the device queue 13 and accelerator 11 of APL1, and the device queue 23 and accelerator 21 of APL2 were fixedly allocated).

[0059] In contrast to this, in this embodiment, the offload destination accelerator is dynamically changed by performing ACC scale-in when there are excess resources and by performing ACC scale-out when there are insufficient resources, as shown in Fig. 8. In other words, the accelerator at the change destination is shared.

[0060] <ACC Scale-In in Feature <1>> ACC scale-in in feature <1> will be described with reference to the control sequence in FIG. 9 and the flowchart in FIG. 10. FIG. 9 shows a control sequence that explains ACC scale-in in feature <1>. The traffic collection unit 140 collects traffic for each ACC and notifies the offload destination determination unit 130 of the traffic information (step S101). The offload destination determination unit 130 determines the scale-in target ACC (the ACC to be powered off) and the new offload destination ACC for the tasks being offloaded to the ACC based on the traffic information for each ACC (step S102). The flowchart for ACC scale-in will be described later with reference to FIG. 10.

[0061] The offload destination determination unit 130 notifies the offload destination connection unit 120 of the scale-in target ACC information and the offload destination information of each task (step S103). The offload destination connection unit 120 updates the offload destination of each task (device mapping table 201 ( FIG. 2 )) based on the offload destination information of each task (step S104), and notifies the offload destination determination unit 130 of the completion of the update (step S105).

[0062] The offload destination connection unit 120 confirms that all tasks in the scale-in target ACC have been processed (step S106). After completion, the offload destination connection unit 120 notifies the offload destination determination unit 130 of the completion of all tasks (step S107). The offload destination determination unit 130 notifies the device control unit 150 of the scale-in target ACC information and a power-off instruction for the corresponding ACC (step S108). The device control unit 150 performs a power-off operation for the corresponding ACC (accelerator 11) based on the scale-in target ACC information (step S109). The device control unit 150 checks the power status of the corresponding ACC until it confirms that the power has been turned off (step S110). Once it confirms that the power has been turned off, the device control unit 150 updates the device management table 205 ( FIG. 6 ) and notifies the offload destination determination unit 130 that the power has been turned off (step S111).

[0063] 10 is a flowchart of ACC scale-in by the offload destination determination unit 130 in the control sequence of FIG. 9 . In step S11, the traffic collection unit 140 acquires the traffic volume for each ACC. In step S12, the offload destination determination unit 130 selects the scale-in target ACC with the largest free capacity. In other words, the ACC with the largest free capacity is assumed to be easiest to scale in. Here, the free capacity of the ACC is calculated by subtracting the total traffic volume of the tasks from the maximum traffic volume in a state where the performance of APL1 and APL2 is satisfied. Note that the number of offload tasks or physical capacity such as power margin and temperature margin may be used instead of or in addition to the free capacity of the ACC.

[0064] In step S13, the offload destination determination unit 130 selects the task with the largest traffic volume among the tasks in the scale-in target ACC. In step S14, the offload destination determination unit 130 determines whether the task can be offloaded to the ACC with the largest available capacity among the remaining ACCs. If the task cannot be offloaded to the ACC with the largest available capacity (S14: No), the process proceeds to step S19. If the task can be offloaded to the ACC with the largest available capacity (S14: Yes), the offload destination determination unit 130 changes the offload destination of the task in step S15.

[0065] In step S16, the offload destination determination unit 130 determines whether there is a task whose offload destination has not been changed. If there is a task whose offload destination has not been changed (S16: Yes), in step S17, the offload destination determination unit 130 selects the task with the next largest traffic volume and returns to step S14. If there is no task whose offload destination has not been changed (S16: No), the offload destination determination unit 130 executes ACC scale-in and ends the processing of this flow (step S18).

[0066] On the other hand, if offloading to the ACC with the largest free capacity is not possible in step S14 (S14: No), the offload destination determination unit 130 determines in step S19 whether or not there are other scale-in target ACCs. If there are other scale-in target ACCs (S19: Yes), in step S20 the offload destination determination unit 130 selects the scale-in target ACC with the next largest free capacity and returns to step S13. If there are no other scale-in target ACCs (S19: No), the processing of this flow ends (no ACC scale-in) (step S21).

[0067] FIG. 11 is a diagram schematically showing ACC scale-in when the scale-in target ACC is ACC2. The thin solid line blocks in FIG. 11 indicate the allowable traffic of ACC1, and the thick solid line blocks in FIG. 11 indicate the allowable traffic of ACC2. In FIG. 11, the allowable traffic (free capacity) of ACC1 is larger than the allowable traffic (free capacity) of ACC2, and is represented by the size of the block. Also, the shading in FIG. 11 indicates ACC traffic.

[0068] Figure 11<1> The allowable traffic of both ACC1 and ACC2 stores ACC traffic. However, ACC1 has a larger allowable traffic than ACC2. An ACC that is easy to scale in is an ACC with a large amount of free capacity. Therefore, ACC2 is selected as the ACC to be scaled in.

[0069] FIG. 11<2> Task b is selected from ACC2 as a task with a large traffic volume.

[0070] <3> in Fig. 11: ACC1 is selected as the offload destination of task b in ACC2. In <3> in Fig. 11, task b is offloaded to ACC1.

[0071] FIG. 11<4> Next, task a is selected from ACC2 as a task with a large traffic volume.

[0072] <5> in Figure 11: ACC1 is selected as the offload destination for task a of ACC2. In <5> in Figure 11, task a is offloaded to ACC1 in addition to task b. In this state, the allowable traffic of ACC1 does not have enough free capacity to accept a new task.

[0073] <6> in Figure 11: Since all tasks of ACC2 have been transferred, scale-in of ACC2 is now possible. ACC scale-in is executed. Here, ACC scale-in is performed by turning off the power of ACC2 (the power-off is indicated by the dashed line box).

[0074] <Derivation of ACC scale-in in feature <1>> Figures 9 to 11 have explained scale-in accompanied by power-off. Instead of scale-in accompanied by power-off, the following forms can also be taken. - If power saving by degenerate operation is possible on the ACC side, the offload destination of tasks that exceed the performance of the degenerate operation of the ACC is changed, and the ACC degenerate operation is further performed. Example (1): Reduction in the number of IP cores in FPGA and power control on an IP core basis Example (2): Reduction in circuit size by dynamic reconfiguration in FPGA Example (3): Reduction in voltage in ACC Example (4): Reduction in clock frequency in ACC

[0075] 12 shows the control sequence for <ACC scale-in derivation in feature <1>>. Fig. 12 corresponds to the control sequence for <ACC scale-in in feature <1>> in Fig. 9. The traffic collection unit 140 collects traffic for each ACC and notifies the offload destination determination unit 130 of the traffic information (step S201). The offload destination determination unit 130 determines the ACC to be subjected to degenerate operation and a new offload destination ACC for the task being offloaded to the ACC, based on the traffic information for each ACC (step S202).

[0076] The offload destination determination unit 130 notifies the offload destination connection unit 120 of the degenerate operation target ACC information and the offload destination information of each task (step S203). The offload destination connection unit 120 updates the offload destination of each task (device mapping table 201 ( FIG. 2 )) based on the offload destination information of each task (step S204), and notifies the offload destination determination unit 130 of the completion of the update (step S205).

[0077] The offload destination connection unit 120 confirms that all tasks of the ACC targeted for degenerate operation have been completed (step S206). After completion, the offload destination connection unit 120 notifies the offload destination determination unit 130 of the completion of all tasks (step S207). The offload destination determination unit 130 notifies the device control unit 150 of the information about the ACC targeted for degenerate operation and the degenerate operation for the ACC (step S208). The device control unit 150 performs degenerate operation for the ACC (accelerator 11) based on the information about the ACC targeted for degenerate operation (step S209). The device control unit 150 checks the power status of the ACC until it confirms degenerate operation (step S210). Once it confirms degenerate operation, the device control unit 150 updates the device management table 205 ( FIG. 6 ) and notifies the offload destination determination unit 130 of the completion of degenerate operation (step S211). This concludes the description of <ACC scale-in in feature <1>>. Next, <ACC scale-out in feature <1>> will be described.

[0078] <ACC Scale-Out in Feature <1>> ACC scale-out in feature <1> will be described with reference to the control sequence in FIG. 13 and the flowchart in FIG. 14. FIG. 13 shows a control sequence for describing ACC scale-out in feature <1>. The traffic collection unit 140 collects traffic for each ACC and notifies the offload destination determination unit 130 of the traffic information (step S301). The offload destination determination unit 130 determines the ACC to be scaled out (a portion of the tasks currently offloaded to the ACC is offloaded to a new offload destination) and the new offload destination ACC for the tasks currently offloaded to the ACC based on the traffic information for each ACC (step S302). The flowchart for ACC scale-out will be described later with reference to FIG. 14.

[0079] The offload destination determination unit 130 notifies the device control unit 150 of the scale-out destination ACC information and a power-on instruction (step S303). The device control unit 150 performs a power-on operation for the ACC based on the scale-out destination ACC information (step S304). The device control unit 150 checks the power status of the ACC until it confirms that the power is on (step S305). As soon as it confirms that the power is on, the device control unit 150 updates the device management table 205 ( FIG. 6 ) and notifies the offload destination determination unit 130 that the power has been turned on (step S306).

[0080] The offload destination determination unit 130 notifies the offload destination information of each task to the offload destination connection unit 120 (step S307). The offload destination connection unit 120 updates the offload destination of each task based on the offload destination information of each task (step S308), and notifies the offload destination determination unit 130 of the completion of the update (step S309).

[0081] 14 is a flowchart of ACC scale-out by the offload destination determination unit 130 in the control sequence of FIG. 13. In step S31, the traffic collection unit 140 acquires the traffic volume for each ACC. In step S32, the offload destination determination unit 130 selects the scale-out target ACC with the smallest free capacity. The free capacity of the ACC is calculated by subtracting the total traffic volume of the tasks from the maximum traffic volume that satisfies the performance of APL1 and APL2. Instead of or in addition to the free capacity of the ACC, the number of offload tasks or physical capacity such as power margin and temperature margin may be used.

[0082] In step S33, the offload destination determination unit 130 selects the ACC with the smallest capacity among the ACCs that are powered off as the scale-out destination ACC. In step S35, the offload destination determination unit 130 determines whether the task can be offloaded to the scale-out destination ACC. If the task cannot be offloaded to the scale-out destination ACC (S35: No), the process proceeds to step S41. If the task can be offloaded to the scale-out destination ACC (S35: Yes), in step S36 the offload destination determination unit 130 changes the offload destination of the task.

[0083] In step S37, it is determined whether or not there is a task for which the free capacity of the scale-out target ACC is smaller than the free capacity of the scale-out destination ACC and the offload destination has not been changed. If the free capacity of the scale-out target ACC is smaller than the free capacity of the scale-out destination ACC and there is a task for which the offload destination has not been changed (S37: Yes), in step S38, a task with the next lowest traffic volume is selected. In step S39, it is determined whether or not the task can be offloaded to the scale-out destination ACC. If the task can be offloaded to the scale-out destination ACC (S39: Yes), the process returns to step S36. In other words, if offloading is still possible, the ACC scale-out step is repeated.

[0084] If in step S37 above the free space of the scale-out target ACC is smaller than the free space of the scale-out destination ACC, or there are no tasks whose offload destination has not been changed (S37: No), or if in step S39 above the task in question cannot be offloaded to the scale-out destination ACC (S39: No), ACC scale-out is performed and processing of this flow is terminated (step S40).

[0085] On the other hand, if the task cannot be offloaded to the scale-out destination ACC in step S35 (S35: No), the offload destination determination unit 130 determines whether there is another powered OFF ACC in step S41. If there is another powered OFF ACC (S41: Yes), the offload destination determination unit 130 selects the powered OFF ACC with the next smallest free capacity in step S42 and returns to step S33. If there is no other powered OFF ACC (S41: No), the processing of this flow ends (no ACC scale-out) (step S43).

[0086] FIG. 15 is a diagram schematically illustrating ACC scale-out when the scale-out target ACC is ACC1. The thin dashed line blocks in FIG. 15 indicate the allowable traffic of ACC2, and the thin solid line blocks in FIG. 15 indicate the allowable traffic of ACC1. In FIG. 15, the allowable traffic (free capacity) of ACC1 is larger than the allowable traffic (free capacity) of ACC2, and is represented by the size of the block. Also, the shading in FIG. 15 indicates ACC traffic. Also, the dashed line box indicates that the power is OFF.

[0087] 15<1> Tasks a, b, c, and d are stored as traffic volumes in ACC1. ACC2 is powered off. Therefore, ACC1 is selected as the scale-out target ACC, and ACC2 is selected as the scale-out destination ACC.

[0088] FIG. 15<2> Task d is selected from ACC2 as a task with a small amount of traffic.

[0089] <3> in Fig. 15: ACC2 is selected as the offload destination of task d. In <3> in Fig. 15, task d is offloaded to ACC2.

[0090] FIG. 15<4> Next, task c is selected from ACC1 as a task with a small amount of traffic.

[0091] <5> in Fig. 15 ACC2 is selected as the offload destination of task c. In <3> in Fig. 15 ACC2 further offloads task d to task c.

[0092] 15<6> When an attempt is made to offload the next task, step S39 in Fig. 14 becomes NG and the process shifts to scale-out execution, so that scale-out execution of ACC2 becomes possible. ACC scale-out is executed.

[0093] <ACC scale-out derivation in feature <1>> Figures 13 to 15 have explained scale-out that involves power-on. Instead of scale-out that involves power-on, the following forms can also be taken. - If it is possible to improve performance on the ACC side by canceling degenerate operation, change the offload destination of tasks that can be handled by canceling degenerate operation of the ACC, and then cancel the degenerate operation of the ACC. Example (1): Increase in the number of IP cores in FPGA and power control on an IP core basis Example (2): Expansion of circuit size through dynamic reconfiguration in FPGA Example (3): Increase in voltage at ACC Example (4): Increase in clock frequency at ACC This concludes the explanation of <ACC scale-out in feature <1>>.

[0094] 16 shows the control sequence for <ACC scale-out derivation in feature <1>>. The traffic collection unit 140 collects traffic for each ACC and notifies the offload destination determination unit 130 of the traffic information (step S401). The offload destination determination unit 130 determines the ACC to be scaled out and the ACC for which degenerate operation is to be canceled (step S402).

[0095] The offload destination determination unit 130 sends the degenerate operation release ACC information and power-on instructions to the device control unit 150 (step S403). The device control unit 150 performs a degenerate operation release operation on the accelerator 11 and confirms the degenerate operation release (step S404). The device control unit 150 confirms the degenerate operation release to the accelerator 11 (step S405). The device control unit 150 notifies the offload destination connection unit 120 that the degenerate operation release has been completed (step S406). The offload destination determination unit 130 notifies the offload destination connection unit 120 of each task offload destination information (step S407). The offload destination connection unit 120 updates each task offload destination information based on the task offload destination information notified by the offload destination determination unit 130 (step S408). The offload destination connection unit 120 notifies the offload destination determination unit 130 that each task offload destination update has been completed (step S409). The above explains the ACC scale-out derivation in feature <1>.

[0096] [Reconfiguration of offload destination taking into consideration optimal placement (Feature <2>)] The following describes reconfiguration of offload destination taking into consideration optimal placement (Feature <2>). Reconfiguration of offload destination taking into consideration optimal placement (Feature <2>) aims to achieve power savings and performance improvements that cannot be achieved by simply scaling in and out as described in Feature <1> above.

[0097] FIG. 17 is an explanatory diagram of the operation of the power-saving accelerator management system to explain the reconfiguration of offload destinations taking into account optimal placement (feature <2>). The same components as in FIG. 1 are assigned the same reference numerals. In the following description, the functional units of the relevant operations are indicated by bold frames. Feature <2> is realized by the offload destination reconfiguration unit 102 taking into account optimal placement, shown in the dashed line area in FIG. 17. Specifically, it is as follows. The offload destination reconfiguration unit 160 shown in FIG. 17 initializes the current offload destinations of APLs 1 and 2 based on traffic information obtained from the traffic collection unit 140, determines the offload destinations to be established from scratch, and provides the offload destination reconfiguration information to the offload destination determination unit 130.

[0098] Reconfiguration of offload destinations with optimal placement in mind (Feature <2>) consists of the features of <power-off ACC redetermination> and <task load balancing>.

[0099] FIG. 18 is a diagram explaining feature <2>, <power OFF, ACC redetermination> and <task load balancing>. Pattern 1 in the upper diagram of FIG. 18 shows <power OFF, ACC redetermination>, and pattern 2 in the lower diagram of FIG. 18 shows <task load balancing>. The solid line blocks in FIG. 18 show the ACC maximum allowable traffic (free capacity), and the shaded areas in FIG. 18 show ACC traffic. The dashed line blocks in FIG. 18 show ACC power OFF.

[0100] <Power OFF ACC Re-determination> The offload destination reconfiguration unit 160 recalculates the offload destination based on the traffic information acquired by the traffic collection unit 140. If the offload destination reconfiguration unit 160 can power off an ACC with higher power consumption than an ACC that has already been powered off by ACC scale-in, it provides offload destination reconfiguration information to the offload destination determination unit 130 to correct the offload destination of the task to that ACC. Upon receiving this information, the offload destination determination unit 130 corrects the offload destination of the task. The device control unit 150 contributes to <Requirement 2: Power saving> by powering off an ACC with higher power consumption.

[0101] For example, in pattern 1 in the upper diagram of FIG. 18, ACC2 is powered off by ACC scale-in. ACC1 has a larger ACC maximum allowable traffic (free capacity) than ACC2. For this reason, powering off ACC1 by ACC scale-in can achieve greater power savings. The offload destination is recalculated based on the traffic information acquired by the traffic collection unit 140, and the offload destination is reconfigured. Here, all ACC traffic (tasks) of ACC1 is moved to ACC2. Then, as shown by the white arrow in the upper diagram of FIG. 18, the power-off ACC is redetermined, and ACC1 is powered off by ACC scale-in.

[0102] <Task Load Balancing> The traffic collection unit 140 recalculates the offload destination based on the acquired traffic information. If task load balancing is possible for each of the minimum required ACCs, the traffic collection unit 140 provides offload destination reconfiguration information to the offload destination determination unit 130 to modify the task offload destination. The offload destination determination unit 130 then modifies the task offload destination. This contributes to <Requirement 3: Performance> by reducing the risk of performance degradation when task load balancing increases traffic.

[0103] For example, in pattern 2 in the lower diagram of FIG. 18, ACC1 has a larger ACC maximum allowable traffic (free capacity) than ACC2. However, the ACC traffic (tasks) is almost the same for ACC1 and ACC2. Therefore, ACC1 has ample free capacity. If traffic increases in this state, ACC2 will have no free capacity, and requests may not be able to be processed due to waiting time or may be lost, leading to performance degradation. Therefore, as shown by the white arrows in the lower diagram of FIG. 18, load balancing is performed by transferring some of the tasks of ACC2 to ACC1, thereby reducing the risk of performance degradation when traffic increases.

[0104] <Reconfiguration of offload destination taking into consideration optimal placement in feature <2>> Reconfiguration of offload destination taking into consideration optimal placement in feature <2> will be described with reference to the control sequence in Fig. 19 and the flowchart in Fig. 20. Fig. 19 is a control sequence that explains reconfiguration of offload destination taking into consideration optimal placement in feature <2>. After the initial determination of the minimum ACC and the determination of the offload destination ACC for each task, the offload destination reconfiguration takes the form of a combination of ACC scale-in and ACC scale-out.

[0105] The traffic collection unit 140 collects traffic for each ACC and notifies the traffic information to the offload destination determination unit 130 (step S501). The offload destination determination unit 130 determines the minimum ACC and the offload destination ACC for the task (step S502).

[0106] The offload destination determination unit 130 instructs the device control unit 150 to notify the scale-out destination ACC information and to power on (step S503). The device control unit 150 performs a power-on operation on the accelerator 11 and confirms that the power has been turned on (step S504). The device control unit 150 notifies the offload destination determination unit 130 that power-on has been completed (step S506). The offload destination determination unit 130 notifies the offload destination connection unit 120 of the scale-in target ACC information and each task offload destination information (step S507).

[0107] The offload destination connection unit 120 updates each task offload destination information based on the task offload destination information notified by the offload destination determination unit 130 (step S508). The offload destination connection unit 120 notifies the offload destination determination unit 130 that each task offload destination update has been completed (step S509). The offload destination connection unit 120 confirms that all tasks in the scale-in target ACC have been completed (step S510). The offload destination connection unit 120 notifies the offload destination determination unit 130 that all tasks have been completed (step S511).

[0108] The offload destination determination unit 130 notifies the device control unit 150 of the scale-in target ACC information and instructs the device control unit 150 to power off (step S512). The device control unit 150 performs a power-off operation on the accelerator 11 (step S513) and confirms that the power-off has been confirmed (step S514). The device control unit 150 notifies the offload destination determination unit 130 that the power-off has been completed (step S515).

[0109] FIG. 20 is a flowchart showing the reconfiguration of the offload destination taking into consideration the optimal placement of the offload destination determination unit 130 in the control sequence of FIG. 19 . In step S51, the traffic collection unit 140 acquires the traffic volume of each task. In step S52, the offload destination determination unit 130 calculates the total traffic volume of the tasks. In step S53, the offload destination determination unit 130 determines the smallest ACC that can handle the total traffic volume as a candidate placement ACC. In step S54, the offload destination determination unit 130 determines the placement ACC with the largest available capacity. In step S55, the offload destination determination unit 130 selects the task with the largest traffic volume. In step S56, the offload destination determination unit 130 determines whether the task can be offloaded to the placement ACC. If the task cannot be offloaded to the placement ACC (S56: No), the processing of this flow ends.

[0110] If the task can be offloaded to the placement destination ACC (S56: Yes), the offload destination determination unit 130 changes the offload destination of the task in step S57. In step S58, the offload destination determination unit 130 determines whether there is any task for which offload destination reconfiguration is not being considered. If there is any task for which offload destination reconfiguration is not being considered (S58: Yes), the process returns to step S54 above. If there is no task for which offload destination reconfiguration is not being considered (S58: No), the offload destination reconfiguration is executed in step S59, and the processing of this flow ends.

[0111] <Determining the Minimum ACC Capable of Processing the Total Traffic Volume> This section explains how to determine the minimum ACC capable of processing the total traffic volume. The basic idea behind reconfiguring the offload destination is to relocate each task to a blank ACC, rather than replacing tasks already in the ACC. (1) The total ACC capacity is calculated by adding up all ACC capacities. (2) The capacity of the remaining ACC with the largest capacity is subtracted from the ACC capacity. (3) The ACC capacity is compared with the total traffic of the task. (3-1) If the ACC capacity is larger, the ACC previously subtracted is determined to be an unnecessary ACC, and the process proceeds to (2) above. (3-2) If the ACC capacity is smaller and there is an ACC with the next smallest capacity, the subtracted capacity is restored, the capacity of the ACC with the next smallest capacity is subtracted, and the process proceeds to (3-1) above. (3-3) If the ACC capacity is smaller and there is no ACC with the next smallest capacity, the process ends. (4) The remaining ACC is determined to be the minimum ACC.

[0112] Fig. 21 is a diagram that schematically illustrates <reconfiguration of offload destinations taking into consideration optimal allocation in feature <2>>. The thin solid line blocks in Fig. 21 indicate the allowable traffic of ACC1 and ACC2, and the shading in Fig. 21 indicates the ACC traffic volume (tasks a, b, c, and d). In Fig. 21, the allowable traffic (free capacity) of ACC1 is larger than the allowable traffic (free capacity) of ACC2, as indicated by the size of the block. Furthermore, tasks a and b have larger traffic volumes than tasks c and d, as indicated by the size of the dashed line blocks.

[0113] FIG. 21<1> Obtain the traffic volume of each task.

[0114] 21<2> ACC1 is selected as the destination ACC, task a is selected as the task with the largest traffic volume, and the offload destination is changed to ACC1. Since the allowable traffic (free capacity) of ACC1 is greater than the allowable traffic (free capacity) of ACC2, task a is selected for ACC1. In this case, tasks a and b with the largest ACC traffic volumes are selected first.

[0115] <3> in Figure 21: ACC2 is selected as the allocation destination ACC, and task b is selected as a task with a large traffic volume and the offload destination is changed to ACC2. Although task a has been selected for ACC1 in <2> above, ACC1 still has free capacity. On the other hand, ACC2 has no ACC traffic volume (task). Therefore, for the purpose of the above-mentioned <task load distribution>, task b is selected and the offload destination is changed to ACC2.

[0116] <4> in Figure 21: ACC1 is selected as the allocation destination ACC, task c is selected as the task with the largest traffic volume (task c has the second largest traffic volume after tasks a and b), and the offload destination is changed to ACC1. In <3> above, tasks a and b with the same traffic volume are offloaded to ACC1 and ACC2, respectively. Therefore, in terms of the relationship between allowable traffic (free capacity), ACC1 and ACC2 can be said to have returned to the state of <1> above. Therefore, task c is selected and the offload destination is changed to ACC1.

[0117] <5> in Figure 21: ACC1 is selected as the allocation destination ACC, task d is selected as a task with a large amount of traffic, and the offload destination is changed to ACC1. Tasks a and c are selected for ACC1, but task d is also selected. That is, although it is possible to select task d for ACC2, <task load distribution> is given priority, task d is selected, and the offload destination is changed to ACC2.

[0118] 21<6>: Tasks a, c, and d are selected for ACC1, and task b is selected for ACC2, and offload destination reconfiguration is performed. The above describes offload destination reconfiguration that takes optimal allocation into consideration (feature <2>).

[0119] [Scale trigger based on traffic and performance information (Feature <3>)] This section explains the scale trigger based on traffic and performance information (Feature <3>). The scale trigger based on traffic and performance information (Feature <3>) triggers a change in the offload destination in a timely manner according to traffic and performance values, thereby simultaneously achieving <Requirement 2: Power saving> and <Requirement 3: Performance>.

[0120] FIG. 22 is an explanatory diagram of the operation of the power-saving accelerator management system to explain a scale trigger based on traffic and performance information (feature <3>). Components identical to those in FIG. 1 are assigned the same reference numerals. In the following description, functional units for the relevant operations are indicated by bold frames. Feature <3> is realized by the scale trigger unit 103 shown within the dashed line in FIG. 22. Specifically, it is as follows: The performance acquisition unit 170 shown in FIG. 22 acquires the current performance (delay, throughput, etc.) of APL1 and APL2 and provides it to the configuration change trigger unit 180. The configuration change trigger unit 180 determines whether to issue an ACC scale-in / scale-out command based on the traffic information obtained from the traffic collection unit 140 and the performance information obtained from the performance acquisition unit 170, and triggers the offload destination determination unit 130 to issue an ACC scale-in / scale-out command. Alternatively, the configuration change trigger unit 180 triggers the offload destination reconfiguration unit 160 to perform an offload destination reconfiguration.

[0121] The power-saving accelerator management system 1000 in Fig. 22 acquires traffic prediction information from a traffic prediction unit 200 present in an external system (reference numeral 11 in Fig. 22). The configuration change trigger unit 180 cooperates with the traffic prediction unit 200 present in the external system to scale in and out according to future traffic predictions.

[0122] The scale trigger based on traffic and performance information (Feature 3) is characterized by ACC scale-in / scale-out that takes into account <excessive resources> and <insufficient resources>.

[0123] Fig. 23 is a diagram explaining ACC scale-in / scale-out taking into consideration feature <3> of <excessive resources> and <insufficient resources>. The upper diagram of Fig. 23 shows the change in traffic over time, and the lower diagram of Fig. 23 shows the change in performance such as throughput over time.

[0124] 23, when the traffic volume obtained from the traffic collection unit 140 falls below the lower threshold, the configuration change trigger unit 180 triggers ACC scale-in to the offload destination determination unit 130. This enables timely scale-in and contributes to <Requirement 2: power saving>.

[0125] <Insufficient Resources> As shown in the lower diagram of Fig. 23, the configuration change trigger unit 180 triggers the offload destination determination unit 130 to perform ACC scale-out when the performance obtained from the performance acquisition unit 170 falls below a lower threshold. Furthermore, the configuration change trigger unit 180 triggers the offload destination determination unit 130 to perform ACC scale-out when the traffic volume exceeds an upper threshold. This enables timely scale-out and contributes to <Requirement 3: Performance>. Here, the configuration change trigger unit 180 may use both the ACC scale-out trigger based on the traffic volume shown in the upper diagram of Fig. 23 and the ACC scale-out trigger based on performance shown in the lower diagram of Fig. 23.

[0126] <Scale Trigger Based on Traffic and Performance Information in Feature <3>> The scale trigger based on traffic and performance information in feature <3> will be described with reference to the control sequence in FIG. 24 and the flowchart in FIG. 25. FIG. 24 is a control sequence illustrating the scale trigger based on traffic and performance information in feature <3>. The performance acquisition unit 170 acquires performance for each APL and notifies the configuration change trigger unit 180 (step S601). The configuration change trigger unit 180 makes a scale-out decision based on the performance for each APL (step S602). If scale-out is not necessary, the configuration change trigger unit 180 requests traffic information from the traffic collection unit 140 (step S603). The traffic collection unit 140 collects traffic for each ACC and notifies the configuration change trigger unit 180 (step S604). The configuration change trigger unit 180 makes a scale-out decision based on the traffic for each ACC (step S605). If scale-out is not required, the configuration change trigger unit 180 requests traffic prediction information from the traffic prediction unit 200 of the external system (step S606).

[0127] The traffic prediction unit 200 notifies the configuration change trigger unit 180 of traffic prediction information for each APL (step S607). The configuration change trigger unit 180 makes a scale-out decision based on the traffic prediction information for each APL (step S608). If scale-out is not necessary, the configuration change trigger unit 180 makes a scale-in decision (step S609). If scale-in is necessary, the configuration change trigger unit 180 issues an ACC scale-in trigger to the offload destination determination unit 130 (step S610).

[0128] FIG. 25 is a flowchart of the scale trigger based on traffic and performance information A of the configuration change trigger unit 180 in the control sequence of FIG. 24. In step S61, the performance acquisition unit 170 acquires the performance for each APL. In step S62, the configuration change trigger unit 180 determines whether the number of APLs below the performance threshold is 0. If the number of APLs below the performance threshold is 0 (S62: Yes), the configuration change trigger unit 180 acquires the traffic for each ACC in step S63. In step S64, the configuration change trigger unit 180 determines whether the number of ACCs above the traffic upper limit threshold is 0. If the number of ACCs above the traffic upper limit threshold is 0 (S64: Yes), future traffic prediction information is acquired in step S65. In step S66, the configuration change trigger unit 180 determines whether the number of ACCs predicted to exceed the traffic upper limit threshold is 0.

[0129] If the APL below the performance threshold is not 0 in step S62 (S62: No), if the ACC above the traffic upper threshold is not 0 in step S64 (S64: No), or if the ACC predicted to exceed the traffic upper threshold is not 0 in step S66 (S66: No), the configuration change trigger unit 180 triggers ACC scale-out in step S68 and ends the processing of this flow.

[0130] If the ACC predicted to exceed the traffic upper threshold is 0 in step S66 (S66: Yes), the configuration change trigger unit 180 determines in step S67 whether the ACC below the traffic lower threshold is 0. If the ACC below the traffic lower threshold is not 0 (S68: No), the configuration change trigger unit 180 triggers ACC scale-in in step S69 and ends the processing of this flow.

[0131] If the ACC below the traffic lower limit threshold is 0 (S67: Yes), in step S70 the configuration change trigger unit 180 determines whether a certain amount of time has passed since the previous offload destination reconfiguration. If a certain amount of time has passed since the previous offload destination reconfiguration (S70: Yes), in step S71 the configuration change trigger unit 180 triggers offload destination reconfiguration and ends the processing of this flow. If a certain amount of time has not passed since the previous offload destination reconfiguration (S70: No), the processing of this flow ends.

[0132] The determination of whether or not there is an ACC that is predicted to exceed the traffic upper limit threshold will be explained. The determination of whether or not there is an ACC that is predicted to exceed the traffic upper limit threshold is performed, for example, as follows: Update the traffic volume of each task corresponding to the traffic of each APL1, 2 after a certain time t. Calculate the total traffic volume of the task after a certain time t for each ACC. Compare with the traffic upper limit threshold for each ACC. In this case, if t is very small, the upper limit threshold may be lowered for the determination.

[0133] [Hardware Configuration] The accelerator offload device 100 according to the above embodiment is realized by a computer 900 having a configuration such as that shown in Fig. 26. Fig. 26 is a hardware configuration diagram showing an example of the computer 900 that realizes the functions of the accelerator offload device 100. The computer 900 has a CPU 901, a RAM 902, a ROM 903, a HDD 904, an accelerator 905, an input / output interface (I / F) 906, a media interface (I / F) 907, and a communication interface (I / F) 908. The accelerator 905 corresponds to the accelerator 11 in Figs. 1, 7, 17, and 22.

[0134] The accelerator 905 is an accelerator (device) 11, 21 ( FIGS. 1 , 7 , 17 , and 22 ) that processes at least one of data from the communication I / F 908 and data from the RAM 902 at high speed. Note that the accelerator 905 may be of a type (look-aside type) that executes processing from the CPU 901 or RAM 902 and then returns the execution result to the CPU 901 or RAM 902. On the other hand, the accelerator 905 may be of a type (in-line type) that performs processing between the communication I / F 908 and the CPU 901 or RAM 902.

[0135] The accelerator 905 is connected to an external device 915 via a communication I / F 908. The input / output I / F 906 is connected to an input / output device 916. The media I / F 907 reads and writes data from and to a recording medium 917.

[0136] The CPU 901 operates based on a program stored in the ROM 903 or HDD 904, and controls each unit of the accelerator offload device 100 shown in Figures 1, 7, 17, and 22 by executing a program (also called an application or an app for short) loaded into the RAM 902. This program can also be distributed via a communication line or recorded on a recording medium 917 such as a CD-ROM. The ROM 903 stores a boot program executed by the CPU 901 when the computer 900 starts up, programs that depend on the hardware of the computer 900, and the like.

[0137] The CPU 901 controls an input / output device 916, which is made up of input units such as a mouse and a keyboard, and output units such as a display and a printer, via an input / output I / F 906. The CPU 901 acquires data from the input / output device 916 via the input / output I / F 906, and outputs generated data to the input / output device 916. Note that a GPU (Graphics Processing Unit) or the like may be used as a processor together with the CPU 901.

[0138] The HDD 904 stores programs executed by the CPU 901 and data used by the programs. The communication I / F 908 receives data from other devices via a communication network (e.g., NW (Network)) and outputs the data to the CPU 901, and also transmits data generated by the CPU 901 to other devices via the communication network.

[0139] The media I / F 907 reads a program or data stored in a recording medium 917 and outputs it to the CPU 901 via the RAM 902. The CPU 901 loads a program related to a target process from the recording medium 917 onto the RAM 902 via the media I / F 907, and executes the loaded program. The recording medium 917 is an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase Change Rewritable Disc), a magneto-optical recording medium such as an MO (Magneto Optical Disc), a magnetic recording medium, a conductive memory tape medium, a semiconductor memory, or the like.

[0140] For example, when the computer 900 functions as the accelerator offload device 100 configured as one device according to this embodiment, the CPU 901 of the computer 900 executes a program loaded onto the RAM 902 to realize the functions of the accelerator offload device 100. The HDD 904 also stores data stored in the RAM 902. The CPU 901 reads and executes a program related to a target process from the recording medium 917. Alternatively, the CPU 901 may read a program related to a target process from another device via a communications network.

[0141] [Effects] As described above, the accelerator offload device 100 (FIGS. 1, 7, 17, and 22) according to this embodiment is an accelerator offload device that has a plurality of accelerators and offloads specific processing of an application to the accelerators, and includes: a traffic collection unit 140 that collects current traffic information and / or predicted traffic information of the processing to be offloaded; a device simulation unit 110 that conceals connection destination switching from the perspective of the application so that it appears as if the application is communicating with the accelerator; an offload destination determination unit 130 that determines an offload destination based on the traffic information collected by the traffic collection unit 140 and provides offload destination information to the application; an offload destination connection unit 120 that connects the device simulation unit 110 to the offload destination accelerator in accordance with the offload destination information provided by the offload destination determination unit 130; and a device control unit 150 that performs power operations to turn on or off the power of devices including the accelerators in accordance with instructions from the offload destination determination unit 130.

[0142] In this way, the function that receives and sends tasks from an application on behalf of the accelerator changes the offload destination, switching the offload destination of tasks without changing the application. Also, when traffic decreases, the offload destination of all tasks on one accelerator is switched to another accelerator and powered off (accelerator scale-in), or powered on a surplus accelerator is switched on, switching the offload destination of some tasks on an accelerator with performance concerns to that accelerator (accelerator scale-out).

[0143] This reduces power consumption by dynamically running the minimum number of accelerators required without the application being aware of it. By using accelerator scale-out and load balancing mechanisms, it is possible to prevent performance degradation caused by a small number of accelerators in operation.

[0144] As a result, it is possible to reduce the power consumption of accelerators without causing performance degradation of applications due to reduced accelerator power consumption, while eliminating the need to add power-saving functions to applications. In other words, power consumption can be reduced by dynamically operating the minimum number of accelerators required without the application being aware of this. By using accelerator scale-out and load balancing mechanisms, it is possible to prevent performance degradation due to a small number of accelerators in operation.

[0145] When multiple accelerators are installed on a server that runs applications such as vRAN, applying the accelerator offload device 100 provides the following benefits: Power saving: By operating only the accelerators that are needed at any given time, unnecessary power consumption can be reduced. Performance: By adding operating accelerators when demand increases, service impacts due to insufficient performance can be avoided.

[0146] In the accelerator offload device 100 according to this embodiment, the offload destination determination unit 130 switches the offload destination of a task on one accelerator to another accelerator when traffic decreases, and the device control unit 150 performs accelerator scale-in, which turns off the power of the accelerator.

[0147] By doing this, when there are excess accelerator resources, it is possible to reduce the power consumption of the accelerator without degrading the performance of the application, thereby minimizing the power consumption of multiple accelerators installed in a server.

[0148] In the accelerator offload device 100 according to this embodiment, the device control unit 150 performs power operations to turn on the power of surplus accelerators, and the offload destination determination unit 130 performs accelerator scale-out, switching the offload destination of some of the tasks on an accelerator with performance concerns to that accelerator.

[0149] In this way, power management such as turning on surplus accelerators is performed, and some tasks on the accelerators with performance concerns are offloaded to those accelerators (accelerator scale-out). This prevents performance degradation caused by a small number of operating accelerators.

[0150] The accelerator offload device 100 according to this embodiment is characterized by including an offload destination reconfiguration unit 160 that recalculates the offload destination based on the traffic information collected by the traffic collection unit 140 and provides the offload destination reconfiguration information to the offload destination determination unit 130.

[0151] In this way, by triggering a change in the offload destination in a timely manner according to traffic information, it is possible to simultaneously achieve <Requirement 2: power saving> and <Requirement 3: performance>.

[0152] In the accelerator offload device 100 according to this embodiment, the offload destination reconfiguration unit 160 reconfigures the offload destination taking into consideration optimal placement. For example, in <power OFF ACC re-determination>, if it is possible to turn off the power of an accelerator that has higher power consumption than an accelerator that has been turned off by accelerator scale-in, the offload destination reconfiguration unit 160 provides offload destination reconfiguration information to the offload destination determination unit 130 so as to correct the offload destination of the task to that accelerator. The offload destination determination unit 130 corrects the offload destination of the task based on the offload destination reconfiguration information, and the device control unit 150 turns off the power of the accelerator that has higher power consumption. Furthermore, in <task load balancing>, as a reconfiguration of the offload destination that takes into consideration optimal allocation, the traffic collection unit 140 recalculates the offload destination based on the acquired traffic information, and if task load balancing is possible for each of the minimum required ACCs, provides offload destination reconfiguration information to the offload destination determination unit 130 so as to correct the offload destination of the task, and the offload destination determination unit 130 corrects the offload destination of the task.

[0153] In this way, the device control unit 150 can contribute to <Requirement 2: Power saving> by turning off the power of the ACC with high power consumption through <Power OFF ACC Re-determination>. Also, <Task load distribution> reduces the risk of performance degradation when traffic increases, thereby contributing to <Requirement 3: Performance>.

[0154] The accelerator offload device 100 according to this embodiment is characterized by comprising a performance acquisition unit 170 that acquires performance information including application delay and throughput, and a configuration change trigger unit 180 that determines whether to issue an accelerator scale-in or accelerator scale-out command based on the traffic information collected by the traffic collection unit 140 and the performance information acquired by the performance acquisition unit 170, and triggers the offload destination determination unit 130 to issue an accelerator scale-in or accelerator scale-out command, or triggers the offload destination reconfiguration unit 160 to issue an offload destination reconfiguration command.

[0155] In this manner, the configuration change trigger unit 180 triggers a change in the offload destination in a timely manner in accordance with the traffic and performance values, thereby simultaneously achieving <Requirement 2: power saving> and <Requirement 3: performance>. Specifically, when the traffic volume obtained from the traffic collection unit 140 is less than a threshold, the configuration change trigger unit 180 determines that there are <excessive resources> and triggers ACC scale-in to the offload destination determination unit 130, thereby achieving timely scale-in and contributing to <Requirement 2: power saving>. Furthermore, when the traffic volume obtained from the traffic collection unit 140 is greater than a threshold or when the performance obtained from the performance acquisition unit 170 is lower than a threshold, the configuration change trigger unit 180 determines that there are <insufficient resources> and triggers ACC scale-out to the offload destination determination unit 130, thereby achieving timely scale-out and contributing to <Requirement 3: performance>.

[0156] Note that, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. Furthermore, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. Furthermore, the components of each device shown in the drawings are functionally conceptual and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown in the drawings, and all or part of the devices can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

[0157] Furthermore, the above-described configurations, functions, processing units, processing means, etc. may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. The above-described configurations, functions, etc. may also be implemented by software that causes a processor to interpret and execute programs that implement the respective functions. Information such as programs, tables, and files that implement the respective functions may be stored in a memory, a recording device such as a hard disk or a solid-state drive (SSD), or a recording medium such as an integrated circuit (IC) card, a secure digital (SD) card, or an optical disk.

[0158] 1, 2, 30 APL (application / application program) 11, 21 ACC (accelerator) (device) 13, 23 Device queue (physical queue) 15, 25 Application thread 100 Accelerator offload device 110 Device simulation unit 120 Offload destination connection unit 130 Offload destination determination unit 140 Traffic collection unit 150 Device control unit 160 Offload destination reconfiguration unit 170 Performance acquisition unit 180 Configuration change trigger unit 200 Traffic prediction unit 201 Device mapping table 202 Application management table 203 Task table 204 Traffic table 205 Device management table 1000 Power-saving accelerator management system

Claims

1. An accelerator offload device having a plurality of accelerators and offloading a specific process of an application to the accelerators, a traffic collection unit that collects traffic information including information on current traffic performing offload target processing and / or traffic prediction information; a device pseudo unit that conceals the switching of the connection destination from the application so that the application appears to be communicating with one of the plurality of accelerators; an offload destination determination unit that determines an offload destination based on traffic information collected by the traffic collection unit and provides offload destination information for the application; an offload destination connection unit that connects the device simulation unit and an offload destination accelerator in accordance with offload destination information provided by the offload destination determination unit; a device control unit that performs power operation to turn on or off a power supply of one of the plurality of accelerators in accordance with an instruction from the offload destination determination unit.

1. An accelerator offload device comprising:

2. the offload destination determination unit switches an offload destination of a task on a first accelerator among the plurality of accelerators to a second accelerator among the plurality of accelerators when traffic decreases; The device control unit performs accelerator scale-in by turning off the power supply of the first accelerator.

2. The accelerator offload device according to claim 1.

3. The device control unit performs power operation to turn on a surplus accelerator among the plurality of accelerators, and the offload destination determination unit performs accelerator scale-out to switch an offload destination of a part of tasks on an accelerator having a performance concern among the plurality of accelerators to the surplus accelerator.

2. The accelerator offload device according to claim 1.

4. and an offload destination reconfiguration unit that recalculates an offload destination based on traffic information collected by the traffic collection unit and provides offload destination reconfiguration information to the offload destination determination unit. The accelerator offload device according to claim 1 .

5. the offload destination reconfiguration unit provides the offload destination determination unit with the offload destination reconfiguration information to modify the offload destination of the task to the first accelerator when a power supply of a second accelerator of the plurality of accelerators that consumes more power than a first accelerator of the plurality of accelerators that has been turned off due to accelerator scale-in can be turned off; the offload destination determination unit modifies an offload destination of a task based on the offload destination reconfiguration information; The device control unit turns off the power supply of the second accelerator.

5. The accelerator offload device according to claim 4.

6. the offload destination reconfiguration unit recalculates an offload destination based on the traffic information collected by the traffic collection unit, and when a minimum required load distribution of tasks for each accelerator is possible, provides the offload destination reconfiguration information to the offload destination determination unit so as to correct the offload destination of the tasks; The offload destination determination unit modifies the offload destination of the task based on the offload destination reconfiguration information.

5. The accelerator offload device according to claim 4.

7. a performance acquisition unit that acquires performance information including a delay and a throughput of the application; and a configuration change trigger unit that determines whether to issue an accelerator scale-in or accelerator scale-out based on the traffic information collected by the traffic collection unit and the performance information acquired by the performance acquisition unit, and triggers the offload destination determination unit to issue an accelerator scale-in or accelerator scale-out, or triggers the offload destination reconfiguration unit to issue an offload destination reconfiguration.

5. The accelerator offload device according to claim 4.

8. An accelerator offloading method for an accelerator offloading device having a plurality of accelerators and offloading a specific process of an application to the accelerators, comprising: The accelerator offload device is collecting traffic information including information about current traffic performing the offload target processing and / or predicted traffic forecast information; hiding the connection switch from the application's perspective such that the application appears to be communicating with one of the plurality of accelerators; determining an offload destination based on the collected traffic information and providing offload destination information for the application; connecting the device simulation unit and the offload destination accelerator according to the provided offload destination information; performing a power operation to power on or off one of the plurality of accelerators according to the instruction.

13. An accelerator offload method comprising:

9. A program for causing a computer to function as the accelerator offload device according to any one of claims 1 to 7.

Images