Device data access apparatus, device data access method, and program

Abstract

A device data access apparatus (100) for speeding up data access from a user application unit (31) to an external device unit (14) comprises: a data transfer completion detection unit (110) that detects completion of data transfer from the external device unit (14) to a memory (13) when a CPU (11) and the external device unit (14) exchange data via the memory (13); and a prefetch instruction unit (120) that prefetches transferred data in a main memory into a memory cache (12) present between the CPU (11) and the memory (13) at the time of completion of the data transfer detected by the data transfer completion detection unit (110).

Description

Device data access apparatus, device data access method and program

[0001] The present invention relates to a device data access apparatus, a device data access method, and a program.

[0002] Depending on the type of processor, different workloads are suited to different tasks (high processing power). While general-purpose central processing units (CPUs) are capable of quickly and efficiently processing highly parallel workloads that CPUs are not good at (low processing power), accelerators (hereinafter referred to as ACCs) such as field programmable gate arrays (FPGAs), graphics processing units (GPUs), and application-specific integrated circuits (ASICs) are available. By combining these heterogeneous processors and offloading workloads that CPUs are not good at (low processing power) to the ACCs, offloading technology is being increasingly utilized to improve overall processing time and efficiency.

[0003] In a virtual radio access network (vRAN) or the like, when the performance of a CPU alone is insufficient to meet requirements, some processing is offloaded to an accelerator capable of high-speed calculations, such as an FPGA or a GPU. Typical examples of specific workloads for which ACC offloading is performed include encoding / decoding processing (FEC: Forward Error Correction processing) in a vRAN, media processing of audio and video, and encryption / decryption processing.

[0004] In a computer system, a computer (hereinafter referred to as an accelerator-equipped server) may be configured to have hardware (CPU) that supports general-purpose processing and hardware (accelerator) specialized for specific calculations, and to offload some of the calculation processing from the general-purpose processor running the software to the accelerator.

[0005] FIG. 17 is a simplified block diagram showing the hardware of a server in a computer system. The server hardware 10 includes a CPU (Central Processing Unit) 11, a memory cache 12, a memory (main memory) 13, and an external device unit 14. The CPU 11 is one of the computational resources that performs computations together with an accelerator. The external device unit 14 includes a NIC (Network Interface Card) and an accelerator (ACC). The NIC is NIC hardware that realizes a network interface. The accelerator is a computational accelerator device such as an FPGA / GPU specialized for a specific computation.

[0006] In the computer system, data is sent and received between a CPU 11 and an external device unit 14. The CPU 11 and the external device unit 14 exchange data via a memory (main memory) 13. The server also has a user application unit (not shown), which accesses data stored in the external device unit 14 via the memory 13.

[0007] The external device unit 14 performs DMA (Direct Memory Access) transfer, which transfers data directly to and from the memory 13 without going through the CPU 11 .

[0008] [Existing Technology] Existing technologies include Intel Data Direct I / O Technology (Intel is a registered trademark) (Non-Patent Document 1) and effective use of Intel Data Direct I / O Technology (Non-Patent Document 2). Both Non-Patent Documents 1 and 2 are technologies related to the function of writing directly from a device to a memory cache, thereby reducing memory access time from an application.

[0009] In the example of applying the above-described existing technology shown in FIG. 17 , a direct write from the external device unit 14 to the memory cache 12 is performed, as indicated by the white arrow aa in FIG. 17 . Here, the direct write to the memory cache 12 is an L3 cache hit (see FIG. 18 for details on L3). Because the direct write from the external device unit 14 to the memory cache 12 is performed, requirement 1, which allows for a reduction in access time due to an L3 cache hit, is met. Furthermore, requirement 2, which provides application transparency, meaning that no modifications to the application are required, is met. However, in order to perform a direct write from the external device unit 14 to the memory cache 12, the external device unit and the CPU (memory cache) require dedicated functions and mechanisms, and therefore requirement 3, which requires hardware versatility, is not met.

[0010] This section explains the data reception process from an external device unit and the overhead caused by a cache miss in the existing technology. FIG. 18 is a diagram explaining the data reception process from an external device unit of a server in a computer system and the overhead caused by a cache miss. The same components as in FIG. 17 are assigned the same reference numerals. The server shown in FIG. 18 includes hardware 10, an OS 20, and a user application unit 31 in a user space 30. The user application unit 31 is an application that requires device IO. The OS 20 has an interrupt handler 21. The user application unit 31 and the OS 20 are connected via a Socket API or the like.

[0011] There are two application modes: access by interrupt and access by polling. As shown in Fig. 18, in the case of access by interrupt, the external device unit 14 starts the interrupt handler 21 of the OS 20, and the user application unit 31 accesses the data transferred to the memory 13 in response to an interrupt from the interrupt handler 21. The external device unit 14 also notifies the interrupt handler 21 of the completion of the transfer by interrupt.

[0012] On the other hand, in the case of a polling method in which the user application unit 31 communicates with the external device unit 14 by polling, the user application unit 31 accesses the memory 13 by polling. The polling method can be faster than the interrupt method depending on the polling interval.

[0013] The left diagram in Figure 18 shows the access time for one access (8 bytes) to the hardware 10. L1, L2, L3, and DRAM access in the left diagram in Figure 18 indicate the level of access time, with the closer to the CPU the shorter the access time. For example, L1 is the access time to the CPU's internal memory or memory cache 12, and the access time for one access (8 bytes) is the shortest at 1.0-4.0 ns. When accessing one packet of 9000 bytes, an estimated time of 1125 accesses is approximately 30-50 us, including 64-bit bulk transfers on the PCIe bus and hardware prefetching.

[0014] “Intel Data Direct I / O Technology”, [online], [searched March 1, 2024], Internet <URL: https: / / www.intel.com / content / www / us / en / io / data-direct-io-technology.html> “Effective Utilization of Intel Data Direct I / O Technology”, [online], [searched March 1, 2024], Internet <URL: https: / / www.isus.jp / products / vtune / vtune-cookbook-effective-utilization-of-data-direct-io-technology / >

[0015] However, the conventional technology has the following problem. In cases where the CPU 11 and the external device unit 14 exchange data via main memory, a cache miss always occurs when the CPU 11 accesses data transferred from the external device unit 14 to memory, resulting in overhead because the CPU 11 must access memory every time. For example, as shown in Figure 18, in the case of an access due to an interrupt, there is no cache hit on the first access (see the dashed-line box around the memory cache 12 in Figure 18), resulting in overhead. This results in an increase in direct access to the main memory due to cache misses, which poses a problem in that access speed from the user application unit 31 to the main memory cannot be increased ("Requirement 1: Access time" is not met).

[0016] Existing technologies (Non-Patent Documents 1 and 2) have the problem that they require dedicated functions and mechanisms in the external device unit and CPU (memory cache), which means that existing applications, external device units, and CPUs need to be modified (they do not satisfy "Requirement 3: Hardware versatility").

[0017] The present invention was made in light of this background, and its objective is to process access to data transferred by an external device unit in a short time without changing existing applications or hardware.

[0018] In order to solve the above-mentioned problems, the present invention provides a device data access device that speeds up data access from an application unit to an external device unit, characterized in that it comprises: a data transfer completion detection unit that detects the completion of data transfer from the external device unit to the main memory when a processor and the external device unit exchange data via main memory; and a prefetch instruction unit that, at the time of data transfer completion detected by the data transfer completion detection unit, prefetches the data from the main memory whose transfer has been completed into a memory cache located between the processor and the main memory.

[0019] According to the present invention, access to data transferred by an external device unit can be processed in a short time without changing existing applications or hardware.

[0020] FIG. 1 is a schematic configuration diagram of a device data high-speed access system according to a first embodiment of the present invention. FIG. 1 is a diagram illustrating a hardware configuration of a device data high-speed access system that logically increases the number of CPU cores by using CPU simultaneous multithreading by the device data high-speed access system according to an embodiment of the present invention, and assigns data processing threads to the logical cores to execute data processing in parallel. FIG. 2 is a diagram illustrating a correspondence table between device names and process names that access data transferred from the corresponding devices, which is provided in a device / data use core correspondence recording unit of a device data access device of the device data high-speed access system according to an embodiment of the present invention. FIG. 3 is a diagram illustrating a correspondence table between process names and operating logical core IDs, which is provided in a device / data use core correspondence recording unit of a device data access device of the device data high-speed access system according to an embodiment of the present invention. FIG. 4 is a diagram illustrating a basic operation of a device data access device of the device data high-speed access system according to an embodiment of the present invention. FIG. 5 is a diagram illustrating a proactive prefetch operation by an interrupt of a device data access device of the device data high-speed access system according to an embodiment of the present invention. FIG. 6 is a diagram illustrating a proactive prefetch operation by an interrupt trigger of a device data access device of the device data high-speed access system according to an embodiment of the present invention, in comparison with a polling method before application. FIG. 7 is a flowchart illustrating a proactive prefetch process by an interrupt of a device data access device of the device data high-speed access system according to an embodiment of the present invention. Fig. 1 is a diagram for explaining the proactive prefetch operation by polling of the device data access device of the high-speed device data access system according to an embodiment of the present invention. Fig. 2 is a diagram for explaining the proactive prefetch operation by polling of the device data access device of the high-speed device data access system according to an embodiment of the present invention, in comparison with the interrupt method before application. Fig. 3 is a flowchart showing the proactive prefetch processing by polling of the device data access device of the high-speed device data access system according to an embodiment of the present invention. Fig. 4 is a schematic configuration diagram of a high-speed device data access system according to a second embodiment of the present invention.Fig. 1 is a schematic configuration diagram of a device data high-speed access system according to a third embodiment of the present invention. Fig. 2 is a configuration example of a device data usage core corresponding recording unit of a device data access apparatus of the device data high-speed access system according to the third embodiment of the present invention. Fig. 3 is a schematic configuration diagram of a device data high-speed access system according to a fourth embodiment of the present invention. Fig. 4 is a hardware configuration diagram showing an example of a computer that realizes the functions of the device data high-speed access system according to the fourth embodiment of the present invention. Fig. 5 is a simplified block diagram showing hardware of a server of a computer system. Fig. 6 is a diagram explaining data reception processing from an external device unit of a server of a computer system and overhead due to a cache miss.

[0021] Hereinafter, a device data high-speed access system and the like in an embodiment for carrying out the present invention (hereinafter referred to as "the present embodiment") will be described with reference to the drawings. (First Embodiment) Fig. 1 is a schematic diagram of a device data high-speed access system according to a first embodiment of the present invention. Components that are the same as those in Fig. 18 are assigned the same reference numerals.

[0022] As shown in FIG. 1, the device data high-speed access system 1000 includes hardware 10, a device data access apparatus 100 arranged on an OS 20, and a user application section 31 on a user space 30.

[0023] [Hardware 10] The hardware 10 includes a CPU 11, a memory cache 12, a memory (main memory) 13, and an external device unit 14.

[0024] <CPU 11> The CPU 11 is a typical CPU installed in a server-type computer, and has a plurality of processing cores and a communication function with the external device unit 14 via interrupts.

[0025] <Memory cache 12> The memory cache 12 is a functional unit within the CPU 11 or independent from the CPU 11, and temporarily caches and holds data from the memory 13. The memory cache 12 is made up of elements that are capable of a faster response than the memory 13. For example, the memory 13 is made up of a charge-type dynamic random access memory (DRAM), and the memory cache 12 is made up of a static random access memory (SRAM).

[0026] <Memory 13> The memory 13 is a functional unit configured with elements such as DRAM, and receives and records data transferred from the external device unit 14. In addition, the memory 13 responds with the recorded data in response to a request from the CPU 11.

[0027] <External Device Unit 14> The external device unit 14 includes a NIC and an accelerator (ACC). The NIC and accelerator (ACC) have the function of transferring data to the memory 13. Furthermore, when the transfer is complete, they can notify the CPU by an interrupt. In this embodiment, the external device unit 14 includes a NIC and an accelerator (ACC). However, the external device unit 14 may include either a NIC or an accelerator (accelerator or NIC). Furthermore, the type and number of NICs and accelerators may be any. For example, accelerators include FPGAs, GPUs, and ASICs, as well as AI (artificial intelligence) accelerators. The external device unit 14 is connected to a server via a bus such as PCI Express. The external device unit 14 performs DMA transfer, which transfers data directly to and from the memory 13 without going through the CPU 11.

[0028] [User Application Unit 31] The user application unit 31 is an application that receives data from the external device unit 14. Methods that trigger data reception include a "polling method" in which the user application unit 31 accesses the memory 13 at regular intervals, and a method that is triggered by an interrupt from the external device unit 14. Data reception methods include a method in which the user application unit 31 directly accesses the memory 13, and a method in which the user application unit 31 accesses the memory 13 via the OS 20.

[0029] [Device Data Access Apparatus 100] The device data access apparatus 100 speeds up data access from the user application unit 31 to the external device unit 14 when the processor and the external device unit 14 exchange data via the memory (main memory) 13.

[0030] The device data access device 100 includes a data transfer completion detection unit 110, a prefetch instruction unit 120, an interrupt handling unit 130, a polling unit 140, a data access suppression unit 150, an interrupt suppression unit 160, a polling suppression unit 170, a data transfer completion detection execution core selection unit 180, a prefetch target cache selection unit 190, and a device / data usage core correspondence recording unit 200.

[0031] <Data Transfer Completion Detection Unit 110> The data transfer completion detection unit 110 detects the completion of data transfer from the external device unit 14 to the memory (main memory) 13. Specifically, the data transfer completion detection unit 110 detects the completion of data transfer by an interrupt from the external device unit 14 (FIGS. 6 to 8) or by polling processing from the CPU 11 (processor) (FIGS. 9 to 11).

[0032] The data transfer completion detection unit 110 detects the completion of data transfer from the external device unit 14 to the memory 13 via the polling unit 140 and the interrupt handling unit 130. After detecting the completion of the data transfer, the data transfer completion detection unit 110 notifies the prefetch instruction unit of the "size," "memory address," and "device ID" of the transferred data.

[0033] <Prefetch instruction unit 120> When the data transfer completion detection unit 110 detects that the data transfer has been completed, the prefetch instruction unit 120 prefetches the data from the memory (main memory) 13 whose transfer has been completed into the memory cache 12 located between the CPU 11 (processor) and the memory (main memory) 13.

[0034] The prefetch instruction unit 120 instructs the CPU 11 to load data from memory into the cache. Specifically, the prefetch instruction unit 120 instructs the CPU 11 to issue a memory prefetch command based on the memory address, physical core, and memory cache hierarchy (left diagram in FIG. 18). The physical core and memory cache hierarchy to be instructed may be inquired of by the prefetch target cache selection unit 190. Taking an Intel CPU as an example, the command to be issued is selected from multiple types of prefetch commands for different load target caches, and an instruction is instructed to be issued.

[0035] <Interrupt Handling Unit 130> The interrupt handling unit 130 detects the completion of data transfer by an interrupt from the external device unit 14 notifying the completion of data transfer.

[0036] <Polling Unit 140> The polling unit 140 detects the completion of data transfer by repeatedly accessing the memory 13 at a fixed cycle.

[0037] <Data Access Suppression Unit 150> The data access suppression unit 150 suppresses memory access until prefetching to the memory cache is completed. Specifically, the data access suppression unit 150 suppresses data access from the user application unit 31 via the polling suppression unit 170 and the interrupt suppression unit 160 until the prefetch instruction from the prefetch instruction unit 120 is completed. The data access suppression unit 150 suppresses data access in response to an instruction from the prefetch instruction unit 120, and releases the suppression in response to an instruction from the prefetch instruction unit 120. For example, the data access suppression unit 150 suppresses access from the user application unit 31 to data transferred from the external device unit 14 and loaded onto the memory 13.

[0038] <Interrupt suppression unit 160> The interrupt suppression unit 160 suppresses and releases memory access by masking or unmasking interrupt processing. Specifically, the interrupt suppression unit 160 suppresses notifications by interrupts from the external device unit 14 while a suppression command is instructed. One method of suppressing interrupts is to temporarily disable interrupt notifications from the device by masking the interrupt controller of the CPU.

[0039] <Polling suppression unit 170> The polling suppression unit 170 suppresses and releases memory access by stopping or starting the polling thread on the receiving side. Specifically, the polling suppression unit 170 suppresses memory access by polling while the suppression is instructed. There are two examples of polling suppression methods. Example 1: A method in which the allocation of calculation time to the polling thread of a user application is set to zero by scheduler settings. Example 2: A method in which a control flag for polling suppression is provided in the polling thread, and polling is suppressed according to the control flag.

[0040] <Data transfer completion detection execution core selection unit 180> The data transfer completion detection execution core selection unit 180 selects a core in the external device unit 14 that will detect the completion of data transfer, and sets the relevant core to detect the completion of data transfer. Based on the device name, the data transfer completion detection execution core selection unit 180 queries the device / data usage core correspondence recording unit 200 to obtain information about the core on which the process that uses the device is running, and sets the core to detect the completion of data transfer.

[0041] Example 1: When the proactive method is an interrupt When the detection method adopted by the data transfer completion detection unit 110 is an interrupt, an interrupt handler for detection is started in the relevant core by setting the interrupt mask of the CPU.

[0042] Example 2: When the proactive method is polling When the detection method adopted by the data transfer completion detection unit 110 is polling, the process that performs the polling is fixed to the core by using the Linux taskset command or the like.

[0043] <Prefetch target cache selection unit 190> The prefetch target cache selection unit 190 selects a memory cache to be prefetched by identifying information about the physical core in which the process using the data is running, based on the ID of the data source device. Here, the relationship between the device ID, the process using the data, and the physical core of that process is obtained from the user application unit 31.

[0044] <Device / Data-Used-Core Correspondence Recording Unit 200> The device / data-use-core correspondence recording unit 200 stores the device that performs the data transfer, the name of the process that accesses the data transferred by the device, and the physical core number on which the process runs.

[0045] 2 to 4, an example of a data structure stored in the device-data-usage-core correspondence recording unit 200 will be described. There is a system on a general-purpose server that logically increases the number of CPU cores by utilizing CPU simultaneous multithreading (SMT) to execute data processing in parallel. The CPU has a simultaneous multithreading (SMT) function, and can operate by separating one physical core into multiple logical cores. For example, SMT such as Intel Hyper Threading allows logical cores to share the CPU's microinstruction arithmetic units, thereby increasing the utilization rate of the arithmetic units.

[0046] 2 is a diagram showing the hardware configuration of a device data high-speed access system that uses simultaneous multithreading of a CPU to logically increase the number of CPU cores and assign data processing threads to the logical cores to execute data processing in parallel. The hardware 10 shown in FIG. 2 includes a CPU 11, a memory cache 12, and a memory (main memory) 13.

[0047] The CPU 11 has a physical core #A and a physical core #B (physical core 15). The physical core #A can be separated into a logical core #1 and a logical core #2 (logical core 16) and can operate separately, and the physical core #B can be separated into a logical core #3 and a logical core #4 (logical core 16). The logical core #1 and the logical core #2 are cores on which a process named router_process (FIG. 3) accessed by the NIC runs, and the logical core #3 and the logical core #4 are cores on which a process named acc_process_1 (FIG. 3) accessed by the accelerator runs.

[0048] The memory cache 12 has an L2 cache [individual core] 12A corresponding to the physical core #A, an L2 cache [individual core] 12A corresponding to the physical core #B, and a memory cache-L3 [common between cores] 12B.

[0049] 3 is a diagram showing a correspondence table 210 stored in the device-data-using-core correspondence recording unit 200, which lists device names and process names that access data transferred from the corresponding device. The device name "NIC" is associated with the process name "router_process" that accesses data transferred from the device. The device name "Accelerator" is associated with the process name "acc_process_1" that accesses data transferred from the device. The process name correspondence table 210 is predetermined for each application and system, and is therefore set in advance by the designer.

[0050] 4 is a diagram showing a correspondence table 220 between process names and operating logical core IDs stored in the device-data-usage-core correspondence recording unit 200. The process name "router_process" corresponds to the operating logical core IDs "1, 2" (logical core #1 and logical core #2), and the process name "acc_process" corresponds to the operating logical core IDs "3, 4" (logical core #3 and logical core #4). The logical core ID correspondence table 220 dynamically acquires the cores (affinity settings) on which each process may operate from the OS 20 (FIG. 1) and reflects them in the table.

[0051] The prefetch target cache selection unit 190 (Figure 1) refers to the process name correspondence table 210 in the device / data usage core correspondence recording unit 200, obtains the process name that accesses the data transferred from the device in question from the device name, and then refers to the logical core ID correspondence table 220 from this process name to identify the operating logical core ID, thereby selecting the memory cache to be prefetched (L2 cache [individual core] 12A, L3 [common between cores] 12B).

[0052] Next, the operation of the device data access apparatus 100 configured as described above will be described. (Principle explanation) First, the basic concept of the present invention will be described. The present invention prefetches data in the main memory for which the transfer has been completed into the memory cache at the time when data transfer from the external device unit to the main memory is completed. FIG. 5 is a diagram for explaining the basic operation of the device data access apparatus 100 of the device data high-speed access system 1000. FIG. 5 is shown in comparison with the explanatory diagram of the technical background and problems in FIG. 18. The user application unit 31 shown in FIG. 5 is an application that requires device I / O, here an application that receives data from the external device unit 14. The device data access apparatus 100 is arranged on the OS 20. The device data access apparatus 100 prefetches data in the memory (main memory) 13 for which the transfer has been completed into the memory cache 12 at the time when data transfer from the external device unit 14 to the memory (main memory) 13 is completed.

[0053] The device data access apparatus 100 detects the completion of data transfer from the external device unit 14 in advance and performs preloading into the memory cache 12 by prefetching, thereby reducing the memory access overhead of the CPU 11.

[0054] The device data access apparatus 100 detects the completion of transfer by an interrupt from the external device unit 14 or by polling processing from the CPU 11. Access by polling or access by an interrupt from the external device unit 14 is determined by the form of the application in the user application unit 31. Hereinafter, the operation by prefetching in advance of the device data access apparatus 100 will be described separately for <prefetching in advance by interrupt> (FIGS. 6 to 8) and <prefetching in advance by polling> (FIGS. 9 to 11).

[0055] [Preemptive Prefetching by Interrupt] (FIGS. 6 to 8) <Outline of Preemptive Prefetching by Interrupt> (FIG. 6) FIG. 6 is a diagram for explaining the preemptive prefetching operation by interrupt of the device data access device 100. The same components as those in FIG. 5 are given the same reference numerals. The device data access device 100 performs prefetching to the memory cache 12 when triggered by detection of an interrupt from the hardware 10 (see "Prefetch instruction" in FIG. 6). This enables the device data access device 100 to process access to data transferred by an external device unit in a short time without changing existing applications or hardware.

[0056] <Interrupt-Based Prefetch Operation> (FIG. 7) FIG. 7 is a diagram illustrating a proactive prefetch operation triggered by an interrupt, compared with the polling method before application. The upper diagram of FIG. 7 shows the behavior of an application in a comparative example before application of this embodiment, and the lower diagram of FIG. 7 shows the proactive prefetch operation triggered by an interrupt after application of this embodiment. As shown in the upper diagram of FIG. 7, the polling interval (white arrow b in the upper diagram of FIG. 7) between polling (symbol a1 in the upper diagram of FIG. 7) and polling (symbol a2 in the upper diagram of FIG. 7) is the first overhead (overhead #1). Furthermore, the access speed of the memory 13, which is composed of DRAM, is slower than that of the memory cache 12, such as SRAM, and this DRAM access is repeated multiple times, which is the second overhead (overhead #2) (shaded arrow c in the upper diagram of FIG. 7). The problems with the behavior (polling method) of the application in the comparative example before application are as follows. That is, all memory accesses after polling result in cache misses, so the DRAM access time is long (overhead #2).

[0057] The following describes a proactive prefetch operation triggered by an interrupt after application of this embodiment. As shown by CPU core #1 (proactive) in the lower diagram of FIG. 7, in this embodiment, when data transfer is completed, a transfer completion interrupt switch (symbol d in the lower diagram of FIG. 7) is used as a trigger to prefetch 12 data (here, three times) (symbol e in the lower diagram of FIG. 7) to the memory cache. Furthermore, the prefetching of 12 data to the memory cache is performed within a polling interval (symbols a1 and a2 in the lower diagram of FIG. 7). The polling interval is a factor in overhead (overhead #1), but this polling interval is used to prefetch 12 data to the memory cache within this polling interval (symbol e in the lower diagram of FIG. 7). Because prefetching (symbol e in the lower diagram of FIG. 7) is performed within the polling interval (symbols a1 and a2 in the lower diagram of FIG. 7), all memory accesses (symbol f in the lower diagram of FIG. 7) during subsequent polling (symbol a2 in the lower diagram of FIG. 7) result in cache hits (symbol g in the lower diagram of FIG. 7). This reduces the memory access time of the polling thread. As a result, overhead #2, which occurs due to long DRAM access times, can be reduced. According to the inventors' calculations, accessing one packet of 9,000 bytes required a maximum of 17 us, a reduction of approximately 15 us.

[0058] <Flowchart of Preemptive Prefetch Operation by Interrupt> (FIG. 8) FIG. 8 is a flowchart showing the preemptive prefetch process by interrupt. This flow is an example of a case where the user application unit 31 detects data arrival by polling using a method of preemptive detection by interrupt. Prior to the processing of this flow, the data transfer completion detection execution core selection unit 180 selects a core in the external device unit 14 that will perform data transfer completion detection, and sets the selected core to perform data transfer completion detection. Furthermore, if the detection method adopted by the data transfer completion detection unit 110 is an interrupt, the CPU's interrupt mask is set so that an interrupt handler for detection is activated in the selected core.

[0059] In step S11, when the external device unit 14 completes the transfer, it notifies the device data access apparatus 100 of an interrupt.

[0060] In step S12, the data transfer completion detection unit 110 detects the completion of data transfer from the external device unit 14 to the memory 13 via the polling unit 140 and the interrupt handling unit 130.

[0061] In step S13, the interrupt handling unit 130 detects the completion of data transfer by means of a notification interrupt of data transfer completion from the external device unit 14.

[0062] In step S14, the prefetch instruction unit 120 instructs the CPU 11 to load data on the memory 13 into the cache.

[0063] In step S15, until the prefetch instruction by the prefetch instruction unit 120 is completed, the data access suppression unit 150 suppresses data access from the user application unit 31 through the polling suppression unit 170 and the interrupt suppression unit 160.

[0064] In step S16, the polling suppression unit 170 suppresses memory access by polling during the period when suppression is instructed.

[0065] In step S17, the prefetch target cache selection unit 190 selects a target memory cache for prefetching by identifying information on the physical core in which the process using the data is running based on the ID of the data transfer source device.

[0066] In step S18, the device - data usage core correspondence recording unit 200 holds the device that performs data transfer, the process name that accesses the data transferred by the corresponding device, and further the physical core number in which the process operates.

[0067] In step S19, the prefetch instruction unit 120 issues a prefetch instruction to the prefetch target core.

[0068] In step S20, the user application unit 31 detects data reception at the polling opportunity, performs a memory access, and ends the processing of this flow. The above describes the look - ahead and prefetch by interrupt. Next, the look - ahead and prefetch by Polling will be described.

[0069] [Proactive Prefetching by Polling] (FIGS. 9 to 11) <Outline of Proactive Prefetching by Polling> (FIG. 9) FIG. 9 is a diagram for explaining the proactive prefetching operation by polling of the device data access device 100. The same components as those in FIG. 5 are given the same reference numerals. The device data access device 100 performs prefetching to the memory cache 12 when it is triggered by detecting the completion of data transfer from the hardware 10 by polling (see "Prefetch instruction" in FIG. 9).

[0070] <Proactive Prefetch Operation by Polling> (FIG. 9) FIG. 10 is a diagram illustrating proactive prefetch operation by polling, in comparison with the interrupt method before application. The upper diagram of FIG. 10 shows the behavior of an application using the interrupt method of a comparative example before application of this embodiment, and the lower diagram of FIG. 10 shows proactive prefetch operation using an interrupt trigger after application of this embodiment. As indicated by the white arrow in the upper diagram of FIG. 10, the transfer completion interrupt switch (symbol h in the upper diagram of FIG. 10) is the first overhead (overhead #3) (symbol i in the upper diagram of FIG. 10). Furthermore, the DRAM access (symbol j in the upper diagram of FIG. 10) to the memory 13, which is composed of DRAM, has a slow access speed and is repeated multiple times, which is the second overhead (overhead #2) (shaded arrow k in the upper diagram of FIG. 10).

[0071] The following describes a proactive prefetch operation triggered by an interrupt after the application of this embodiment. As shown in the lower diagram of FIG. 10, the physical cores are divided into CPU core #1 (proactive) and CPU core #2 (application). In this embodiment, CPU core #1 (proactive) prefetches 12 data (here, three times) (symbol l in the lower diagram of FIG. 10) to the memory cache, and then performs short-interval polling (here, three times) (symbol m in the lower diagram of FIG. 10) corresponding to the prefetch. That is, by performing short-interval polling (symbol m in the lower diagram of FIG. 10) before the transfer completion interrupt switch (symbol n in the lower diagram of FIG. 10), which will be described later, it becomes possible to prefetch 12 data (symbol l in the lower diagram of FIG. 10) to the memory cache.

[0072] Here, when an application communicates with the external device unit 14 via interrupts, polling at short intervals may result in less overhead, allow for proactive processing, and may enable faster processing through prefetching. In CPU core #2 (application), upon completion of data transfer, a transfer completion interrupt switch (symbol n in the lower diagram of FIG. 10) triggers a prefetch of 12 to the memory cache (symbol l in the lower diagram of FIG. 10). Because CPU core #1 (proactive) performs prefetching (symbol l in the lower diagram of FIG. 10), subsequent memory access in polling (symbol o in the lower diagram of FIG. 10) results in a cache hit (symbol p in the lower diagram of FIG. 10). This reduces the memory access time of the polling thread. As a result, overhead #2 (symbol q in the lower diagram of FIG. 10) caused by long DRAM access times can be reduced (dashed arrow r in the lower diagram of FIG. 10).

[0073] <Flowchart of proactive / prefetch operation by polling> (FIG. 11) FIG. 11 is a flowchart showing proactive / prefetch processing by polling. This flow is an example of a case where proactive detection is performed by polling, and the user application unit 31 detects data arrival by interrupt. Prior to the processing of this flow, the data transfer completion detection execution core selection unit 180 selects a core in the external device unit 14 that will perform data transfer completion detection, and sets the selected core to perform data transfer completion detection. Furthermore, if the detection method adopted by the data transfer completion detection unit 110 is an interrupt, the CPU's interrupt mask is set so that an interrupt handler for detection is activated in the selected core.

[0074] In step S21, the external device unit 14 transfers the data to the memory 13, and when the transfer is complete, notifies the CPU 11 by an interrupt.

[0075] In step S22, the data transfer completion detection unit 110 detects the completion of the data transfer from the external device unit 14 to the memory 13 via the polling unit 140 and the interrupt handling unit 130.

[0076] In step S23, the polling unit 140 detects the completion of data transfer by repeatedly accessing the memory 13 at a fixed interval.

[0077] In step S24, the prefetch instruction unit 120 instructs the CPU to load the data in the memory 13 into the cache.

[0078] In step S25, the data access suppression unit 150 suppresses data access from the user application unit 31 through the polling suppression unit 170 and the interruption suppression unit 160 until the prefetch instruction from the prefetch instruction unit 120 is completed.

[0079] In step S26, the interrupt suppression unit 160 suppresses notifications by interrupts from external devices during the period for which the suppression is instructed.

[0080] In step S27, the prefetch target cache selection unit 190 selects a memory cache to be prefetched by identifying information about the physical core in which the process using the data is running, based on the ID of the data source device.

[0081] In step S28, the device / data-using-core correspondence recording unit 200 stores the device that performs the data transfer, the name of the process that accesses the data transferred by the device, and the physical core number on which the process operates.

[0082] In step S29, the prefetch instruction unit 120 issues a prefetch command to the prefetch target core.

[0083] In step S30, the user application unit 31 receives the interrupt, accesses the memory, and ends the processing of this flow. The preceding paragraphs have described proactive prefetching using polling.

[0084] Second Embodiment FIG. 12 is a schematic diagram of a device data high-speed access system according to a second embodiment of the present invention. Components identical to those in FIG. 5 are designated by the same reference numerals. As shown in FIG. 5, the device data high-speed access system 1000A includes hardware 10, an OS 20, and a user application unit 31 in a user space 30. The OS 20 includes a device data access device 100, an access suppression unit 230, and a scheduler 240. The access suppression unit 230 suppresses access to a memory area from an application until the device data access device 100 completes prefetching. The access suppression unit 230 receives from the device data access device 100 a "notification to the scheduler 220 / suppression of polling from an application by CPU setting" (suppression method 1) and a "temporary suspension of interrupts by interrupt mask setting" (suppression method 2).

[0085] The device data access apparatus 100 includes the access inhibiting unit 230, thereby making it possible to prevent device access from occurring from an application before prefetching is completed.

[0086] By making the user application unit 31 wait until the prefetch is completed, it is possible to reduce the resources on the application side.

[0087] Third Embodiment In a third embodiment of the present invention, a prefetch target cache and a hierarchy are selected by identifying a core that uses data transferred from a device. FIG. 13 is a schematic configuration diagram of a device data high-speed access system 1000B according to a third embodiment of the present invention. The same components as those in FIGS. 2, 5, and 18 are assigned the same reference numerals. FIG. 13 shows the relevant units (prefetch instruction unit 120, data transfer completion detection execution core selection unit 180, prefetch target cache selection unit 190, and device / data use core correspondence record unit 200) extracted from the device data access apparatus 100 in FIG. 5.

[0088] Fig. 14 shows an example of the configuration of the device / data utilization core correspondence recording unit 200 shown in Fig. 13. The device / data utilization core correspondence recording unit 200 shown in Fig. 13 stores the device that performs data transfer, the name of the process that accesses the data transferred by the device, and the physical core number on which the process runs, as well as the data utilization core and device shown in Fig. 14. For example, the data utilization core is physical core #B and the device is a NIC, and physical core #B and the device is an accelerator.

[0089] In this embodiment, in order to prevent an application from accessing a device before prefetching is completed, the prefetch instruction unit 120 suppresses access to a memory area from the application until prefetching is completed. Specifically, the prefetch instruction unit 120 "prevents polling from an application by setting a scheduler" or "temporarily suspends interrupts by setting an interrupt mask."

[0090] When prefetching main memory data, it is desirable to prefetch to a cache as close as possible to the CPU core that will access the data. As shown in the enlarged left diagram of Figure 13, it is desirable to prefetch to a cache closer to the CPU core, for example, L2 (approximately 256 kb) rather than L3 (approximately 2-4 mb), and to prefetch to L1 (approximately 64 kb) rather than L2 (approximately 256 kb).

[0091] Therefore, in this embodiment, the device data access device 100 selects a cache to be prefetched based on information in the "device-data-using-core correspondence recording unit 200" that records the correspondence between devices and data-using cores, and realizes prefetching to a location close to the core that uses the data. Furthermore, in this embodiment, the proactive prefetch operation described in the first and second embodiments is activated in the corresponding prefetch target core. By making it possible to execute the prefetch command in a core close to the core that uses the data, overhead due to transitions between cores is further reduced.

[0092] (Fourth Embodiment) A fourth embodiment of the present invention describes a specific example of an application. FIG. 15 is a schematic configuration diagram of a device data high-speed access system according to a fourth embodiment of the present invention. Components that are the same as those in FIG. 5 are assigned the same reference numerals. In the device data high-speed access system 1000C shown in FIG. 15, a web server application 31A and a device data access apparatus 100 are placed in a user space 30. The web server application 31A receives and responds to requests from clients via a network. The web server application is, for example, Apache or Nginx. Requests from the web server application 31A are received in the form of packets via the NIC. Packet reception processing from the NIC is performed by "memory access triggered by an interrupt."

[0093] In this embodiment, the proactive prefetch operation (proactive memory prefetching by polling) described in the first to third embodiments is used to anticipate memory access from the Web server application, resulting in a cache hit when the Web server application accesses memory, thereby reducing access time.

[0094] [Hardware Configuration] The device data access apparatus 100 ( FIG. 1 ) according to the above embodiment is realized by, for example, a computer 900 configured as shown in FIG. 16 . FIG. 16 is a hardware configuration diagram showing an example of the computer 900 that realizes the functions of the device data access apparatus 100. The device data access apparatus 100 has a CPU 901, a RAM 902, a ROM 903, a HDD 904, an accelerator 905, an input / output interface (I / F) 906, a media interface (I / F) 907, and a communication interface (I / F: Interface) 908. The accelerator 905 corresponds to the accelerator of the external device unit 14 in FIG. 1 .

[0095] The accelerator 905 is an accelerator (external device unit 14) ( FIG. 1 ) that processes at least one of data from the communication I / F 908 and data from the RAM 902 at high speed. Note that the accelerator 905 may be of a type that returns the execution results to the CPU 901 or RAM 902 after executing processing from the CPU 901 or RAM 902 (look-aside type). On the other hand, the accelerator 905 may be of a type that performs processing between the communication I / F 908 and the CPU 901 or RAM 902 (in-line type).

[0096] The accelerator 905 is connected to an external device 915 via a communication I / F 908. The input / output I / F 906 is connected to an input / output device 916. The media I / F 907 reads and writes data from and to a recording medium 917.

[0097] The CPU 901 operates based on a program stored in the ROM 903 or HDD 904, and controls each part of the device data access apparatus 100 shown in Fig. 1 by executing a program (also called an application or an app for short) loaded into the RAM 902. This program can also be distributed via a communication line or recorded on a recording medium 917 such as a CD-ROM. The ROM 903 stores a boot program executed by the CPU 901 when the computer 900 starts up, programs that depend on the hardware of the computer 900, and the like.

[0098] The CPU 901 controls an input / output device 916, which is made up of input units such as a mouse and a keyboard, and output units such as a display and a printer, via an input / output I / F 906. The CPU 901 acquires data from the input / output device 916 via the input / output I / F 906, and outputs generated data to the input / output device 916. Note that a GPU (Graphics Processing Unit) or the like may be used as a processor together with the CPU 901.

[0099] The HDD 904 stores programs executed by the CPU 901 and data used by the programs. The communication I / F 908 receives data from an external device 915 via a communication network (e.g., NW (Network)) and outputs the data to the CPU 901, and also transmits data generated by the CPU 901 to the external device 915 via the communication network.

[0100] The media I / F 907 reads a program or data stored in a recording medium 917 and outputs it to the CPU 901 via the RAM 902. The CPU 901 loads a program related to a target process from the recording medium 917 onto the RAM 902 via the media I / F 907, and executes the loaded program. The recording medium 917 is an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase Change Rewritable Disc), a magneto-optical recording medium such as an MO (Magneto Optical Disc), a magnetic recording medium, a conductive memory tape medium, a semiconductor memory, or the like.

[0101] For example, when the computer 900 functions as the device data access apparatus 100 ( FIG. 1 ) configured as one apparatus according to this embodiment, the CPU 901 of the computer 900 executes a program loaded onto the RAM 902 to realize the functions of the device data access apparatus 100. The HDD 904 also stores data in the RAM 902. The CPU 901 reads and executes a program related to a target process from a recording medium 917. Alternatively, the CPU 901 may read a program related to a target process from an external device 915 via a communication network.

[0102] [Effects] As described above, the device data access apparatus 100 (FIG. 1) that speeds up data access from the application unit (user application unit 31) to the external device unit 14 includes a data transfer completion detection unit 110 that detects the completion of data transfer from the external device unit 14 to the main memory (memory 13) when the processor (CPU 11) and the external device unit 14 exchange data via the main memory (memory 13), and a prefetch instruction unit 120 that, at the time of data transfer completion detected by the data transfer completion detection unit 110, prefetches the data in the main memory whose transfer has been completed into the memory cache 12 located between the processor (CPU 11) and the main memory (memory 13).

[0103] In this way, the device data access apparatus 100 can proactively detect the completion of data transfer from the external device unit 14 and pre-load the data into the memory cache 12 by pre-fetching, thereby reducing the memory access overhead of the CPU 11. Therefore, access to data transferred by the external device unit 14 can be processed in a short time without changing existing applications or hardware.

[0104] Therefore, the device data access device 100 can satisfy Requirement 1: (Access time) Speeding up access from an application to the main memory (reducing direct access to the main memory due to cache misses). Furthermore, the device data access device 100 can satisfy Requirement 2: (Application transparency) No changes to existing applications are required. Furthermore, the device data access device 100 can satisfy Requirement 3: (Hardware versatility) No special functions such as direct loading to a cache are required for the hardware.

[0105] In the device data access apparatus 100 (FIG. 1), the data transfer completion detection unit 110 detects the completion of data transfer by an interrupt from the external device unit 14 or by polling processing from the processor.

[0106] By doing this, the device data access apparatus 100 performs prefetching proactively, independent of the user application, by having the data transfer completion detection unit 110 start prefetching when it detects the completion of data transfer. This allows the device data access apparatus 100 to satisfy "Requirement 1: Access time" and "Requirement 2: Application transparency." Furthermore, since the detection of data transfer completion can be achieved by a general-purpose hardware function such as polling or an interrupt, it can satisfy "Requirement 3: Hardware versatility."

[0107] The device data access apparatus 100 (FIG. 1) includes a data access suppression unit 150 that suppresses memory access until prefetching to the memory cache 12 is completed.

[0108] By doing this, the data access suppression unit 150 can suppress data access from the user application unit 31 through the polling suppression unit 170 and the interrupt suppression unit 160 until the prefetch instruction from the prefetch instruction unit 120 is completed.

[0109] The device data access apparatus 100 (FIG. 1) includes a polling suppression unit 170 that suppresses and releases memory access by stopping or starting a polling thread on the receiving side.

[0110] By doing so, the polling suppression unit 170 can suppress memory access by polling during the period when the suppression is instructed.

[0111] The device data access apparatus 100 (FIG. 1) includes an interrupt suppression unit 160 that suppresses and releases memory access by masking or unmasking interrupt processing.

[0112] By doing so, it is possible to suppress notifications by interrupt from the external device unit 14 during the period when the suppression is instructed.

[0113] In the device data access apparatus 100 (FIG. 1), the physical core of the processor is separated into a prefetch-only core and a CPU core that performs data access.

[0114] By doing this, for the proactive prefetch operation triggered by an interrupt, the polling interval shown for CPU core #1 (proactive) in the lower diagram of Figure 7 is used, and 12 prefetches (symbol e in the lower diagram of Figure 7) are performed to the memory cache within this polling interval, and prefetches (symbol e in the lower diagram of Figure 7) are performed within the polling interval (symbols a1 and a2 in the lower diagram of Figure 7).This has the effect of ensuring that all memory accesses (symbol f in the lower diagram of Figure 7) in the subsequent polling (symbol a2 in the lower diagram of Figure 7) result in cache hits (symbol g in the lower diagram of Figure 7).

[0115] Furthermore, when data transfer is completed in CPU core #2 (application), a transfer completion interrupt switch (symbol n in the lower diagram of FIG. 10) is used as a trigger to perform a prefetch (symbol l in the lower diagram of FIG. 10) of 12 to the memory cache. Because CPU core #1 (proactive) performs a prefetch (symbol l in the lower diagram of FIG. 10), the memory access during subsequent polling (symbol o in the lower diagram of FIG. 10) can have the effect of resulting in a cache hit (symbol p in the lower diagram of FIG. 10).

[0116] Furthermore, among the processes described in the above embodiments and variations, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using known methods. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. Furthermore, the components of each device shown in the drawings are functionally conceptual and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown in the drawings, and all or part of the devices can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

[0117] Furthermore, the above-described configurations, functions, processing units, processing means, etc. may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. The above-described configurations, functions, etc. may also be implemented by software that causes a processor to interpret and execute programs that implement the respective functions. Information such as programs, tables, and files that implement the respective functions may be stored in a memory, a recording device such as a hard disk or a solid-state drive (SSD), or a recording medium such as an integrated circuit (IC) card, a secure digital (SD) card, or an optical disk.

[0118] 10 Hardware 11 CPU 20 OS 30 User space 31 User application unit (application unit) 31A Web server application 12 Memory cache 13 Memory (main memory) 14 External device unit 100 Device data access device 110 Data transfer completion detection unit 120 Prefetch instruction unit 130 Interrupt handling unit 140 Polling unit 150 Data access suppression unit 160 Interrupt suppression unit 170 Polling suppression unit 180 Data transfer completion detection execution core selection unit 190 Prefetch target cache selection unit 200 Device / data use core correspondence recording unit 230 Access suppression unit 240 Scheduler 1000, 1000A, 1000B, 1000C Device data high-speed access system

Claims

1. A device data access device that speeds up data access from an application unit to an external device unit, comprising: a data transfer completion detection unit that detects the completion of data transfer from the external device unit to the main memory when a processor and the external device unit exchange data via main memory; and a prefetch instruction unit that, at the time of data transfer completion detected by the data transfer completion detection unit, prefetches the data from the main memory whose transfer has been completed into a memory cache located between the processor and the main memory.

2. The device data access device according to claim 1, wherein said data transfer completion detection unit detects the completion of data transfer by an interrupt from said external device unit or by polling processing from said processor.

3. The device data access device according to claim 1, further comprising a data access suppression unit that suppresses memory access until prefetching to the memory cache is completed.

4. The device data access device according to claim 3, further comprising a polling suppression unit that suppresses and releases the memory access by stopping or starting a polling thread on the receiving side.

5. The device data access device according to claim 3, further comprising an interrupt suppression unit that suppresses and releases the memory access by masking or unmasking an interrupt process.

6. The device data access device according to claim 3, wherein the physical core of the processor is separated into a prefetch-only core and a CPU core that performs data access.

7. A device data access method for a device data access device that speeds up data access from an application unit to an external device unit, wherein the device data access device executes, when a processor and the external device unit exchange data via main memory, a data transfer completion detection step of detecting completion of data transfer from the external device unit to the main memory, and a step of prefetching the data in the main memory whose transfer has been completed, into a memory cache located between the processor and the main memory, at the time of data transfer completion detected by the data transfer completion detection step.

8. A program for causing a computer to function as the device data access apparatus according to any one of claims 1 to 6.

Images