Computational resource management program, computational resource management method, and information processing device.

Abstract

This will allow users to transparently specify the computing resources on which to run deep learning applications. [Solution] When a call is made to the deep learning framework 30 during the execution of a deep learning application 20, a first identifier that identifies the computing resources described in the application 20 is obtained, and based on the first identifier, a second identifier corresponding to the first identifier is obtained by referring to the correspondence relationship 105 between the first identifier and a second identifier that identifies the physical computing resources managed by the framework 30, the first identifier included in the call is replaced with the obtained second identifier to generate a new call, and the new call is sent to the framework 30.

Description

Technical Field

[0001] The present invention relates to a computing resource management program, a computing resource management method, and an information processing apparatus.

Background Art

[0002] For example, it is known that by using a GPU (Graphics Processing Unit) instead of a CPU (Central Processing Unit) to execute a deep learning (DL: Deep Learning) application program (hereinafter referred to as a DL application), the processing performance can be improved. The CPU and GPU are examples of computing resources.

[0003] In order for a programmer of a DL application to benefit from the acceleration of operations by a dedicated device such as a GPU, explicit device specification is performed in the DL application. When multiple DL applications are executed simultaneously, if the explicit device specifications conflict between the DL applications, even if there is sufficient system-wide resources, the resources cannot be allocated and a conflict occurs. In order to utilize the system-wide resources, the programmer needs to consider the hardware environment and determine the allocation of device IDs in cooperation between the DL applications. For example, a method is known in which the device ID in a DL application is rewritten to a physical device ID by setting an environment variable such as CUDA_VISIBLE_DEVICES.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Patent Document 3

[0005] However, pre-determining the assignment of device IDs through coordinated efforts between DL applications results in DL applications becoming execution environment-dependent code, leading to low portability and making it difficult to transparently apply DL applications to different execution environments. "Transparent application" means applying DL applications without fundamentally rewriting them. Furthermore, it requires manual mapping between device IDs in DL applications and physical device IDs, which is cumbersome.

[0006] In one aspect, the present invention aims to enable the transparent specification of computing resources on which deep learning applications are to run. [Means for solving the problem]

[0007] Therefore, in a computer with multiple computing resources, when a call to a deep learning framework is made during the execution of a deep learning application, this scheduling program causes the computer to perform the following processes: obtain a first identification that identifies the computing resources described in the application; obtain a second identification that corresponds to the first identification by referring to the correspondence between the first identification and a second identification that identifies the physical computing resources managed by the framework based on the first identification; replace the first identification included in the call with the obtained second identification to generate a new call; and send the new call to the framework. [Effects of the Invention]

[0008] According to one embodiment, computing resources can be specified to run a deep learning application transparently. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram showing an example of a computer hardware (HW) configuration that implements the functions of a computing resource management system according to one embodiment. [Figure 2] This diagram schematically illustrates the relationship between computing resources and download applications. [Figure 3] This figure shows an example of dynamic changes to memory allocation in a computing resource management system. [Figure 4] This diagram schematically shows the configuration of a computing resource management system according to one embodiment. [Figure 5] This diagram illustrates the correspondence between virtual device IDs and physical device IDs. [Figure 6] This figure shows an example of tensor data for each virtual device ID. [Figure 7] This figure shows an example of the process of moving tensor data between devices. [Figure 8] This flowchart shows an example of processing performed by the device usage detection unit. [Figure 9] This flowchart shows an example of processing performed by the Device Management Unit. [Figure 10] This flowchart shows an example of processing performed by the scheduling unit. [Figure 11] This flowchart shows an example of processing performed by the tensor tracking and movement unit. [Modes for carrying out the invention]

[0010] The following describes embodiments of the computing resource management program, computing resource management method, and information processing device with reference to the drawings. However, the embodiments shown below are merely illustrative, and there is no intention to exclude various modifications or applications of technologies not explicitly shown in the embodiments. In other words, these embodiments can be implemented with various modifications without departing from their spirit. Furthermore, each figure is not intended to represent only the components shown in the figure, but may include other functions, etc.

[0011] (A) Configuration FIG. 1 is a block diagram showing a hardware (HW) configuration example of a computer 10 that realizes the functions of a computing resource management system according to an embodiment.

[0012] (A-1) Hardware Configuration Example When a plurality of computers are used as HW resources for realizing the functions of the computing resource management system, each computer may have the HW configuration illustrated in FIG. 1.

[0013] As shown in FIG. 1, the computer 10 is an information processing device, and as a HW configuration, illustratively, one or more (one in the example shown in FIG. 1) CPUs 10a, a plurality (two in the example shown in FIG. 2) of GPUs 10b-1, 10b-2, a memory 10c, a storage unit 10d, an IF (Interface) unit 10e, an IO (Input / Output) unit 10f, and a reading unit 10g may be provided. Hereinafter, when the GPUs 10b-1, 10b-2 are not particularly distinguished, they are denoted as GPU 10b.

[0014] The CPU 10a is an example of an arithmetic processing device that performs various controls and operations, and is a control unit (reference numeral 100 in FIG. 4) that executes various processes. The CPU 10a may be communicably connected to each block in the computer 10 via a bus 10j. The bus 10j may be a PCIe (Peripheral Component Interconnect-Express) bus. Note that the CPU 10a may be a multiprocessor including a plurality of processors, a multi-core processor having a plurality of processor cores, or a configuration having a plurality of multi-core processors.

[0015] The GPU 10b may be an accelerator such as a GPGPU (General Purpose computing on Graphics Processing Unit). Furthermore, the GPU 10b may be used to control screen display for output devices such as monitors in the I / O unit 10f. The GPU 10b may be configured as an accelerator that performs machine learning processing and inference processing using machine learning models. Regarding machine learning processing and inference processing, the GPU 10b can be said to have higher processing performance than the CPU 10a.

[0016] These CPU10a, GPU10b-1, and 10b-2 are examples of computing resources allocated to DL applications, which are applications in the field of deep learning. GPU10b-1 and 10b-2 are examples of first-class computing resources, and CPU10a is an example of a second-class computing resource with lower processing performance than the first-class computing resources.

[0017] Figure 2 schematically illustrates the relationship between computing resources and DL applications 20-1 and 20-2. Hereafter, unless otherwise specified, DL applications 20-1 and 20-2 will be referred to as DL application 20.

[0018] The execution of DL application 20 utilizes not only CPU 10a but also GPUs 10b-1 and 10b-2. GPU 10b is used to perform specific calculations for DL ​​at high speed. In this specification, the case in which GPU 10b is used as an accelerator (acceleration device) will be explained as an example.

[0019] The main memory capacity of CPU10a is greater than the main memory capacity of GPU10b. Main memory is a storage device that devices such as CPU10a and GPU10b can directly read and write information to.

[0020] Returning to Figure 1, memory 10c is an example of hardware that stores various data and program information. Examples of memory 10c include volatile memory such as DRAM (Dynamic Random Access Memory) and non-volatile memory such as PM (Persistent Memory), or both. Memory 10c may include the main memory of the CPU 10a and the main memory of the GPU 10b.

[0021] The storage unit 10d is an example of hardware that stores various data and program information. Examples of storage units 10d include magnetic disk devices such as HDDs (Hard Disk Drives), semiconductor drive devices such as SSDs (Solid State Drives), and various storage devices such as non-volatile memory. Examples of non-volatile memory include flash memory, SCM (Storage Class Memory), and ROM (Read Only Memory).

[0022] The memory unit 10d may store a program 10h (computation resource management program) that implements all or part of the various functions of the computer 10.

[0023] For example, the CPU 10a of the computing resource management system can implement the computing resource management function described later by loading the program 10h stored in the memory 10c into memory 10c and executing it.

[0024] The IF unit 10e is an example of a communication interface that controls the connection and communication between this computer 10 and other computers. For example, the IF unit 10e may include an adapter compliant with LAN (Local Area Network) such as Ethernet®, or optical communication such as FC (Fibre Channel). The adapter may support wireless, wired, or both communication methods. The program 10h may be downloaded from the network to the computer 10 via the communication interface and stored in the storage unit 10d.

[0025] The I / O unit 10f may include either an input device or an output device, or both. Examples of input devices include keyboards, mice, and touch panels. Examples of output devices include monitors, projectors, and printers. The I / O unit 10f may also include a touch panel or the like that integrates the input and output devices. The output device may be connected to the GPU 10b. The I / O unit 10f may also be an input or output device of another information processing device remotely connected to this computer 10 via SSH (Secure Shell) or the like.

[0026] The reading unit 10g is an example of a reader that reads data and program information recorded on the recording medium 10i. The reading unit 10g may include a connection terminal or device to which the recording medium 10i can be connected or inserted. Examples of the reading unit 10g include an adapter compliant with USB (Universal Serial Bus), a drive device for accessing a recording disk, and a card reader for accessing flash memory such as an SD card. The recording medium 10i may store a program 10h, and the reading unit 10g may read the program 10h from the recording medium 10i and store it in the storage unit 10d.

[0027] Examples of recording media 10i include non-temporary computer-readable recording media such as magnetic / optical discs and flash memory. Examples of magnetic / optical discs include flexible discs, CDs (Compact Discs), DVDs (Digital Versatile Discs), Blu-ray discs, and HVDs (Holographic Versatile Discs). Examples of flash memory include semiconductor memory such as USB memory and SD cards.

[0028] The hardware configuration of computer 10 described above is illustrative. Therefore, the addition or deletion of hardware within computer 10 (for example, adding or deleting arbitrary blocks), division, integration in any combination, or addition or deletion of buses may be performed as appropriate.

[0029] (A-2) Example of Functional Configuration Figure 3 shows an example of dynamic changes to memory allocation in a computing resource management system. The resource management technology of this embodiment dynamically changes the correspondence between the computing resources described in DL application 20 and the physical computing resources such as CPU 10a, GPU 10b-1, 10b-2 that are actually allocated, while DL application 20 is running.

[0030] In Figure 3, the memory allocation for each of the multiple DL applications #A, #B, and #C, which are referred to as DL application 20, is dynamically changed. Figure 3 shows the memory allocation in a time series in the order of (1), (2), and (3). In Figure 3, while DL application 20 (app #A, app #B, and app #C in the figure) is running, the computing resources allocated to each DL application 20 are dynamically changed in accordance with the changes in the DL application 20 process over time. Computing resources may include the memory allocation.

[0031] In (2), the memory allocation for application #B is increased compared to case (1) (see symbol T1). Also, in (3), the memory allocation for application #C is increased compared to case (2) (see symbol T2). (3) indicates that memory allocation for application #B becomes unnecessary because the data of application #B is saved to the main memory for the host (CPU10a) or because the process terminates (see symbol T2).

[0032] Figure 4 is a schematic diagram showing the configuration of a computing resource management system 1 according to one embodiment. In Figure 4, the computing resource management system 1 includes a control unit 100.

[0033] The DL application 20 calls functions included in the DL framework 30. The DL framework 30 may refer to foundational software and a universally applicable design model or general processing pattern for efficiently advancing deep learning. The DL framework 30 may also be a deep learning library. The DL framework 30 may be a framework such as TensorFlow, PyTorch, Keras, MXNet, and Chainer. The DL framework 30 is publicly known, and its description is omitted.

[0034] The device ID used in DL application 20 to identify a device (computational resource) can be called a virtual device ID. The virtual device ID may be used (described) explicitly (or implicitly) in DL application 20. The virtual device ID is an example of a primary identifier used to identify computational resources described in a deep learning application.

[0035] On the other hand, in DL framework 30, the device ID that identifies a device that is recognized (managed) and used for computation can be called a physical device ID. A physical device is an example of a physical computing resource.

[0036] As shown in Figure 4, the control unit 100 includes a device usage detection unit 101, a device management unit 102, a tensor tracking and movement unit 103, and a scheduling unit 104. However, the scheduling unit 104 may be implemented as an external system, such as a scheduler (not shown). In this case, the control unit 100 does not need to include the scheduling unit 104.

[0037] Normally, the DL application 20 would access the DL framework 30 for reading and other operations. However, in this computing resource management system 1, the control unit 100 functions as an intermediary module between the DL application 20 and the DL framework 30.

[0038] The DL application 20 makes an API (Application Programming Interface) call to the DL framework 30. An API is an interface that enables the exchange of information and functions between software applications. APIs exchange data in the form of sending a call (request) and receiving a response. The API call to the DL framework 30 by the DL application 20 is an example of a call to a deep learning framework during the execution of a deep learning application.

[0039] The device usage detection unit 101 intercepts the DL application 20's API call to the DL framework 30 and detects device usage based on this API call (see arrow A1 in Figure 4). Based on the API call, the device usage detection unit 101 obtains the tensor being used and a virtual device ID that identifies the device (computational resource) described by the DL application 20.

[0040] Furthermore, the device usage detection unit 101 notifies the device management unit 102 of the acquired virtual device ID (see arrow A2 in Figure 4).

[0041] The virtual device ID is used not only as the device ID held by the tensor used in the calculation, but also as the device ID that indicates the output device for the output tensor.

[0042] The device usage detection unit 101 sends a device usage notification to the device management unit 102. The device usage notification may include a virtual device ID and tensor data.

[0043] The device usage detection unit 101 also obtains an object from the API call that stores a set of the data to be processed and information about the device on which the data is placed, and notifies the device management unit 102. This object is called a "tensor (tensor object)". The virtual device ID may include the device ID held by the tensor (included in the tensor) and a device ID that indicates the output destination device of the output tensor.

[0044] Furthermore, the device usage detection unit 101 monitors the return value, which is the response from the DL framework 30 to the API call (see arrow A3 in Figure 4). If the return value is a tensor, the device usage detection unit 101 obtains the tensor (see arrow A4 in Figure 4), notifies the tensor tracking / movement unit 103 (described later) of the tensor, and has the tensor tracking / movement unit register it as a new tensor (see arrow A5 in Figure 4).

[0045] The device management unit 102 manages the correspondence (mapping) between virtual device IDs and physical device IDs. A physical device ID is an example of a second identification that identifies the physical computing resources managed by the deep learning framework. The device management unit 102 modifies the allocation map 105, which is the correspondence between virtual device IDs and physical device IDs, according to the execution status of the DL application 20.

[0046] The device management unit 102, upon receiving a virtual device ID from the device usage detection unit 101, generates an API call to the DL framework 30 using the physical device ID, if a corresponding physical device ID has already been determined. Specifically, the device management unit 102 refers to the assignment map 105 to obtain the physical device ID corresponding to the notified virtual device ID. Then, using the obtained physical device ID, the device management unit 102 replaces (i.e., substitutes) the virtual device ID included in the DL framework API call with the physical device ID to generate a new API call (see arrow A6 in Figure 4). The device management unit 102 sends the generated new API call to the DL framework 30 (see arrow A7 in Figure 4).

[0047] On the other hand, if the corresponding physical device ID is not registered (undetermined) in the allocation map 105, the device management unit 102 queries the scheduling unit 104 (see arrow A8 in Figure 4) to secure a device such as a GPU 10b as a computing resource corresponding to the DL application 20 process. The scheduling unit 104 notifies the device management unit 102 of the physical device ID of the secured device (see arrow A9 in Figure 4). The device management unit 102 associates the physical device ID that identifies the secured device with the virtual device ID notified by the device usage detection unit 101 and registers it in the allocation map 105 as a new second identification.

[0048] Furthermore, the device management unit 102 determines that a device such as the GPU 10b that has not been used for a certain period of time is inactive and instructs the tensor tracking and movement unit 103 to save the tensor to host memory (see arrow A10 in Figure 4). The device management unit 102 switches a saving flag, which indicates that the tensor data is being saved, ON or OFF.

[0049] The device management unit 102 may further notify the scheduling unit 104 that it will return the usage rights of the GPU 10b that has not been used for a certain period of time, and may update the allocation map 105 by removing the physical device ID of the GPU 10b from the allocation map 105.

[0050] Figure 5 illustrates the correspondence between virtual device IDs and physical device IDs. As illustrated in the allocation map 105a, virtual device IDs and physical device IDs do not necessarily have to correspond in order (number) (see symbol T3). For example, the scheduling unit 104 may appropriately allocate an available physical device (such as a GPU 10b) when it receives an inquiry about available physical devices, and therefore the values ​​(e.g., numbers) used for the corresponding virtual device ID and physical device ID do not necessarily have to match.

[0051] Furthermore, as illustrated in the assignment map 105b, the correspondence may include relationships in which multiple virtual device IDs (GPU#V1 and GPU#V2 in the example shown in Figure 5) are associated with a single common physical device ID (GPU#P1 in the example shown in Figure 5) (see reference numeral T4).

[0052] For example, if there is ample resources available on GPU#P1 (such as the amount of main memory allocated), the number of GPU10b (devices) used by a single process can be reduced. For instance, a download application 20 that uses three GPU10b can be run on a system with only two GPU10b.

[0053] Furthermore, as illustrated in allocation map 105c, the virtual device IDs of the first computing resource, GPU10b (GPU#V1 and GPU#V3 in the example shown in Figure 5), may be associated with the physical device IDs of the second computing resource, CPU10a (host) (CPU#P1 in the figure) (see symbol T5). This makes it possible to realize functions similar to unified memory technology in the DL application layer. In this case, the data corresponding to GPU#V1 and GPU#V3 is saved to host memory. As a physical device, the CPU10a (host) can save and reserve data, thereby saving memory usage in the main memory of GPU10b.

[0054] The tensor tracking and movement unit 103 manages the tensor to be tracked. The tensor tracking and movement unit 103 tracks the tensor belonging to the virtual device ID and records it in tensor management information that does not illustrate the physical device where the actual data resides. The tensor tracking and movement unit 103 may manage tensors for each virtual device ID. Note that the tensor management information may be included as part of each tensor (D#1, D#2, ...).

[0055] The tensor tracking and movement unit 103 receives an instruction (data movement instruction) from the device-compatible management unit 102 to save the tensor to host memory (see arrow A10 in Figure 4). In accordance with this data movement instruction from the device-compatible management unit 102, the tensor tracking and movement unit 103 moves the data of the tensor between devices, including the CPU 10a (host) (i.e., between CPU 10a, GPU 10b-1, and GPU 10b-2) before the tensor is used in actual calculations. The tensor tracking and movement unit 103 notifies the device-compatible management unit 102 of the completion of the data movement (see arrow A11 in Figure 4).

[0056] Figure 6 shows an example of tensor data managed for each virtual device ID. In the example shown in Figure 6, a set of managed tensors is associated with each virtual device ID, and a mapping is created that associates a list of tensor objects with each virtual device.

[0057] Specifically, tensor objects D#1 and D#2 are managed for the virtual device (#V1) with virtual device ID GPU#V1. Similarly, tensor objects D#3, D#4, and D#5 are managed for the virtual device (#2V2) with virtual device ID GPU#2V2. Each tensor object D#1 to D#5 contains a pointer (reference) to data located in the main memory of CPU10a or the main memory of GPU10b (physical device #1 memory and physical device #2 memory in the diagram). In other words, each tensor object has a reference (pointer) to data located on the physical device.

[0058] Figure 7 shows an example of the process of moving tensor data between devices. In Figure 7, the tensor tracking and movement unit 103 shows an example of moving the tensor data of tensor object D#3, which corresponds to the virtual device with virtual device ID #2 (GPU#V2), from the main memory of CPU10a (CPU memory in the figure) to the main memory of GPU10b (physical device #1 memory in the figure). The tensor tracking and movement unit 103 may move tensor data between devices by changing the value of the pointer to the tensor data managed for each virtual device ID.

[0059] The scheduling unit 104 manages the computing resources (devices such as CPU 10a and GPU 10b) for the entire system. The scheduling unit 104 also receives inquiries (device requests) for available physical devices from the device management unit 102 (see arrow A8 in Figure 4). In response to these device requests from the device management unit 102, the scheduling unit 104 allocates devices (computing resources) and notifies the device management unit 102 of the physical device ID of the allocated device (see arrow A9 in Figure 4).

[0060] The scheduling unit 104 appropriately utilizes the computing resources of the CPU 10a and GPU 10b by allocating the GPU 10b in real time and preferentially to processes that are expected to have high execution efficiency, even during program processing that utilizes the GPU. The scheduling unit 104 allocates the CPU 10a and GPU 10b to each process of the DL application 20 in a time-exclusive manner according to requests from the DL application 20.

[0061] In one example, the scheduling unit 104 registers the execution of a process as a target for management of the allocation status of the first computing resource, the GPU 10b, and when a notification is output from the process requesting the allocation of the GPU 10b, it determines whether or not there is a GPU 10b available for allocation to the process. If there is a GPU 10b available for allocation, the scheduling unit 104 may allocate the process to the GPU 10b, or if there is no GPU 10b available for allocation, it may allocate the process to the second computing resource, the CPU 10a.

[0062] (B) Operation A method for managing computing resources in a computing resource management system 1 according to one embodiment configured as described above will be explained with reference to Figures 8 to 11.

[0063] The processing of the device usage detection unit 101 of the computing resource management system 1 according to one embodiment will be explained according to the flowchart (steps S11 to S15) shown in Figure 8. The device usage detection unit 101 monitors the DL framework API (step S11).

[0064] If the device usage detection unit 101 detects device usage (see the YES route in step S12), it sends a device usage notification to the device management unit 102 (step S13). Specifically, the device usage detection unit 101 obtains a virtual device ID that identifies the device (computational resource) described in the DL application 20. The device usage notification may include the virtual device ID and tensor data. After the execution of the process in step S13, the process returns to step S11. If no device usage is detected (see the NO route in step S12), the process proceeds to step S14.

[0065] If the device usage detection unit 101 detects a new tensor as a return value from a DL framework API call (see the YES route in step S14), it notifies the tensor tracking / movement unit 103 to register the new tensor (step S15). After the execution of step S15, the process returns to step S11. If no new tensor is detected (see the NO route in step S14), the process also returns to step S11.

[0066] The processing of the device-compatible management unit 102 of the computing resource management system 1 according to one embodiment will be explained according to the flowchart (steps S21 to S32) shown in Figure 9.

[0067] If the device management unit 102 does not receive a device usage notification from the device usage detection unit 101 (see NO route in step S21), or if device usage is not confirmed after a certain period of time (see YES route in step S22), the process proceeds to step S23. Until a certain period of time has elapsed, step S23 is bypassed, and the process proceeds to step S31.

[0068] In step S23, the device management unit 102 may determine that a device such as the GPU 10b that has not been used for a certain period of time is unavailable, and notify the tensor tracking and movement unit 103 that the device is unavailable, instructing it to save the tensor to the host (CPU 10a) main memory. The device management unit 102 may also notify the scheduling unit 104 to return the usage rights of the GPU 10b, delete the physical device ID of the GPU 10b from the allocation map 105, and update the allocation map 105.

[0069] However, the tensor tracking and movement unit 103 does not have to immediately execute the instruction to save (move) the tensor to the host's (CPU 10a) main memory and release the memory (code T6). By allocating the same device again when it becomes available and requested, overhead can be reduced.

[0070] When the device management unit 102 receives a device usage notification from the device usage detection unit 101 (see the YES route in step S21), it checks whether a physical device ID is assigned to the virtual device ID notified by the device usage detection unit 101 (step S24). If a corresponding physical device ID is assigned to the virtual device ID notified by the device usage detection unit 101 (YES route in step S24), the process proceeds to step S25.

[0071] In step S25, the device management unit 102 may determine whether the tensor data is being saved to the CPU 10a by checking whether the tensor saving flag is ON. If the tensor saving flag is ON, that is, if the tensor data is being saved (see the YES route in step S25), the process proceeds to step S26.

[0072] Furthermore, if the corresponding physical device ID is not assigned (unregistered) in step S24 (see NO route in step S24), the process proceeds to step S26.

[0073] On the other hand, if the tensor saving flag is OFF in step S25 (see NO route in step S25), proceed to step S29.

[0074] In step S26, the device management unit 102 requests the scheduling unit 104 to reserve a device such as a GPU 10b as a computing resource corresponding to the process of the DL application 20.

[0075] When the device management unit 102 receives an assignment response from the scheduling unit 104 that includes the assigned physical device ID, it updates the assignment map 105 by registering the physical device ID in association with the virtual device ID (step S27). The device management unit 102 turns off the tensor saving flag. Next, the device management unit 102 instructs the tensor tracking and movement unit 103 to move the data (step S28). As a result, as shown in Figure 7, the data is moved from the memory of the CPU 10a, which is saving data, to the memory of the GPU 10b.

[0076] If the assigned physical device ID does not match the device on which the tensor is actually located (see NO route in step S29), the device management unit 102 instructs the tensor tracking and movement unit 103 to move the data (step S28). The device on which the tensor is actually located refers to the device on which the tensor is actually located at the time of processing in step S28, as a result of the tensor movement performed by the tensor tracking and movement unit 103.

[0077] Basically, the device on which a tensor is located is the same as the device specified by the physical device ID. However, in the following cases, the device on which a tensor is located may differ from the device specified by the physical device ID. For example, if the scheduling unit 104 is notified that a device will not be used after a certain period of time, but the memory on that device has not yet been released, the device on which the tensor is located will differ from the device specified by the physical device ID. In this case, the physical device ID has been removed from the allocation map 105 so that the device will not be used, but the tensor is actually still located on that device. In this case, the memory contents are saved according to an additional request from the scheduling unit 104.

[0078] If the answer in step S29 is YES, the device management unit 102 uses the assignment map 105 (correspondence) to replace the virtual device ID (first identification) included in the API call with the physical device ID (second identification) to generate a new call and execute the API call (step S30).

[0079] Furthermore, if a memory release request is received from the scheduling unit 104 in step S31 (see the YES route in step S31), the device management unit 102 instructs the tensor tracking / movement unit 103 to save the data to the CPU 10a and updates the allocation map 105 (step S32). The device management unit 102 turns on the saving flag to indicate that the tensor data is being saved. If no memory release request has been made (see the NO route in step S31) or if the processing in step S32 is completed, the process returns to step S21.

[0080] Next, the processing performed by the scheduling unit 104 of the computing resource management system 1 according to one embodiment will be explained according to the flowchart (steps S41 to S47) shown in Figure 10.

[0081] The scheduling unit 104 waits until it receives a device request from the device management unit 102 (see NO route in step S41).

[0082] When the scheduling unit 104 receives a device request from the device management unit 102 (YES route in step S41), it determines whether there is an assignable device (step S42). If there is an assignable device (see YES route in step S42), the scheduling unit 104 notifies the device management unit 102 of the assigned physical device ID (step S43). The process then returns to step S41.

[0083] Furthermore, if there are no assignable devices (see NO route in step S42), the scheduling unit 104 determines whether there is a device that can free the device's memory (step S44). If there is a device that can free the memory (see YES route in step S44), the scheduling unit 104 requests the device management unit 102 to free the device memory (step S45). The process then returns to step S41.

[0084] On the other hand, if there is no device that can free the memory (see NO route in step S44), the scheduling unit 104 then checks whether a certain amount of time has elapsed since it started determining whether there is a device that can free the device's memory (step S46).

[0085] If a certain period of time has elapsed, that is, if a device with freeable device memory is not available for a certain period of time or longer (see the YES route in step S46), the process proceeds to step S47. The scheduling unit 104 notifies the device management unit 102 of the allocation failure (step S47). After that, the process returns to step S41.

[0086] If a certain period of time has not elapsed in step S46 (see NO route in step S46), the process returns to step S44. However, the scheduling unit 104 may be configured to notify of an allocation failure without waiting for a certain period of time to elapse. In other words, the process in step S46 may be omitted. The elapsed period of time may be designed as a timeout for the caller of the scheduling unit 104, i.e., the device management unit 102. In this case, the starting point of the period of time may be when the device request (allocation request) reaches the scheduling unit 104.

[0087] Next, the processing of the tensor tracking and movement unit 103 of the computational resource management system 1 according to one embodiment will be explained according to the flowchart (steps S51 to S54) shown in Figure 11.

[0088] When the tensor tracking / movement unit 103 receives a request from the device usage detection unit 101 to add a new tensor that is a return value of a DL framework API call (see the YES route in step S51), it adds that tensor to the tracking target (step S52). After that, the process proceeds to step S53. Also, if the device usage detection unit 101 has not received a request to add a new tensor that is a return value of a DL framework API call in step S51 (see the NO route in step S51), the process proceeds to step S53.

[0089] The tensor tracking and movement unit 103 checks whether it has received a data movement request from the device-compatible management unit 102 (step S53). If the tensor tracking and movement unit 103 has received a data movement request from the device-compatible management unit 102 (see the YES route in step S53), it executes the process in step S54.

[0090] In step S54, the tensor tracking and movement unit 103 moves all tensor data of the virtual device ID corresponding to the data movement request to the corresponding physical device. Then, the process returns to step S51. Also, in step S53, if the tensor tracking and movement unit 103 has not received a data movement request from the device management unit 102 (see NO route in step S53), the process returns to step S51.

[0091] (C) Effects As described above, the computing resource management system 1, as an example of an embodiment, obtains a first identification (virtual device ID) that identifies the computing resources described in the DL application 20 when calling the DL framework 30 during the execution of the DL application 20. Based on the first identification, the computing resource management system 1 refers to the correspondence (allocation map 105) between the first identification and a second identification (physical device ID) that identifies the physical computing resources managed by the DL framework 30, and obtains a second identification that corresponds to the first identification. The computing resource management system 1 replaces the first identification included in the call with the second identification to generate a new call, and sends the new call to the framework.

[0092] This allows the DL application 20 to transparently and dynamically specify the GPU 10b on which to run the DL application 20. The computing resource management system 1 does not need to modify the DL application 20 or the DL framework 30 itself. The DL application 20 does not need to fix the relationship between the virtual device ID and the physical device ID using environment variables or the like before execution.

[0093] Even when multiple DL applications 20-1 and 20-2 are running, programmers do not need to perform conservative memory allocation for each DL application 20-1 and 20-2, assuming a peak memory usage (worst-case scenario). Therefore, a decrease in the utilization rate of computing resources can be suppressed.

[0094] In general technology, it is assumed that the memory of GPU10b will remain occupied during job execution, regardless of whether calculations are being performed on GPU10b. For example, DL application 20 may continue to store parameters of a deep learning model on GPU10b, or DL ​​framework 30 may internally continue to reserve memory on GPU10b regardless of whether DL application 20 is using it. On the other hand, according to this embodiment, the scheduling unit 104 can control the allocation of GPU10b so that the allocation of GPU10b does not overlap in time, or even if there is a time overlap, it can take memory usage into consideration. Therefore, the programmer of DL application 20 can program assuming that all of GPU10b's memory is available, and thus does not need to perform the conservative memory allocation described above.

[0095] It is also possible to allocate GPU 10b to multiple DL applications 20 simultaneously within the allowable memory limits of GPU 10b without time-exclusive allocation. In this case as well, the scheduling unit 104 performs scheduling that takes memory usage into consideration. Therefore, DL applications 20 do not need to consider interference from memory usage by other jobs.

[0096] Since the control unit 100 acts as an intermediary between the DL application 20 and the DL framework 30 in the application layer of the DL application 20, the control unit 100 can instruct the application layer to save (transfer) data to host memory. Therefore, it is possible to avoid data transfer occurring at undesirable times for the execution of the DL application 20. Consequently, the impact on the performance degradation of the DL application 20 can be suppressed.

[0097] On the DL framework 30 side, it is difficult to change the assignment of device IDs, in other words, the correspondence between virtual device IDs and physical device IDs. However, there is a technology called AntMan that enables transparent data movement in the main memory of the CPU 10a and GPU 10b by modifying the DL framework 30 (14th USENIX Symposium on Operating Systems Design and Implementation, November 4-6, 2020, Wencong Xiao et al. https: / / www.usenix.org / conference / osdi20 / presentation / xiao). In this respect, the computing resource management system 1 of this embodiment, unlike AntMan, etc., eliminates the need to modify the DL framework 30, thus simplifying implementation. Since modifications do not need to be made in line with the development of the DL framework 30 itself, the increase in maintenance burden can be suppressed.

[0098] While general technologies such as virtual memory are known in fields such as operating systems (OS), the computational resource management system 1 of this embodiment is in the DL (Digital Download) field and is implemented at the DL application layer. Therefore, the computational resource management system 1 makes it easier to allocate and schedule resources in accordance with the execution characteristics of the DL application 20. Furthermore, implementation and deployment costs can be reduced. In addition, the portability of the DL application 20 can be improved.

[0099] The programmer of DL app 20 can specify explicit or implicit device assignments in a way that is independent of specific hardware environments. The programmer can specify device assignments without worrying about interference from other jobs. Specifically, the programmer can specify device assignments without worrying about issues such as memory oversubscription leading to insufficient memory capacity.

[0100] The computing resource management system 1 modifies the allocation map 105 according to the application's execution status. Therefore, it is possible to dynamically change the device allocation depending on the application's execution status.

[0101] Even when the DL application 20 includes multiple types of computing resources, such as a first computing resource (CPU 10a) and a second computing resource (GPU 10b), it is possible to specify the CPU 10a and GPU 10b that should run the DL application 20 transparently and dynamically.

[0102] The allocation map 105c includes at least a relationship in which a virtual device ID identifying the first computing resource (GPU 10b) is associated with a physical device ID identifying the second computing resource (CPU 10a). This allows data to be saved and secured on the CPU 10a (host) side, saving memory usage in the main memory of the GPU 10b. This enables the implementation of a technology similar to unified memory technology, which reduces the complexity of data movement between the CPU 10a and the GPU 10b, at the application layer. Since the timing of data movement between the CPU 10a and the GPU 10b can be controlled from the DL application 20, performance degradation of the DL application 20 can be suppressed depending on the data movement execution pattern.

[0103] Unified memory is a feature of general-purpose parallel computing platforms (parallel computing architectures) and programming models (e.g., CUDA) for GPUs provided by GPU vendors. It is a technology that transparently moves memory between devices for memory allocated using specific APIs.

[0104] There is also a technology called TGS (20th USENIX Symposium on Networked Systems Design and Implementation, April 17-19, 2023, Bingyang Wu et al. https: / / www.usenix.org / conference / nsdi23 / presentation / wu). TGS is a technology that allows applications to benefit from unified memory management without modification by replacing normal GPU memory allocation calls in the program with unified memory memory allocation APIs (at runtime). While using unified memory generally requires application modifications (such as rewriting memory allocation APIs), TGS makes it unnecessary to modify the application by transparently performing a replacement equivalent to this at runtime (at the device driver layer).

[0105] However, with TGS, the methods for obtaining information about the application's execution (application context) are limited (only the behavior seen from the outside is known), so for example, TensorFl The DL framework `ow` sometimes attempts to allocate significantly more memory than is actually used, making it difficult to estimate the truly necessary amount of memory based solely on external observation. Furthermore, as mentioned above, the timing of memory movement using unified memory cannot be controlled, which can affect application performance depending on the execution pattern. This embodiment solves these problems.

[0106] The allocation map 105b may include relationships in which multiple virtual device IDs are associated with a single common physical device ID. Therefore, when there is sufficient memory allocation (memory capacity) in the main memory of the GPU 10b, the number of GPU 10b (devices) used by a single process can be reduced. For example, a DL application 20 that uses three GPU 10b can be run on a system with only two GPU 10b.

[0107] If a physical device ID corresponding to a virtual device ID is not registered in the allocation map 105, the scheduling unit 104 reserves a physical device corresponding to the DL application 20 process. The device management unit 102 registers the physical device ID that identifies the reserved physical computing resource in the allocation map 105. Since the scheduling unit 104 manages the entire system's computing resources, it prevents situations where GPU 10a used by multiple DL applications 20-1 and 20-2 compete for memory, resulting in insufficient memory. Therefore, programmers can specify device allocations without worrying about interference from other jobs.

[0108] (D) Other Each component and process of this embodiment can be selected or omitted as needed, or combined as appropriate.

[0109] Furthermore, the disclosed technology is not limited to the embodiments described above, and can be implemented in various modified forms without departing from the spirit of this embodiment.

[0110] For example, in the embodiment described above, the computer 10 constituting the computing resource management system 1 is used as a single computing node and is executed on this computer 10, but it is not limited to this. A cluster configuration having multiple computing nodes (computers 10) may be formed, and the computing resource management system 1 may be constructed using this cluster configuration.

[0111] Furthermore, while the above-described embodiment shows a configuration example in which the computer 10 is equipped with one CPU 10a and two GPUs 10b-1 and 10b-2, the configuration is not limited to this. At least one of the CPU 10a and GPU 10b may be provided in one, two, or three or more units.

[0112] Furthermore, the above disclosure makes it possible for those skilled in the art to implement and manufacture this embodiment.

[0113] (E) Note The following additional information is disclosed regarding the embodiments described above.

[0114] (Note 1) In a computer with multiple computing resources, When calling a deep learning framework during the execution of a deep learning application, a first identifier is obtained that identifies the computational resources described in the application. Based on the first identification, the second identification corresponding to the first identification is obtained by referring to the correspondence between the first identification and a second identification that identifies the physical computing resources managed by the framework. A new call is generated by replacing the first identification included in the aforementioned call with the acquired second identification. Send the aforementioned new call to the framework. A computing resource management program characterized by causing the computer to perform the processing.

[0115] (Note 2) The aforementioned correspondence is changed according to the execution status of the application. The computing resource management program described in Appendix 1, characterized in that it causes the computer to perform the processing.

[0116] (Note 3) The computing resource management program according to Appendix 1, characterized in that the computing resources and physical computing resources described in the application each include a first computing resource and a second computing resource having lower processing performance than the first computing resource.

[0117] (Note 4) The computing resource management program according to Appendix 3, characterized in that the correspondence relationship includes at least a relationship in which the first identifier that identifies the first computing resource is associated with the second identifier that identifies the second computing resource.

[0118] (Note 5) The computing resource management program according to Appendix 1, characterized in that the correspondence relationship includes a relationship in which a plurality of first identities are associated with a common second identity.

[0119] (Note 6) If the second identification corresponding to the first identification is not registered in the aforementioned correspondence, the physical computing resources corresponding to the application process are secured, and the second identification identifying the secured physical computing resources is registered in the aforementioned correspondence. A computing resource management program according to any one of the appendices 1 to 5, characterized in that it causes the computer to perform the processing.

[0120] (Note 7) In a computer with multiple computing resources, When calling a deep learning framework during the execution of a deep learning application, a first identifier is obtained that identifies the computational resources described in the application. Based on the first identification, the second identification corresponding to the first identification is obtained by referring to the correspondence between the first identification and a second identification that identifies the physical computing resources managed by the framework. A new call is generated by replacing the first identification included in the aforementioned call with the acquired second identification. Send the aforementioned new call to the framework. A method for managing computing resources, characterized in that the computer performs the processing.

[0121] (Note 8) The aforementioned correspondence is changed according to the execution status of the application. The computing resource management method according to Appendix 7, characterized in that the processing is performed by the computer.

[0122] (Note 9) The computing resource management method according to Appendix 7, characterized in that the computing resources and physical computing resources described in the application each include a first computing resource and a second computing resource having lower processing performance than the first computing resource.

[0123] (Note 10) The computing resource management method according to Appendix 9, characterized in that the correspondence relationship includes at least a relationship in which the first identifier that identifies the first computing resource is associated with the second identifier that identifies the second computing resource.

[0124] (Note 11) The computing resource management method according to Appendix 7, characterized in that the correspondence relationship includes a relationship in which a plurality of first identities are associated with a common second identity.

[0125] (Note 12) If the second identification corresponding to the first identification is not registered in the aforementioned correspondence, the physical computing resources corresponding to the application process are secured, and the second identification identifying the secured physical computing resources is registered in the aforementioned correspondence. A computing resource management method according to any one of the appendices 7 to 11, characterized in that the computer performs the processing.

[0126] (Note 13) An information processing device having multiple computing resources, When calling a deep learning framework during the execution of a deep learning application, a first identifier is obtained that identifies the computational resources described in the application. Based on the first identification, the second identification corresponding to the first identification is obtained by referring to the correspondence between the first identification and a second identification that identifies the physical computing resources managed by the framework. A new call is generated by replacing the first identification included in the aforementioned call with the acquired second identification. Send the aforementioned new call to the framework. An information processing apparatus characterized by having a control unit that performs processing.

[0127] (Note 14) The information processing apparatus according to Appendix 13, characterized in that the control unit performs a process to change the correspondence relationship according to the execution status of the application. (Note 15) The information processing apparatus according to Appendix 13, characterized in that the computing resources and physical computing resources described in the application each include a first computing resource and a second computing resource having lower processing performance than the first computing resource.

[0128] (Note 16) The information processing apparatus according to Appendix 15, characterized in that the correspondence relationship includes at least a relationship in which the first identifier that identifies the first computing resource is associated with the second identifier that identifies the second computing resource.

[0129] (Note 17) The information processing apparatus according to Appendix 13, characterized in that the correspondence relationship includes a relationship in which a plurality of first identities are associated with a common second identity.

[0130] (Note 18) If the control unit finds that the second identification corresponding to the first identification is not registered in the correspondence, it reserves the physical computing resources corresponding to the application process and registers the second identification identifying the reserved physical computing resources in the correspondence. An information processing device according to any one of the appendices 13 to 17, characterized by performing processing. [Explanation of Symbols]

[0131] 1. Computational Resource Management System 10 Computers 10a CPU 10b-1, 10b-2, 10b GPU 10c memory 10d storage section 10e IF section 10f IO section 10g reading unit 10-hour program 10i recording media 10j bus 20-1, 20-2, 20 Download App 30 DL Frameworks 100 Control Unit 101 Device Usage Detection Unit 102 Device Support Management Department 103 Tensor Tracking and Movement Unit 104 Scheduling Unit 105 Assignment Map

Claims

1. In a computer with multiple computing resources, When calling a deep learning framework during the execution of a deep learning application, a first identifier is obtained that identifies the computational resources described in the application. Based on the first identification, the second identification corresponding to the first identification is obtained by referring to the correspondence between the first identification and a second identification that identifies the physical computing resources managed by the framework. A new call is generated by replacing the first identification included in the aforementioned call with the acquired second identification. Send the aforementioned new call to the framework. A computing resource management program characterized by causing the computer to perform the processing.

2. The computing resource management program according to claim 1, characterized in that it causes the computer to perform a process to change the aforementioned correspondence relationship according to the execution status of the application.

3. The computing resource management program according to claim 1, characterized in that the computing resources and physical computing resources described in the application each include a first computing resource and a second computing resource having lower processing performance than the first computing resource.

4. The computing resource management program according to claim 3, characterized in that the correspondence relationship includes at least a relationship in which the first identifier that identifies the first computing resource is associated with the second identifier that identifies the second computing resource.

5. The computing resource management program according to claim 1, characterized in that the correspondence relationship includes a relationship in which a plurality of first identities are associated with a common second identity.

6. If the second identification corresponding to the first identification is not registered in the aforementioned correspondence, the physical computing resources corresponding to the application process are secured, and the second identification identifying the secured physical computing resources is registered in the aforementioned correspondence. A computing resource management program according to any one of claims 1 to 5, characterized in that it causes the computer to perform the processing.

7. In a computer with multiple computing resources, When calling a deep learning framework during the execution of a deep learning application, a first identifier is obtained that identifies the computational resources described in the application. Based on the first identification, the second identification corresponding to the first identification is obtained by referring to the correspondence between the first identification and a second identification that identifies the physical computing resources managed by the framework. A new call is generated by replacing the first identification included in the aforementioned call with the acquired second identification. Send the aforementioned new call to the framework. A method for managing computing resources, characterized in that the computer performs the processing.

8. An information processing device having multiple computing resources, When calling a deep learning framework during the execution of a deep learning application, a first identifier is obtained that identifies the computational resources described in the application. Based on the first identification, the second identification corresponding to the first identification is obtained by referring to the correspondence between the first identification and a second identification that identifies the physical computing resources managed by the framework. A new call is generated by replacing the first identification included in the aforementioned call with the acquired second identification. Send the aforementioned new call to the framework. An information processing apparatus characterized by having a control unit that performs processing.

Images