Scheduling program, information processing device, and scheduling method

Abstract

To efficiently allocate a computing resource to a process.SOLUTION: A program includes: registering execution of a process, as a target of management of allocation state of a first computing resource 10b; determining, when a request is issued from the process to allocate the first computing resource 10b, whether there is a first computing resource 10b allocatable to the process; allocating, when the allocatable first computing resource 10b exists, the process to the first computing resource 10b; or allocating, when the allocatable first computing resource 10b does not exist, the process to a second computing resource.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a scheduling program, an information processing device, and a scheduling method. [Background technology]

[0002] It is known that using a GPU (Graphics Processing Unit) instead of a CPU (Central Processing Unit) to execute a deep learning application (hereinafter referred to as a deep learning app) improves processing performance (for example, Patent Document 1).

[0003] Since GPUs are more expensive than CPUs, it is important to effectively share a small number of GPUs among multiple processes.

[0004] Known job schedulers such as Slurm occupy GPUs from the start to the end of process execution, so jobs that exceed the number of GPUs cannot be executed simultaneously. Jobs that cannot secure GPUs are submitted to the job queue and wait until the process using the GPU has completely finished.

[0005] GPU preemption is also known as a method for efficiently utilizing GPUs. GPU preemption allows a job using the GPU to be stopped externally and the right to use the GPU to be transferred to another job. By periodically performing this GPU preemption, it is possible to switch GPU-using processes on an hourly basis, allowing subsequent jobs to use the GPU without waiting for the preceding job to completely stop. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Special Publication No. 2022-515302 [Patent Document 2] International Publication No. 2022 / 269870 [Patent Document 3] Japanese Patent Application Publication No. 2019-57303 Summary of the Invention [Problem to be solved by the invention]

[0007] However, with GPU preemption, the GPU continues to be occupied even if there is a period during process execution when the GPU is not in use. To prevent this GPU occupancy, it is necessary to switch GPU allocation according to the changes in the processing content of the deep learning application over time.

[0008] However, with GPU preemption, it is difficult to determine whether a process is using the GPU during execution. For example, in order for an application to be able to notify the external world of the timing of GPU usage, it is necessary to rewrite the application code.

[0009] In one aspect, the present invention aims to enable efficient allocation of GPUs to processes. [Means for solving the problem]

[0010] For this reason, this scheduling program causes a computer having a first computing resource and a second computing resource having processing performance lower than that of the first computing resource to execute a process of starting a process and managing the allocation state of the first computing resource, and in the starting process, registers the execution of the process as a target for managing the allocation state of the first computing resource, and when a notification requesting allocation of the first computing resource is output from the process, determines whether or not there is the first computing resource that can be allocated to the process, and if there is the first computing resource that can be allocated, assigns the process to the first computing resource, or if there is no first computing resource that can be allocated, assigns the process to the second computing resource. [Effects of the Invention]

[0011] According to one embodiment, computational resources can be efficiently allocated to processes. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram illustrating a configuration of a scheduling system according to an embodiment. [Figure 2] FIG. 2 is a block diagram showing an example of the hardware (HW) configuration of a computer that realizes the functions of the scheduling system according to one embodiment. [Figure 3] FIG. 2 is a diagram illustrating GPU allocation by a scheduler in a scheduling system according to an embodiment. [Figure 4] FIG. 1 is a diagram illustrating GPU allocation based on priority by a scheduler in a scheduling system according to an embodiment. [Figure 5] FIG. 10 is a diagram illustrating processing during execution of a user program in a scheduling system according to an embodiment. [Figure 6]FIG. 2 is a sequence diagram illustrating the flow of processing in a driver program, a user process, and a scheduler of a scheduling system according to an embodiment. [Figure 7] FIG. 1 is a diagram illustrating dynamic allocation of computing resources in a scheduling system according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present scheduling program, information processing device, and scheduling method will be described with reference to the drawings. However, the embodiments shown below are merely examples, and are not intended to exclude the application of various modifications and techniques not explicitly stated in the embodiments. In other words, the present embodiment can be implemented with various modifications within the scope of its purpose. Furthermore, each figure does not intend to include only the components shown in the figure, but can also include other functions, etc.

[0014] (A) Configuration FIG. 1 is a diagram showing a schematic configuration of a scheduling system 1 according to one embodiment, and FIG. 2 is a block diagram showing an example of the hardware (HW) configuration of a computer 10 that realizes the functions of the scheduling system 1 according to one embodiment.

[0015] (A-1) Hardware configuration example When multiple computers are used as HW resources for realizing the functions of the scheduling system 1, each computer may have the HW configuration exemplified in FIG.

[0016] As shown in Figure 2, computer 10 is an information processing device, and may include, as its hardware configuration, one or more (two in the example shown in Figure 2) CPUs 10a-1 and 10a-2, one or more (two in the example shown in Figure 2) GPUs 10b-1 and 10b-2, memory 10c, storage unit 10d, IF (Interface) unit 10e, IO (Input / Output) unit 10f, and reading unit 10g. Hereinafter, when there is no need to distinguish between CPUs 10a-1 and 10a-2, they will be referred to as CPU 10a. When there is no need to distinguish between GPUs 10b-1 and 10b-2, they will be referred to as GPU 10b.

[0017] The CPU 10a is an example of a processing unit that performs various controls and calculations, and is a control unit that executes various processes. The CPU 10a may be connected to each block in the computer 10 via a bus 10j so that they can communicate with each other. The bus 10j may be a PCIe (Peripheral Component Interconnect-Express) bus. The CPU 10a may be a multiprocessor including multiple processors, a multi-core processor having multiple processor cores, or a configuration having multiple multi-core processors.

[0018] The GPU 10b may be, for example, an accelerator such as a general-purpose computing on graphics processing unit (GPGPU). The GPU 10b may also be used to control screen display on an output device such as a monitor in the IO unit 10f. The GPU 10b may be configured as an accelerator that executes machine learning processing and inference processing using a machine learning model. It can be said that the GPU 10b has higher processing performance in machine learning processing and inference processing than the CPU 10a.

[0019] These CPUs 10a-1 and 10a-2 and GPUs 10b-1 and 10b-2 are computational resources allocated to a user program 104, which will be described later. The GPUs 10b-1 and 10b-2 are an example of a first computational resource, and the CPUs 10a-1 and 10a-2 are an example of a second computational resource.

[0020] The GPU 10b-1 may be referred to as GPU#1, and the GPU 10b-2 may be referred to as GPU#2. The CPU 10a-1 may be referred to as CPU#1, and the CPU 10a-2 may be referred to as CPU#2.

[0021] The memory 10c is an example of HW that stores various types of data, programs, and other information. The memory 10c may be, for example, a volatile memory such as a dynamic random access memory (DRAM) or a non-volatile memory such as a persistent memory (PM), or both.

[0022] The storage unit 10d is an example of HW that stores various types of data, programs, and other information. Examples of the storage unit 10d include various storage devices such as a magnetic disk device such as a hard disk drive (HDD), a semiconductor drive device such as a solid state drive (SSD), and a nonvolatile memory. Examples of nonvolatile memory include a flash memory, a storage class memory (SCM), and a read-only memory (ROM).

[0023] The storage unit 10d may store a program 10h (scheduling program) that realizes all or part of the various functions of the computer 10.

[0024] For example, the CPU 10a of the scheduling system 1 can implement a scheduling function, which will be described later, by loading a program 10h stored in the storage unit 10d into the memory 10c and executing it.

[0025] The IF unit 10e is an example of a communication IF that controls connections and communications between the computer 10 and other computers. For example, the IF unit 10e may include an adapter that complies with a LAN (Local Area Network) such as Ethernet (registered trademark) or optical communications such as FC (Fibre Channel). The adapter may support either or both wireless and wired communication methods. The program 10h may be downloaded to the computer 10 from a network via the communication IF and stored in the storage unit 10d.

[0026] The IO unit 10f may include one or both of an input device and an output device. Examples of input devices include a keyboard, a mouse, and a touch panel. Examples of output devices include a monitor, a projector, and a printer. The IO unit 10f may also include a touch panel or the like that combines an input device and an output device. The output device may be connected to the GPU 10b. The IO unit 10f may also be an input device or an output device of another information processing device that is remotely connected to the computer 10 via SSH (Secure Shell) or the like.

[0027] The reading unit 10g is an example of a reader that reads data and program information recorded on the recording medium 10i. The reading unit 10g may include a connection terminal or device to which the recording medium 10i can be connected or inserted. Examples of the reading unit 10g include an adapter that complies with USB (Universal Serial Bus) or the like, a drive device that accesses a recording disk, and a card reader that accesses a flash memory such as an SD card. Note that the recording medium 10i may store the program 10h, and the reading unit 10g may read the program 10h from the recording medium 10i and store it in the memory unit 10d.

[0028] Examples of the recording medium 10i include non-transitory computer-readable recording media such as magnetic / optical disks and flash memories. Examples of magnetic / optical disks include flexible disks, CDs (Compact Discs), DVDs (Digital Versatile Discs), Blu-ray Discs, and HVDs (Holographic Versatile Discs). Examples of flash memories include semiconductor memories such as USB memories and SD cards.

[0029] The above-described HW configuration of the computer 10 is an example. Therefore, the HW in the computer 10 may be increased or decreased (for example, adding or deleting any block), divided, integrated in any combination, or the HW may be added or deleted as needed.

[0030] (A-2) Example of functional configuration 1, the scheduling system 1 may exemplarily include functions as a scheduler 101, a driver program 102, a relay module 103, a user program 104, and a deep learning library 105. These functions may be realized by the hardware of a computer 10 (see FIG. 2).

[0031] The user program 104, the deep learning library 105, and the relay module 103 may be referred to as a user process 106. The user process 106 is an example of a process.

[0032] The driver program 102 activates a user program 104 (described later) and causes the user program 104 to communicate with a relay module 103 (described later).

[0033] More specifically, the driver program 102 causes the user program 104 to communicate with a scheduler 101 (described later) via a relay module 103. The driver program 102 also causes the user program 104 to load and execute a deep learning library 105 (described later).

[0034] The driver program 102 enables these communications by the user program 104 without modifying the source code of the deep learning program that implements the user program 104.

[0035] Normally, the user program 104 accesses the deep learning library 105, such as for reading. In the present scheduling system 1, the driver program 102 causes the user program 104 to access the deep learning library 105, such as for reading, via the relay module 103.

[0036] Furthermore, the user program 104 would normally transmit GPU requests and receive allocation notifications to and from the scheduler 101. In the present scheduling system 1, the driver program 102 causes the user program 104 to transmit and receive data and the like to and from the scheduler 101 via the relay module 103.

[0037] The driver program 102 executes the user program 104 after changing the semantics (variables, etc.) of module loading, thereby replacing the module loaded into the user program 104 .

[0038] As a result, the user program 104 performs an import or other access to the relay module 103 instead of performing an import access to the deep learning library 105. Also, the user program 104 transmits a GPU request to the relay module 103 instead of transmitting a GPU request to the scheduler 101.

[0039] For example, the driver program 102 may set, in the above-mentioned replacement module, a predetermined storage area in the memory 10c or the storage unit 10d that can be accessed by the relay module 103. Also, for example, the driver program 102 may set, in the replacement module, a communication port or the like used for communication with the relay module 103. Also, the driver program 102 may generate a job identifier that remains unchanged when the user program 104 is restarted, and transmit the generated job identifier to the relay module 103.

[0040] The driver program 102 may allow the user program 104 to use these storage areas, communication ports, etc., so that the user program 104 communicates with the relay module 103 as if it were communicating with the deep learning library 105.

[0041] The semantics changed by the driver program 102 may differ depending on the implementation language. For example, in the case of Python, the driver program 102 can change the behavior when the user program 104 executes an import statement by changing the variables "sys.path" and "sys.meta_path" built into the interpreter.

[0042] As a result, the driver program 102 redirects the access destination so that the reading performed by the user program 104 on the deep learning library 105 using an import statement is performed on the relay module 103 having an equivalent API (Application Programming Interface). Also, as a result, the driver program 102 causes the transmission of a GPU request or the like made by the user program 104 to the scheduler 101 to be performed via the relay module 103 having an equivalent API.

[0043] The driver program 102 changes the destination from which the deep learning library 105 is read by the user program 104 to the relay module 103. The driver program 102 also causes the user program 104 to automatically communicate with the scheduler 101 via the relay module 103 at an appropriate timing, such as the start of a learning iteration.

[0044] It can be said that the driver program 102 registers the execution of the user process 106 in the scheduler 101 as a target for managing the allocation state of the GPU 10b.

[0045] Furthermore, the driver program 102 may be said to cause a notification to be output when a specific process such as a specific API call is performed in the user process 106 (user program 104).

[0046] Furthermore, when it is necessary to move (change) the computational resource (device) on which the user program 104 is executed, the driver program 102 temporarily stops the user program 104 and restarts it on the new computational resource.

[0047] The user program 104 is a program that realizes a process of training (deep learning) a deep learning model (machine learning model) (not shown) and executes jobs related to deep learning. The user program 104 is, for example, a deep learning program.

[0048] The user program 104 calls the API of the deep learning library 105 during the learning process of the deep learning model, thereby calling the libraries provided by each API.

[0049] For example, in deep learning, preprocessing and machine learning (hereinafter sometimes simply referred to as learning) are repeatedly executed, and the user program 104 may call an API when transitioning from preprocessing to learning, and when transitioning from the end of learning to preprocessing of the next data, which is the subsequent process of the learning.

[0050] The user program 104 may also call an API when transitioning to a specific process with a relatively low load that does not require the use of the GPU 10b, or when transitioning to a specific process with a relatively high load that recommends the use of the GPU 10b. These API calls are examples of specific processes.

[0051] In the user program 104, the user program 104 itself may call a specific API of the deep learning library 105 through a hook while it is running.

[0052] A hook is a process that performs some operation starting from the call of a specific function or API, and may be provided as part of the functionality executed by the user program 104. For example, the user program 104 may execute a fit() function or the like using a hook to call the deep learning library 105.

[0053] The user program 104 may call the deep learning library 105 via the relay module 103 by loading and executing the relay module 103 using an import mechanism or the like.

[0054] In the present scheduling system 1, an API call made from the user program 104 to the deep learning library 105 is input to the relay module 103 via a hook. When the API call to the deep learning library 105 is input, the relay module 103 makes the call to the deep learning library 105 on behalf of the user program 104. The relay module 103 may input a response from the deep learning library 105 to the user program 104 as necessary. The deep learning library 105 may also send a response to the user program 104. It can be said that the relay module 103 relays communication between the user program 104 and the deep learning library 105.

[0055] Furthermore, the user program 104 in execution communicates with the scheduler 101 indirectly via the relay module 103 through a hook, thereby realizing dynamic allocation of computing resources.

[0056] The user program 104 communicates with the scheduler 101 by loading and executing the relay module 103 using an import mechanism or the like. It can be said that the relay module 103 relays communication between the user program 104 and the scheduler 101.

[0057] When a job is assigned, the scheduler 101 sends to the user program 104 via the relay module 103 an instruction to start the assigned job and information on the computational resource (GPU 10b) that will execute the job.

[0058] In addition, when the user program 104 moves a job that the user program 104 is executing to another computing resource, the scheduler 101 may input an instruction to stop the user program 104 or an instruction to restart the job on the computing resource after the move via the relay module 103.

[0059] In the present scheduling system 1, a computational resource (CPU 10a, GPU 10b) may be allocated to each of a plurality of user programs 104, and scheduling may be performed to execute jobs.

[0060] When the user program 104 receives job information for a job to be executed, it processes the information via the deep learning library 105. The user program 104 executes the learning process until the epoch ends.

[0061] When the user program 104 has finished using the computational resources, it notifies the scheduler 101 via the relay module 103 that the computational resources will be released (release notification). For example, when the user program 104 completes processing using the GPU 10b, it sends a GPU release notification to the scheduler 101 via the relay module 103. Also, when the user program 104 completes processing using the CPU 10a, it sends a CPU release notification to the scheduler 101 via the relay module 103.

[0062] The deep learning library 105 is software that functions as the base of the user program 104. The deep learning library 105 may be provided for each user program 104.

[0063] The deep learning library 105 is software that serves as a foundation for efficiently carrying out machine learning by the user program 104, and may include, for example, processing patterns that are frequently used in the user program 104.

[0064] The deep learning library 105 may include, for example, known deep learning libraries such as Keras and Pytorch Lightning. These deep learning libraries may function as APIs. The user program 104 uses these high-level APIs such as Keras and Pytorch Lightning.

[0065] The relay module 103 relays communication between the user program 104 and the scheduler 101 and communication between the user program 104 and the deep learning library 105.

[0066] A call (API call) made from the user program 104 to the deep learning library 105 is input to the relay module 103. When the API call to the deep learning library 105 is input to the relay module 103, the relay module 103 makes the API call to the deep learning library 105 on behalf of the user program 104. The relay module 103 may input a response from the deep learning library 105 to the user program 104. Therefore, it can be said that the user program 104 indirectly loads and executes the deep learning library 105 via the relay module 103.

[0067] The relay module 103 hooks a specific API, such as a fit() function, among API calls sent from the user program 104, and communicates with the scheduler 101. This communication may be, for example, transmission of information indicating the content of the specific API call made by the user program 104 to the deep learning library 105.

[0068] Furthermore, the call of a specific API includes, for example, the call of an API made when transitioning from preprocessing to learning in deep learning, and when transitioning from learning to preprocessing of the next data (post-process).

[0069] When the user program 104 calls a specific API of the deep learning library 105, the relay module 103 inserts a communication to the scheduler 101 immediately before or after the actual call of the function of the deep learning library 105. A response from the scheduler 101 to this communication is passed to the relay module 103, which then issues an instruction to switch the communication device or the like as necessary.

[0070] The relay module 103 also receives communications such as GPU requests and GPU release notifications sent from the user program 104 to the scheduler 101 .

[0071] The relay module 103 also receives communications such as GPU allocation result notifications sent from the scheduler 101 to the user program 104 .

[0072] The scheduler 101 allocates a computational resource (GPU 10b) to a user program 104. The scheduler 101 also manages the allocation state of jobs to computational resources in the scheduling system 1. The scheduler 101 manages the allocation state of the GPU 10b (first computational resource).

[0073] The scheduler 101 manages the allocation status of the GPU 10b using a resource table 111. For example, when a job is assigned to the GPU 10b, information identifying the GPU 10b is stored in the resource table 111 in association with information identifying the assigned job. When no job is assigned to the GPU 10b, the resource table 111 stores information identifying the GPU 10b in association with information indicating that the GPU 10b is free.

[0074] When a GPU allocation request is sent from a user process 106, the scheduler 101 checks whether there is a GPU 10b that can be allocated to the job of this user process 106, i.e., whether there is an available GPU 10b that is free. If there is an available GPU 10b that can be allocated, the scheduler 101 allocates the GPU 10b to the job and further returns a GPU allocation notification to the user process 106. Upon receiving the GPU allocation notification, the user process 106 processes the job (performs deep learning) using the allocated GPU 10b.

[0075] When a notification requesting allocation of the GPU 10b is output from the user process 106, it is determined whether or not there is a GPU 10b that can be allocated to this user process 106.

[0076] Furthermore, when a GPU allocation request is received from a user process 106, if the GPU 10b is allocated to another user process 106 and the GPU 10b cannot be allocated to the job, the scheduler 101 notifies the user process 106 that sent the GPU allocation request of the failure to allocate the GPU 10b.

[0077] The user process 106 that receives the GPU allocation failure may process the job (deep learning) using the CPU 10a.

[0078] It can be said that the scheduler 101 assigns the user process 106 to the GPU 10b when there is an allocatable GPU 10b, and assigns the user process 106 to the CPU 10a when there is no allocatable GPU 10b.

[0079] The scheduler 101 also allocates the GPU 10b when a communication is input from the relay module 103 by hooking a call to a specific API that is made in the user program 104 at the time of transition from preprocessing to learning and at the time of transition from learning to preprocessing of the next data.

[0080] That is, the scheduler 101 may treat the communication performed by hooking the calls of specific APIs made in the user program 104 when transitioning from preprocessing to learning and when transitioning from learning to preprocessing of the next data as an allocation request for the GPU 10b.

[0081] Communication performed by hooking a call to a specific API is an example of a notification output that is performed when a specific process is performed.

[0082] When a notification is output in the user process 106, the scheduler 101 determines whether or not there is a GPU 10b that can be assigned to the user process 106.

[0083] The scheduler 101 dynamically changes the allocation of the GPU 10b while performing pull-type and push-type communication with the user program 104.

[0084] Now, pull-type communication is mainly used for communication from the user program 104 to the scheduler 101. Pull-type communication enables fine-grained allocation according to the processing content.

[0085] For example, immediately before starting to use the GPU 10b, the user program 104 requests allocation of the GPU 10b to the scheduler 101 through pull-type communication. Note that if allocation of the GPU 10b is not performed in response to this request to use the GPU 10b, that is, if allocation of the GPU 10b fails, the user program 104 is executed by the CPU 10a.

[0086] Furthermore, immediately after the end of use of the GPU 10b, the user program 104 notifies the scheduler 101 of the release of the GPU 10b by pull-type communication. That is, after the user process 106 has used the allocated GPU 10b (the first computational resource), it outputs a notification of the release of the computational resource (the release of the GPU 10b).

[0087] On the other hand, push-type communication is mainly used for communication from the scheduler 101 to the user program 104 .

[0088] The scheduler 101 uses push-type communication to request, for example, the release of GPU 10b from the user program 104 that is using GPU 10b. The request to release GPU 10b is an example of a resource allocation change request. In other words, the scheduler 101 communicates with the user program 104 to request a resource allocation change.

[0089] Furthermore, the scheduler 101 notifies the user program 104, which is waiting for the release of the GPU 10b (and is being executed by the CPU 10a), of the availability of the GPU 10b by means of push-type communication.

[0090] In general, push-type communication allows dynamic allocation changes, such as changing from a process with a low allocation priority to a process with a high priority.

[0091] For example, when releasing the GPU 10b, the scheduler 101 restarts the process so that the memory resources of the GPU 10b that are in use can be reused, thereby releasing the memory resources of the GPU 10b.

[0092] FIG. 3 is a diagram illustrating GPU allocation by the scheduler 101 in the scheduling system 1 according to an embodiment.

[0093] Reference symbol A in FIG. 3 shows an example in which GPU allocation is successful. The user process 106 (user process #1) makes a GPU request to the scheduler 101 (see symbol P1). The scheduler 101 refers to the resource table 111 and confirms that GPU #0 is free and available for allocation.

[0094] The scheduler 101 transmits a GPU allocation notification indicating that GPU 10b is available for use (see symbol P2). The scheduler 101 registers the allocation of GPU #0 to user process #1 in the resource table 111. This series of communications indicated by symbols P1 and P2 corresponds to pull-type communications.

[0095] The user process 106 executes the user program 104 using the allocated GPU 10b.

[0096] Also, reference symbol B in FIG. 3 shows an example in which GPU allocation fails. The user process 106 (user process #1) is currently executing the user program 104 using the GPU 10b (GPU #0) (see symbol P0).

[0097] In this state, the user process 106 (user process #2) makes a GPU request to the scheduler 101 (see symbol P1). The scheduler 101 refers to the resource table 111 and confirms that GPU #0 is already allocated and cannot be allocated.

[0098] The scheduler 101 notifies the user process #2 of a GPU allocation failure, indicating that the GPU 10b is unavailable (see symbol P2). The series of communications indicated by symbols P1 and P2 corresponds to pull-type communications. The user process #2 executes the user program 104 using the CPU 10a (CPU #0) (see symbol P3).

[0099] The scheduler 101 may schedule jobs based on the priority set for the jobs.

[0100] The priority may be set, for example, by the user process 106. The user process 106 may set the priority for a job using various methods. For example, the user process 106 may set the priority according to the dummy job execution status in the job information of the job.

[0101] The user process 106 executes a dummy job to measure the performance of each computing resource before executing the user program 104. The dummy job may be a part of a job executed by the user program 104, and may be, for example, one epoch of machine learning processing.

[0102] The user process 106 executes the dummy job on each of the CPU 10a and the GPU 10b, and acquires the execution times (CPU execution time and GPU execution time). The CPU execution time and GPU execution time may be acquired from, for example, the deep learning library 105.

[0103] The user process 106 calculates an acceleration rate based on the CPU execution time and the GPU execution time. The user process 106 calculates the acceleration rate by dividing the CPU execution time by the GPU execution time. The user process 106 sets a job priority based on the calculated acceleration rate. For example, the user process 106 may set the calculated acceleration rate as a job priority value.

[0104] FIG. 4 is a diagram illustrating GPU allocation based on priority by the scheduler 101 in the scheduling system 1 according to an embodiment.

[0105] It should be noted that between user process #1 and user process #2, user process #2 has a higher acceleration and a higher priority.

[0106] The user process 106 (user process #1) is currently executing the user program 104 using the GPU 10b (GPU #0) (see symbol P0).

[0107] In this state, the user process 106 (user process #2) makes a GPU allocation request to the scheduler 101 through pull-type communication (see symbol P1).

[0108] The scheduler 101 refers to the resource table 111 and confirms that GPU #0 is already allocated and is not available for allocation.

[0109] The scheduler 101 requests the user process #1, which has a low priority, to release the GPU 10b by push-type communication (see symbol P2).

[0110] In response to this GPU release request, user process #1 switches to executing user program 104 using CPU 10a (see symbol P3). This releases GPU 10b. Furthermore, when releasing GPU 10b, user process #1 restarts user process #1 so that the GPU memory resources that were in use can be reused by other user processes. This releases the GPU memory. User process #1 notifies scheduler 101 of the release of GPU 10b.

[0111] The scheduler 101 changes the allocation of GPU 10b and allocates it to user process #2 (see symbol P4). The scheduler 101 sends a GPU allocation notification indicating that GPU 10b is available to user process #2 via push-type communication. The scheduler 101 registers in the resource table 111 that GPU #0 has been allocated to user process #2.

[0112] The user process #2 executes the user program 104 using the allocated GPU 10b (see symbol P5).

[0113] (B) Operation The process performed when the user program 104 is executed in the scheduling system 1 according to the embodiment configured as described above will be described with reference to FIG.

[0114] The driver program 102 changes the import mechanism by changing the semantics (variables, etc.) of module loading in the user program 104 (see symbol P1). The driver program 102 starts the user program 104 (see symbol P2). In the user program 104, the module to be loaded is replaced.

[0115] Furthermore, when a computational resource is switched, the user process 106 (user program 104) is temporarily stopped and then resumed on the new computational resource. Specifically, when the user process 106 receives an allocation notification from the scheduler 101 and detects a change in the computational device, the user process 106 temporarily stops (terminates) its own execution, returns control to the driver program 102, and restarts the user process 106 on another computational device.

[0116] The user program 104 performs an import access to read the deep learning library 105. This import access is performed on the relay module 103 (see symbol P3).

[0117] The relay module 103 reads the deep learning library 105 instead of the user program 104 and uses its functions (see symbol P4).

[0118] Furthermore, the relay module 103 hooks a specific API (such as a fit() function) into the communication from the user program 104 to the scheduler 101, and inserts communication to the scheduler 101 (see symbol P5).

[0119] The scheduler 101 checks the allocation status of the GPU 10b by referring to the resource table 111, and if allocation is possible, allocates and secures the GPU 10b to the user program 104 (see symbol P6). The scheduler 101 registers in the resource table 111 that GPU #0 has been allocated to the user process #1.

[0120] The scheduler 101 transmits a GPU allocation notification via the relay module 103. The user process 106 executes the user program 104 using the allocated GPU 10b via the deep learning library 105 (see symbol P7).

[0121] Next, the processing flow in the driver program 102, the user process 106, and the scheduler 101 of the scheduling system 1 according to one embodiment will be described with reference to the sequence diagram shown in FIG.

[0122] The driver program 102 changes the semantics (variables, etc.) of module loading and executes (starts) the user program 104, thereby replacing the module loaded into the user program 104 (see symbol S1).

[0123] The user process 106 starts a job (see symbol S2). When the job is started, a job ID is assigned to the job. In Fig. 6, an example is shown in which "Job#2" is assigned as the job ID.

[0124] The user process 106 transmits a request message requesting the GPU 10b to the scheduler 101 (see reference symbol S3). This request message may include information such as the job ID of the job executed by the user process 106, as well as the acceleration rate or priority.

[0125] The scheduler 101 refers to the resource table 111 and confirms that GPU#1 is unallocated (free) and available for allocation (see reference symbol S4). The scheduler 101 registers in the resource table 111 that GPU#1 has been allocated to job#2.

[0126] The scheduler 101 sends an allocation result notification to the user process 106 indicating that GPU#1 has been allocated to job#2 (see reference symbol S5). The user process 106 processes job#2 by having the user program 104 executed by GPU#1.

[0127] When job#2 is completed, the user process 106 notifies the scheduler 101 of the release of GPU#1 (notification of GPU release, notification of release of computational resources) (see reference symbol S6).

[0128] Upon receiving the release notification of the GPU 10b, the scheduler 101 updates the resource table 111 to make the released GPU 10b available (see reference numeral S7).

[0129] Thereafter, the user process 106 requests the scheduler 101 for the GPU 10b, the scheduler 101 allocates the GPU 10b in response to this request, the user process 106 executes the job and releases the GPU 10b, and so on are repeatedly executed (reference S8).

[0130] Furthermore, when the scheduler 101 changes the allocation of computing resources (devices) and switches devices, the user process 106 stops the user process 106 (see symbol S9), and the driver program 102 restarts the user process 106 on the destination computing device (symbol S10).

[0131] (C) Effects As described above, according to the scheduling system 1 as an example of an embodiment, the driver program 102 changes the import mechanism by changing the semantics of module loading in the user program 104, and starts the user program 104. As a result, the module to be loaded is replaced in the user program 104, and communication is established between the user program 104 and the scheduler 101 via the relay module 103.

[0132] This allows communication between the user program 104 and the scheduler 101 to be automatically performed at an appropriate timing, such as the start of a learning iteration. For example, the user program 104 can notify the scheduler 101 of a GPU allocation request via the relay module 103 immediately before starting to use the GPU 10b. Also, the user program 104 can notify the scheduler 101 of GPU release via the relay module 103 immediately after finishing using the GPU 10b.

[0133] Therefore, the GPU 10b can be appropriately allocated to the job of the user program 104.

[0134] Furthermore, as described above, the driver program 102 changes the semantics of module loading in the user program 104, so that the module to be loaded in the user program 104 is swapped, and the API call made from the user program 104 to the deep learning library 105 is input to the relay module 103 via a hook.

[0135] Then, the relay module 103 hooks a specific API, such as a fit() function, among the API calls sent from the user program 104, and communicates (notifies) the scheduler 101.

[0136] The scheduler 101 allocates the GPU 10b when a notification is input from the relay module 103 by hooking a specific API call made at the transition from preprocessing to learning and at the transition from learning to preprocessing of the next data in the user program 104. That is, in this scheduling system 1, the timing of the call of a specific process (API) from the user program 104 is used to issue a communication from the user process 106 to the scheduler 101 at an appropriate timing.

[0137] For example, when the user program 104 to which the GPU 10b is currently assigned shifts from learning to preprocessing of the next data, the scheduler 101 can assign the GPU 10b to another scheduler 101. That is, the GPU 10b can be flexibly assigned to the user program 104, and the GPU 10b can be used effectively. Furthermore, fine-grained switching of computing resources can be achieved according to the processing content of the user program 104.

[0138] FIG. 7 is a diagram for explaining dynamic allocation of computing resources in the scheduling system 1 according to an embodiment.

[0139] FIG. 7 shows an example in which the scheduler 101 dynamically allocates the GPU 10b to the user process #1 and the user process #2.

[0140] In the present scheduling system 1, the scheduler 101 allocates the GPU 10b at the timing of transition from preprocessing to learning and at the timing of transition from learning to preprocessing of the next data in the user program 104. More specifically, the scheduler 101 allocates the GPU 10b to the user process 106 that is currently executing learning.

[0141] In FIG. 7, the periods (GPU processing) during which the GPU 10b was assigned to process jobs in the user processes #1 and #2 are indicated by shading.

[0142] The user process 106 to which the GPU 10b is not assigned executes the job using the CPU 10a. In Fig. 7, the period during which the job was processed using the CPU 10a (CPU processing) is indicated by white areas.

[0143] 7, GPU 10b is assigned to user process #1, but when user process #1 shifts from learning to preprocessing of the next data (see symbol P1), the computing device (GPU 10b) is switched, and GPU 10b is assigned to user process #2 that is in the middle of learning (see symbol P2). In user process #1, preprocessing is performed using CPU 10a.

[0144] Thereafter, when user process #2 shifts from learning to preprocessing of the next data and user process #1 shifts from preprocessing to learning (see symbol P3), GPU 10b is assigned to user process #1.

[0145] As a result, user process #1 performs learning using GPU 10b, which has high processing performance, and performs pre-processing, which does not require GPU 10, using CPU 10a. Meanwhile, user process #2 can also perform learning using GPU 10b while user process #1 is performing pre-processing. In other words, user process 106 does not unnecessarily occupy GPU 10b while executing processing that does not use GPU 10b, and user program 104 is not executed regardless of whether GPU 10b is allocated or not.

[0146] When using the conventional method, if GPU 10b is assigned to user process #1, user process #2 cannot use GPU 10b, and since only CPU 10a is used for all processing steps, the processing time of user process #2 becomes longer.

[0147] In contrast, in the present scheduling system 1, in a first user process 106 (user process #1) among multiple user processes 106, learning is performed using the GPU 10b, which has high processing performance, and pre-processing that does not require the GPU 10b is performed using the CPU 10a. As a result, by allocating and using the GPU 10b to another user process 106 (user process #2), the processing time of user process #2 can be shortened. In other words, the GPU 10b can be efficiently allocated to multiple user processes 106, thereby improving the utilization efficiency of the GPU 10b.

[0148] Furthermore, by using a hook, the user program 104 accesses the scheduler 101 and the user program 104 accesses the deep learning library 105 via the relay module 103. This allows the scheduler 101 and the like to access the internal state of the user program 104, including the execution timing of processes in the user program 104 and the contents of temporary memory (not shown). By utilizing this, the following effects can be achieved.

[0149] For example, the timing of resource allocation can be synchronized with the execution of the user program 104. In other words, the time from when a computing resource becomes necessary until it is allocated, and the time from when a computing resource becomes unnecessary until it is released can be shortened, thereby improving real-time performance.

[0150] Furthermore, information about the internal state of the user program 104 (for example, details of the neural network model being executed) can be included in the content of communication to the scheduler 101. This information can be used to optimally allocate resources to the user program 104.

[0151] Furthermore, the processing within the hook makes it possible to read the contents of the temporary memory of the GPU 10b, save them in the storage unit 10d, etc., and later restore them. This means that when releasing a computing resource, it is possible to completely release the memory resources specific to the computing resource.

[0152] When the user process 106 (user program 104) has finished using the computational resources (GPU 10b, CPU 10a), it notifies the scheduler 101 via the relay module 103 that the computational resources have been released. This allows the scheduler 101 to know that the process that used the computational resource (GPU 10b) has already finished using it and released the computational resource. This allows another user process 106 to use the computational resource after it has been used by the user process 106, without waiting for the user process 106 to completely terminate.

[0153] (D) Other The configurations and processes of this embodiment can be selected as needed, or can be combined as appropriate.

[0154] The disclosed technology is not limited to the above-described embodiment, and can be implemented in various modifications without departing from the spirit of the present embodiment.

[0155] For example, in the above-described embodiment, the computer 10 constituting the scheduling system 1 is used as a single computing node and is executed on this computer 10, but this is not limited to this. A cluster configuration having multiple computing nodes (computers 10) may be formed, and the scheduling system 1 may be constructed using this cluster configuration.

[0156] In the above-described embodiment, the computer 10 is provided with two CPUs 10a-1 and 10a-2 and two GPUs 10b-1 and 10b-2, but the present invention is not limited to this. At least one of the CPU 10a and the GPU 10b may be provided with one or three or more.

[0157] Furthermore, the above disclosure will enable those skilled in the art to implement and manufacture the present embodiment.

[0158] (E) Supplementary Note The following additional notes are provided regarding the above-described embodiments.

[0159] (Appendix 1) A computer having a first computing resource and a second computing resource having a processing performance lower than that of the first computing resource, Start the process, managing an allocation state of the first computing resource; causing the computer to execute a process; In the process of starting, registering the execution of the process as a target for managing the allocation state of the first computing resource; When a notification requesting allocation of the first computing resource is output from the process, determining whether or not there is the first computing resource available to be allocated to the process; assigning the process to the first computing resource if the first computing resource is available for allocation; If the first computing resource is not available for allocation, the process is allocated to the second computing resource. A scheduling program that causes the computer to execute a process.

[0160] (Appendix 2) The process of managing the allocation status includes: A communication is made to request a change in allocation of computing resources to the process. 2. The scheduling program according to claim 1, further comprising:

[0161] (Appendix 3) The process outputs a notification of release of the allocated first or second computing resource after using the allocated first or second computing resource. 3. The scheduling program according to claim 1 or 2,

[0162] (Appendix 4) A computer having a first computing resource and a second computing resource having a processing performance lower than that of the first computing resource, Start the process, managing an allocation state of the first computing resource; causing the computer to execute a process; In the process of starting, when a specific process is performed in the process, a notification is output; In the process of managing the allocation state, when the notification is output in the process, it is determined whether or not there is the first computing resource that can be allocated to the process; assigning the process to the first computing resource if the first computing resource is available for allocation; If the first computing resource is not available for allocation, the process is allocated to the second computing resource. A scheduling program that causes the computer to execute a process.

[0163] (Appendix 5) The process of managing the allocation status includes: A communication is made to request a change in allocation of computing resources to the process. 5. The scheduling program according to claim 4, further comprising:

[0164] (Appendix 6) The process outputs a notification of release of the allocated first or second computing resource after using the allocated first or second computing resource. 6. The scheduling program according to claim 4 or 5,

[0165] (Appendix 7) An information processing device having a first computational resource and a second computational resource having a processing performance lower than that of the first computational resource, Start the process, managing an allocation state of the first computing resource; causing the information processing device to execute the processing; In the process of starting, registering the execution of the process as a target for managing the allocation state of the first computing resource; When a notification requesting allocation of the first computing resource is output from the process, determining whether or not there is the first computing resource available to be allocated to the process; assigning the process to the first computing resource if the first computing resource is available for allocation; If the first computing resource is not available for allocation, the process is allocated to the second computing resource. An information processing device comprising a control unit that executes processing.

[0166] (Appendix 8) The process of managing the allocation status includes: A communication is made to request a change in allocation of computing resources to the process. 6. The information processing device according to claim 5, further comprising:

[0167] (Appendix 9) The process outputs a notification of release of the allocated first or second computing resource after using the allocated first or second computing resource. 9. The information processing device according to claim 7 or 8.

[0168] (Appendix 10) An information processing device having a first computational resource and a second computational resource having a processing performance lower than that of the first computational resource, Start the process, managing an allocation state of the first computing resource; causing the information processing device to execute the processing; In the process of starting, when a specific process is performed in the process, a notification is output; In the process of managing the allocation state, when the notification is output in the process, it is determined whether or not there is the first computing resource that can be allocated to the process; assigning the process to the first computing resource if the first computing resource is available for allocation; If the first computing resource is not available for allocation, the process is allocated to the second computing resource. An information processing device comprising a control unit that executes processing.

[0169] (Appendix 11) The process of managing the allocation status includes: A communication is made to request a change in allocation of computing resources to the process. 11. The information processing device according to claim 10, further comprising:

[0170] (Appendix 12) The process outputs a notification of release of the allocated first or second computing resource after using the allocated first or second computing resource. 12. The information processing device according to claim 10 or 11.

[0171] (Appendix 13) A computer having a first computing resource and a second computing resource having a processing performance lower than that of the first computing resource, Start the process, managing an allocation state of the first computing resource; causing the computer to execute a process; In the process of starting, registering the execution of the process as a target for managing the allocation state of the first computing resource; When a notification requesting allocation of the first computing resource is output from the process, determining whether or not there is the first computing resource available to be allocated to the process; assigning the process to the first computing resource if the first computing resource is available for allocation; If the first computing resource is not available for allocation, the process is allocated to the second computing resource. A scheduling method characterized in that the processing is executed by the computer.

[0172] (Appendix 14) The process of managing the allocation status includes: A communication is made to request a change in allocation of computing resources to the process. 14. The scheduling method of claim 13, further comprising the steps of:

[0173] (Appendix 15) The process outputs a notification of release of the allocated first or second computing resource after using the allocated first or second computing resource. 15. The scheduling method according to claim 13 or 14,

[0174] (Appendix 16) A computer having a first computing resource and a second computing resource having a processing performance lower than that of the first computing resource, Start the process, managing an allocation state of the first computing resource; causing the computer to execute a process; In the process of starting, when a specific process is performed in the process, a notification is output; In the process of managing the allocation state, when the notification is output in the process, it is determined whether or not there is the first computing resource that can be allocated to the process; assigning the process to the first computing resource if the first computing resource is available for allocation; If the first computing resource is not available for allocation, the process is allocated to the second computing resource. A scheduling method characterized in that the processing is executed by the computer.

[0175] (Appendix 17) The process of managing the allocation status includes: A communication is made to request a change in allocation of computing resources to the process. 17. The scheduling method of claim 16, further comprising the steps of: (Appendix 18) The process outputs a notification of release of the allocated first or second computing resource after using the allocated first or second computing resource. 18. The scheduling method according to claim 16 or 17, [Explanation of symbols]

[0176] 1. Scheduling System 10. Computers 10a-1, 10a-2, 10a CPU 10b-1, 10b-2, 10b GPU 10c memory 10d storage section 10e IF section 10f IO section 10g reading unit 10-hour program 10i recording media 10j bus 101 Scheduler 102 Driver Program 103 Relay Module 104 User Program 105 Deep Learning Libraries 106 User Processes 111 Resource Table

Claims

1. A computer having a first computing resource and a second computing resource having a processing performance lower than that of the first computing resource, Start the process, managing an allocation state of the first computing resource; causing the computer to execute a process; In the starting process, registering the execution of the process as a target for management of the allocation state of the first computing resource; When a notification requesting allocation of the first computing resource is output from the process, determining whether or not there is the first computing resource available to be allocated to the process; assigning the process to the first computing resource if the first computing resource is available for allocation; If the first computing resource is not available for allocation, the process is allocated to the second computing resource. A scheduling program that causes the computer to execute a process.

2. The process of managing the allocation status includes: and performing communication to request a change in allocation of computational resources to the process.

2. The scheduling program according to claim 1, further comprising a process for:

3. The process outputs a notification of release of the allocated first or second computing resource after using the allocated first or second computing resource.

3. The scheduling program according to claim 1, wherein the scheduling program is a program for scheduling a plurality of tasks.

4. A computer having a first computing resource and a second computing resource having a processing performance lower than that of the first computing resource, Start the process, managing an allocation state of the first computing resource; causing the computer to execute a process; In the process of starting, when a specific process is performed in the process, a notification is output; In the process of managing the allocation state, when the notification is output in the process, it is determined whether or not there is the first computing resource that can be allocated to the process; assigning the process to the first computing resource if the first computing resource is available for allocation; If the first computing resource is not available for allocation, the process is allocated to the second computing resource. A scheduling program that causes the computer to execute a process.

5. The process of managing the allocation status includes: A communication is made to request a change in allocation of computing resources to the process.

5. The scheduling program according to claim 4, further comprising a process.

6. The process notifies the release of the allocated first or second computational resource after using the allocated first or second computational resource.

6. The scheduling program according to claim 4 or 5.

7. An information processing apparatus having a first computational resource and a second computational resource having a processing performance lower than that of the first computational resource, Start the process, managing an allocation state of the first computing resource; causing the information processing device to execute the processing; In the step of starting, registering the execution of the process as a target for management of the allocation state of the first computing resource; When a notification requesting allocation of the first computing resource is output from the process, determining whether or not there is the first computing resource available to be allocated to the process; assigning the process to the first computing resource if the first computing resource is available for allocation; If the first computing resource is not available for allocation, the process is allocated to the second computing resource. An information processing device comprising a control unit that executes processing.

8. An information processing apparatus having a first computational resource and a second computational resource having a processing performance lower than that of the first computational resource, Start the process, managing an allocation state of the first computing resource; causing the information processing device to execute the processing; In the process of starting, when a specific process is performed in the process, a notification is output; In the process of managing the allocation state, when the notification is output in the process, it is determined whether or not there is the first computing resource that can be allocated to the process; assigning the process to the first computing resource if the first computing resource is available for allocation; If the first computing resource is not available for allocation, the process is allocated to the second computing resource. An information processing device comprising a control unit that executes processing.

9. A computer having a first computing resource and a second computing resource having a processing performance lower than that of the first computing resource, Start the process, managing an allocation state of the first computing resource; causing the computer to execute a process; In the starting process, registering the execution of the process as a target for management of the allocation state of the first computing resource; When a notification requesting allocation of the first computing resource is output from the process, determining whether or not there is the first computing resource available to be allocated to the process; assigning the process to the first computing resource if the first computing resource is available for allocation; If the first computing resource is not available for allocation, the process is allocated to the second computing resource. A scheduling method characterized in that the processing is executed by the computer.

10. A computer having a first computing resource and a second computing resource having a processing performance lower than that of the first computing resource, Start the process, managing an allocation state of the first computing resource; causing the computer to execute a process; In the process of starting, when a specific process is performed in the process, a notification is output; In the process of managing the allocation state, when the notification is output in the process, it is determined whether or not there is the first computing resource that can be allocated to the process; assigning the process to the first computing resource if the first computing resource is available for allocation; If the first computing resource is not available for allocation, the process is allocated to the second computing resource. A scheduling method characterized in that the processing is executed by the computer.

Images