Scheduling program, information processing device, and scheduling method

Abstract

To efficiently allocate a GPU to a process.SOLUTION: A scheduling program includes: predicting an acceleration rate that accompanies with execution of a deep learning model, with reference to an execution history 107 of a program in which a type, a batch size, and an acceleration rate of the deep learning model are associated, based on information regarding a job to be processed for the deep learning model; and determining an order of a program that allocates a calculation resource 10b, based on a prediction result of the acceleration rate.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a scheduling program, an information processing device, and a scheduling method. [Background technology]

[0002] It is known that using a GPU (Graphics Processing Unit) instead of a CPU (Central Processing Unit) to execute a deep learning application (hereinafter referred to as a deep learning app) improves processing performance (see, for example, Patent Document 1). In recent years, it can be said that a GPU is essential for executing deep learning apps.

[0003] Since GPUs are more expensive than CPUs, it is important to effectively share a small number of GPUs among multiple processes.

[0004] Known job schedulers such as Slurm occupy GPUs from the start to the end of process execution, so it is not possible to simultaneously execute jobs that exceed the number of GPUs. Jobs that cannot secure a GPU are submitted to the job queue and must wait until the process using the GPU has completely finished. Alternatively, the job is processed by a CPU with lower processing performance than a GPU.

[0005] GPU preemption is also known as a method for efficiently utilizing GPUs. GPU preemption allows a job using the GPU to be stopped externally and the right to use the GPU to be transferred to another job. By periodically performing this GPU preemption, it is possible to switch GPU-using processes on an hourly basis, allowing subsequent jobs to use the GPU without waiting for the preceding job to completely stop. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Special Publication No. 2022-515302 Summary of the Invention [Problem to be solved by the invention]

[0007] It is desirable to reduce the cost of switching between CPU and GPU and the execution time cost, and to process processes efficiently.

[0008] In one aspect, the present invention aims to enable efficient allocation of GPUs to processes. [Means for solving the problem]

[0009] Therefore, this scheduling program, based on information about the job to be processed for the deep learning model, refers to the execution history of the program, which corresponds the type of deep learning model, batch size, and acceleration rate, to predict the acceleration rate associated with the execution of the deep learning model, and causes the computer to perform a process of determining the order in which the programs will be allocated to computing resources based on the predicted acceleration rate. [Effects of the Invention]

[0010] According to one embodiment, computational resources can be efficiently allocated to processes. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating a configuration of a scheduling system according to an embodiment. [Figure 2] FIG. 2 is a block diagram showing an example of the hardware (HW) configuration of a computer that realizes the functions of the scheduling system according to one embodiment. [Figure 3] FIG. 1 is an image diagram showing the relationship between batch size and acceleration rate for processing a deep learning program. [Figure 4]FIG. 2 is a diagram illustrating an example of job execution history information in a scheduling system according to an embodiment. [Figure 5] 10 is a diagram illustrating a method for verifying a predicted acceleration rate range determined by an acceleration rate prediction unit of a scheduling system according to an embodiment. FIG. [Figure 6] 10 is a diagram illustrating the processing of a computation resource scheduling unit 103 of a scheduling system according to an embodiment. FIG. [Figure 7] 10 is a flowchart illustrating a training process for a deep learning model in a scheduling system according to an embodiment. [Figure 8] 8 is a flowchart for explaining details of the process of step S3 in the flowchart shown in FIG. 7. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present scheduling program, information processing device, and scheduling method will be described with reference to the drawings. However, the embodiments shown below are merely examples, and are not intended to exclude the application of various modifications and techniques not explicitly stated in the embodiments. In other words, the present embodiment can be implemented with various modifications within the scope of its purpose. Furthermore, each figure does not intend to include only the components shown in the figure, but can also include other functions, etc.

[0013] (A) Related technologies When scheduling the allocation of GPUs to multiple programs, it is effective to determine which programs should be assigned the GPU to speed up the learning process, and then determine the priority of GPU allocation.

[0014] The ratio (execution time ratio) between the execution time when processing is performed using a CPU and the execution time when the same processing is performed using a GPU can be called the acceleration rate.

[0015] One possible method to improve utilization of the GPU resource is to allocate the GPU to programs that are faster on the GPU compared to CPU execution. To achieve this method, it is effective to have each program execute dummy processes on the GPU before executing the program, and measure GPU performance based on the results of these dummy processes. The dummy processes can also be called dummy jobs. The execution of dummy processes can also be called dummy execution. The programs can be deep learning processing programs.

[0016] It is desirable to execute the dummy execution for several steps immediately before the start of the learning process of the deep learning program itself, so as to minimize the impact on the processing of the main deep learning program. Also, in order to quickly switch the GPU allocation to the program, it is necessary to set a high priority for the dummy execution in job scheduling.

[0017] Based on the execution results of such dummy jobs, the execution time of the learning process of the deep learning program itself is estimated.

[0018] Dummy execution is performed on both the CPU and GPU, and the execution time of the learning process on the CPU and the GPU are measured.Then, the ratio between the execution time when the process is performed using the CPU and the execution time when the same process is performed using the GPU (execution time ratio, acceleration rate) is calculated, and the GPU is assigned to the program with the highest acceleration rate.

[0019] However, since the processes executed in the dummy runs are not essential to the execution of the deep learning program, they incur execution time costs. Therefore, it is desirable to minimize the number of dummy runs.

[0020] Furthermore, because dummy execution has a high priority, it takes over the GPU used by other programs in order to measure performance. Furthermore, if the performance measurement results show that the program whose GPU has been taken over by the dummy execution has a high acceleration rate, it is necessary to return the GPU to the program that took over. In such cases, unnecessary GPU switching occurs. In other words, dummy execution incurs CPU / GPU switching costs. Therefore, from this perspective as well, it is desirable to reduce the number of dummy executions.

[0021] One of the purposes of the scheduling system 1 according to an example of the embodiment is to be able to reduce the number of dummy executions.

[0022] (B) Configuration FIG. 1 is a diagram showing a schematic configuration of a scheduling system 1 according to one embodiment, and FIG. 2 is a block diagram showing an example of the hardware (HW) configuration of a computer 10 that realizes the functions of the scheduling system 1 according to one embodiment.

[0023] (B-1) Hardware configuration example When multiple computers are used as HW resources for realizing the functions of the scheduling system 1, each computer may have the HW configuration exemplified in FIG.

[0024] 2, the computer 10 may illustratively include, as its hardware configuration, one or more (two in the example shown in FIG. 2) CPUs 10a-1 and 10a-2, one or more (two in the example shown in FIG. 2) GPUs 10b-1 and 10b-2, memory 10c, storage unit 10d, IF (Interface) unit 10e, IO (Input / Output) unit 10f, and reading unit 10g. Hereinafter, when there is no need to distinguish between the CPUs 10a-1 and 10a-2, they will be referred to as CPU 10a. When there is no need to distinguish between the GPUs 10b-1 and 10b-2, they will be referred to as GPU 10b.

[0025] The CPU 10a is an example of a processing unit that performs various controls and calculations, and is a control unit that executes various processes. The CPU 10a may be connected to each block in the computer 10 via a bus 10j so that they can communicate with each other. The CPU 10a may be a multiprocessor including multiple processors, a multi-core processor having multiple processor cores, or a configuration having multiple multi-core processors.

[0026] The GPU 10b controls screen display for an output device such as a monitor in the IO unit 10f. The GPU 10b may also have a configuration as an accelerator that executes machine learning processing and inference processing using a machine learning model.

[0027] The CPUs 10a-1 and 10a-2 and the GPUs 10b-1 and 10b-2 are examples of computational resources.

[0028] The memory 10c is an example of HW that stores various types of data, programs, and other information. The memory 10c may be, for example, a volatile memory such as a dynamic random access memory (DRAM) or a non-volatile memory such as a persistent memory (PM), or both.

[0029] The storage unit 10d is an example of HW that stores various types of data, programs, and other information. Examples of the storage unit 10d include various storage devices such as a magnetic disk device such as a hard disk drive (HDD), a semiconductor drive device such as a solid state drive (SSD), and a nonvolatile memory. Examples of nonvolatile memory include a flash memory, a storage class memory (SCM), and a read-only memory (ROM).

[0030] The storage unit 10d may store a program 10h (scheduling program) that realizes all or part of the various functions of the computer 10.

[0031] For example, the processor 10a of the scheduling system 1 can implement a scheduling function, which will be described later, by loading a program 10h stored in the storage unit 10d into the memory 10c and executing it. The storage unit 10d also implements the function of the job history storage unit 105 shown in FIG.

[0032] The IF unit 10e is an example of a communication IF that controls connections and communications between the computer 10 and other computers. For example, the IF unit 10e may include an adapter that complies with a LAN (Local Area Network) such as Ethernet (registered trademark) or optical communications such as FC (Fibre Channel). The adapter may support either or both wireless and wired communication methods. The program 10h may be downloaded to the computer 10 from a network via the communication IF and stored in the storage unit 10d.

[0033] The IO unit 10f may include one or both of an input device and an output device. Examples of input devices include a keyboard, a mouse, and a touch panel. Examples of output devices include a monitor, a projector, and a printer. The IO unit 10f may also include a touch panel that combines an input device and an output device. The output device may be connected to the GPU 10b.

[0034] The reading unit 10g is an example of a reader that reads data and program information recorded on the recording medium 10i. The reading unit 10g may include a connection terminal or device to which the recording medium 10i can be connected or inserted. Examples of the reading unit 10g include an adapter that complies with USB (Universal Serial Bus) or the like, a drive device that accesses a recording disk, and a card reader that accesses a flash memory such as an SD card. Note that the recording medium 10i may store the program 10h, and the reading unit 10g may read the program 10h from the recording medium 10i and store it in the memory unit 10d.

[0035] Examples of the recording medium 10i include non-transitory computer-readable recording media such as magnetic / optical disks and flash memories. Examples of magnetic / optical disks include flexible disks, CDs (Compact Discs), DVDs (Digital Versatile Discs), Blu-ray Discs, and HVDs (Holographic Versatile Discs). Examples of flash memories include semiconductor memories such as USB memories and SD cards.

[0036] The above-described HW configuration of the computer 10 is an example. Therefore, the HW in the computer 10 may be increased or decreased (for example, adding or deleting any block), divided, integrated in any combination, or the HW may be added or deleted as needed.

[0037] (B-2) Example of functional configuration 1, the scheduling system 1 may exemplarily include functions as a scheduler 101, an acceleration rate prediction unit 104, a job control unit 106, a user program 201, and a deep learning framework 202. These functions may be realized by the hardware of a computer 10 (see FIG. 2).

[0038] The user program 201 is a program that performs deep learning (training) of a deep learning model (machine learning model) not shown, and executes jobs related to deep learning that are sent (assigned) from the job control unit 106 described later.

[0039] The user program 201 uses libraries provided by each API (Application Programming Interface) by calling the API of the deep learning framework 202 in the learning process of the deep learning model. The user program 201 may call a library using, for example, a fit() function. In addition, a call to a library provided by the API (such as the fit() function) has a hook set in advance for acquiring objects of the deep learning model and input data (deep learning model object, input tensor information). The input tensor information may include the input tensor size, i.e., the batch size.

[0040] The information including the deep learning model object and the input tensor size (batch size) may be referred to as deep learning information. The deep learning information is an example of information about the job to be processed.

[0041] When a job is assigned, the job control unit 106 sends to the user program 201 an instruction to start the assigned job and information on the computational resources (CPU 10a, GPU 10b) that will execute the job.

[0042] In addition, when the user program 201 moves a job that the user program 201 is executing to another computing resource, the job control unit 106 inputs to the user program 201 an instruction to stop the user program 201 and an instruction to restart the job on the computing resource after the move.

[0043] In the present scheduling system 1, a computational resource (GPU 10b) is allocated to each of a plurality of user programs 201, and scheduling is performed to execute jobs.

[0044] Of the user programs 201, the user program 201 that is the target of scheduling may be called the user program 201 to be scheduled.

[0045] In addition, the user program 201 uses a high-level API such as Keras or Pytorch lightning, and the deep learning model to be trained is constructed by combining existing layers (FCN, CNN, LSTM, Dropout, Pooling, etc.).

[0046] When the user program 201 receives job information of a job to be executed from the job control unit 106 (described later), the user program 201 processes the job via the deep learning framework 202. The user program 201 executes the learning process until the epoch ends.

[0047] The deep learning framework 202 is software that functions as the base of the user program 201, and is provided corresponding to the user program 201. The user program 201 is executed on this deep learning framework 202. Therefore, a deep learning framework 202 may be provided for each user program 201.

[0048] The deep learning framework 202 is software that serves as a foundation for efficiently carrying out machine learning by the user program 201, and may include, for example, processing patterns that are frequently used in the user program 201 as a library.

[0049] The deep learning framework 202 may include, for example, the above-mentioned deep learning libraries such as Keras and Pytorch lightning. These deep learning libraries may function as APIs, and the user program 201 uses these high-level APIs such as Keras and Pytorch lightning.

[0050] The deep learning framework 202 acquires deep learning information (deep learning model object, input tensor) via a hook that is provided in advance for calling a library provided by an API such as a fit() function. The deep learning framework 202 acquires the type of deep learning model and the batch size from the acquired object.

[0051] In addition, the deep learning framework 202 transmits job information and deep learning information to the priority calculation unit 102, which will be described later.

[0052] The job information may include, for example, a job ID and a dummy job execution status. The job ID is identification information for identifying a job. The dummy job execution status may include at least one of information on CPU dummy job execution completion, CPU epoch execution time, GPU dummy job execution completion, and GPU epoch execution time.

[0053] "CPU dummy job executed" is included in the job information when the dummy job is executed by the CPU 10a for one epoch. Also, "CPU epoch execution time" is the execution time required for the dummy job to be executed by the CPU 10a for one epoch. "GPU dummy job executed" is included in the job information when the dummy job is executed by the GPU 10b. Also, "GPU epoch execution time" is the execution time required for the dummy job to be executed by the GPU 10b.

[0054] Furthermore, the deep learning framework 202 records the job execution history of the user program 201 in the job execution history information 107 of the job history storage unit 105.

[0055] When the learning process for the deep learning model by the user program 201 is completed, the deep learning framework 202 notifies the scheduler 101 of the resource information (GPU / CPU) passed from the computational resource scheduling unit 103.

[0056] In addition, the deep learning framework 202 transmits job information and deep learning information to the scheduler 101 (priority calculation unit 102).

[0057] The scheduler 101 allocates computing resources (CPUs 10a and 10b) to the user program 201.

[0058] Now, the execution time ratio (acceleration rate) between multiple types of processors depends on the performance of the computing resources and the program, so it cannot be known until it is actually executed. However, the applicant of the present application has discovered that for deep learning programs, the acceleration rate monotonically increases according to the batch size.

[0059] Figure 3 is an illustration showing the relationship between batch size and acceleration rate for deep learning program processing.

[0060] In Figure 3, the horizontal axis is the batch size and the vertical axis is the acceleration rate. As shown in Figure 3, the acceleration rate increases as the batch size increases, and decreases as the batch size decreases.

[0061] This is thought to be because a larger batch size reduces the number of GPU data transfers per epoch, thereby reducing the GPU transfer overhead and improving GPU processing efficiency. Another possible factor is that a larger batch size increases the degree of parallelism in processing, which improves processing efficiency on GPUs with many processor cores.

[0062] As shown in FIG. 1, the scheduler 101 has the functions of a priority calculation unit 102, a computation resource scheduling unit 103, an acceleration rate prediction unit 104, and a job history storage unit 105.

[0063] The job history storage unit 105 stores job execution history information 107. The job execution history information 107 is information representing the execution history when a job for a deep learning model is executed by the user program 201. The job execution history information 107 includes, for example, a batch size indicating the amount of data processed by the deep learning model at one time, and the execution time required for the computational resources to process one epoch. The job execution history information 107 is an example of a program execution history that associates the type of deep learning model, the batch size, and the acceleration rate.

[0064] FIG. 4 is a diagram illustrating an example of the job execution history information 107 in the scheduling system 1 according to an embodiment.

[0065] In the job execution history information 107 illustrated in FIG. 4, multiple types of batch sizes are shown for each model (deep learning model), and at least some of these batch sizes are associated with an acceleration rate, a GPU execution time, and a CPU execution time. The GPU execution time is the time it takes for the GPU 10b to process one epoch of machine learning, and the CPU execution time is the time it takes for the CPU 10a to process one epoch of machine learning. The CPU execution time may be referred to as the CPU epoch execution time. The GPU execution time may also be referred to as the GPU epoch execution time.

[0066] The GPU execution time and the CPU execution time may be measured, for example, by each user program 201 or each deep learning framework 202. The units of the GPU execution time and the CPU execution time may be, for example, nanoseconds or microseconds.

[0067] For example, in the example shown in Figure 4, for model A, an acceleration rate of 1.5, a GPU execution time of 0.2, and a CPU execution time of 0.3 are stored for a batch size of 16. This shows the history of when the user program 201 executed a job with a batch size of 16 for the deep learning model identified by "model A" using both the CPU 10a and the GPU 10b. This execution history shows that the time required for processing one epoch was 0.3 for the CPU 10a and 0.2 for the GPU 10b.

[0068] 4 also shows an acceleration rate calculated based on the CPU execution time and the GPU execution time. This acceleration rate is a value obtained by dividing the CPU execution time by the GPU execution time. Although the acceleration rate is stored in the job execution history information 107 shown in FIG. 4, the present invention is not limited to this. Only the CPU execution time and the GPU execution time may be stored in the job execution history information 107, and the acceleration rate may be calculated as needed by, for example, the acceleration rate prediction unit 104 (described later).

[0069] As described above, the job execution history information 107 stores information about the history of the user program 201 executing a job for training a deep learning model. The deep learning model recorded in the job execution history information 107 may be referred to as a past deep learning model.

[0070] The job execution history may be stored in the job execution history information 107 by the corresponding deep learning framework 202 each time the user program 201 executes a job.

[0071] When the deep learning framework 202 stores a job execution history in the job execution history information 107, if the job execution history information 107 already stores an execution history of the same batch size for the same deep learning model, it is desirable for the deep learning framework 202 to overwrite and update the job execution history information 107 with the new job execution history.

[0072] The acceleration rate prediction unit 104 predicts the acceleration rate of the user program 201 based on deep learning information (deep learning model object and input tensor size (batch size)) about the user program 201 that executes the job and job execution history information 107 stored in the job history storage unit 105. The acceleration rate prediction unit 104 determines a predicted value of the acceleration rate of the user program 201 for each job.

[0073] The acceleration rate prediction unit 104 predicts the acceleration rate associated with the execution of the deep learning model based on deep learning information (information about the job to be processed) and by referring to job execution history information 107.

[0074] The acceleration rate prediction unit 104 first checks whether the job execution history information 107 contains a deep learning model that is identical to the deep learning model being trained for the user program 201 to be scheduled, by referring to the job execution history information 107 based on the deep learning model object for the user program 201 to be scheduled.

[0075] Therefore, the acceleration rate prediction unit 104 determines whether the deep learning model to be trained for the user program 201 to be scheduled is the same as the deep learning model stored in the job execution history information 107.

[0076] The acceleration rate prediction unit 104 may determine whether the deep learning models are identical by comparing multiple types of features of the deep learning models, such as the type, number, and size of each layer that constitutes the deep learning model, for example.

[0077] The acceleration rate prediction unit 104 compares the features of the deep learning model of the user program 201 to be scheduled with the features of the past deep learning model, and, for example, if the features are a perfect match, may consider the features of the deep learning model of the user program 201 to be identical to the past deep learning model.

[0078] Note that the determination of the identity of deep learning models is not limited to cases where multiple types of feature quantities are completely identical. For example, the acceleration rate prediction unit 104 may consider the deep learning model of the user program 201 to be scheduled and the past deep learning model to be identical even if some of the multiple types of feature quantities do not match.

[0079] For example, the acceleration rate prediction unit 104 may determine that the deep learning models are the same if specific features among multiple types of features match, or may determine that the deep learning models are the same if a predetermined number of features among multiple types of features match, and this can be implemented with appropriate modifications.

[0080] In other words, the acceleration rate prediction unit 104 may determine that the deep learning models are identical if the features of the deep learning model of the scheduled user program 201 and the features of a past deep learning model satisfy a predetermined similarity condition.

[0081] If it is determined that the features of the deep learning model of the user program 201 to be scheduled are identical to those of a past deep learning model, the acceleration rate prediction unit 104 compares the batch size for which the acceleration rate is stored in the job execution history information 107 for this deep learning model with the batch size of the user program 201 to be scheduled.

[0082] The acceleration rate prediction unit 104 checks whether an acceleration rate is stored for the same batch size as the batch size of the user program 201 to be scheduled for the deep learning model that is the same as the deep learning model of the user program 201 to be scheduled in the job execution history information 107. Hereinafter, the batch size for which an acceleration rate is recorded in the job execution history information 107 may be referred to as a batch size with execution history.

[0083] In the example shown in Figure 4, batch sizes 16 and 256 for model A are batch sizes with execution history, and batch size 32 for model B is a batch size with execution history.

[0084] For the deep learning model of this job execution history information 107, if an acceleration rate is stored for the same batch size as the batch size of the user program 201 to be scheduled, the acceleration rate prediction unit 104 determines the acceleration rate of this batch size with execution history as the predicted value of the acceleration rate of the user program 201 to be scheduled. The predicted value of the acceleration rate determined in this way may be referred to as the predicted acceleration rate.

[0085] For example, if the deep learning model of the user program 201 to be scheduled is the same as model A of the job execution history information 107 illustrated in Figure 4, and the batch size of the user program 201 to be scheduled is 16, the acceleration rate prediction unit 104 determines the predicted acceleration rate of the batch size of 16 of the user program 201 to be 1.5.

[0086] In addition, if an acceleration rate is not stored for the same batch size as the batch size of the user program 201 to be scheduled for a deep learning model in the job execution history information 107, the acceleration rate prediction unit 104 determines the range (predicted acceleration rate range) within which the acceleration rate value of the user program 201 to be scheduled can take place based on the acceleration rate for the batch size with execution history for the same deep learning model in the job execution history information 107.

[0087] The acceleration rate prediction unit 104 determines the predicted acceleration rate range of the acceleration rate of the user program 201 to be scheduled based on the batch size with execution history, depending on the relative size between the batch size with execution history for the same deep learning model in the job execution history information 107 and the batch size of the user program 201 to be scheduled.

[0088] Specifically, when the deep learning model of the user program 201 to be scheduled is the same as the model of the job execution history information 107 illustrated in Figure 4, if the batch size of the user program 201 to be scheduled is smaller than the batch size with execution history, the acceleration rate prediction unit 104 determines that the predicted acceleration rate range of the batch size of the user program 201 to be smaller than the acceleration rate of the batch size with execution history.

[0089] For example, if the deep learning model of the user program 201 to be scheduled is the same as model A of the job execution history information 107 illustrated in Figure 4, and the batch size of the user program 201 to be scheduled is smaller than 16 (e.g., 8), the acceleration rate prediction unit 104 determines the predicted acceleration rate range for the batch size (e.g., 8) of the user program 201 to be less than 1.5.

[0090] In addition, when the batch size of the user program 201 to be scheduled is larger than the batch size with execution history, the acceleration rate prediction unit 104 determines that the predicted acceleration rate range of the batch size of the user program 201 to be scheduled is larger than the acceleration rate of the batch size with execution history.

[0091] For example, if the deep learning model of the user program 201 to be scheduled is the same as model A of the job execution history information 107 illustrated in Figure 4, and the batch size of the user program 201 to be scheduled is greater than 256 (e.g., 512), the acceleration rate prediction unit 104 determines the predicted acceleration rate range of the batch size (e.g., 512) of the user program 201 to be greater than 5.6.

[0092] Furthermore, when the batch size of the user program 201 to be scheduled is a value between two batch sizes with execution history, the acceleration rate prediction unit 104 determines the predicted acceleration rate range of the batch size of the user program 201 to be scheduled as a value between the acceleration rates of the two batch sizes with execution history.

[0093] For example, if the deep learning model of the user program 201 to be scheduled is the same as model A of the job execution history information 107 illustrated in Figure 4, and the batch size of the user program 201 to be scheduled is greater than 16 and less than 256 (e.g., 64), the acceleration rate prediction unit 104 determines the predicted acceleration rate range of the batch size (e.g., 64) of the user program 201 to be greater than 1.5 and less than 5.6.

[0094] In this way, the acceleration rate prediction unit 104 predicts the value of the acceleration rate of the batch size (e.g., 64) of the user program 201 to be scheduled by reflecting the magnitude relationship between the batch size with execution history for the same deep learning model in the job execution history information 107 and the batch size of the user program 201 to be scheduled in the magnitude relationship between the acceleration rate of the batch size with execution history and the acceleration rate of the user program 201 to be scheduled. This is done based on the above-mentioned characteristic that the acceleration rate monotonically increases with respect to the batch size.

[0095] Furthermore, the acceleration rate prediction unit 104 checks the consistency of the predicted acceleration rate and predicted acceleration rate range of the batch size of the determined (predicted) user program 201 to be scheduled for each job.

[0096] FIG. 5 is a diagram for explaining a method for verifying the predicted acceleration rate range determined by the acceleration rate prediction unit 104 of the scheduling system 1 according to an embodiment.

[0097] The acceleration rate prediction unit 104 checks whether the predicted acceleration rate range determined for one job (x) and the acceleration rate or predicted acceleration rate range determined for all other jobs (y) have a consistent relationship as shown below.

[0098] Here, let the predicted acceleration rate range of job x be set X, and the predicted acceleration rate (or predicted acceleration rate range) of job y be set Y. Also, let X ∩ Y = φ. φ represents the empty set.

[0099] In FIG. 5, reference sign A shows an example where the predicted acceleration rate range determined for job (x) and the predicted acceleration rate or predicted acceleration rate range determined for all other jobs (y) are consistent.

[0100] Also, in this reference sign A, (a) shows the relationship between the predicted acceleration rate range determined for job (x) and the predicted acceleration rate determined for all other jobs (y), and (b) shows the relationship between the predicted acceleration rate range determined for job (x) and the predicted acceleration rate range determined for all other jobs (y).

[0101] In (a) and (b) of reference sign A, the magnitude relationship of x < y is determined between the predicted acceleration rate range determined for job (x) and the predicted acceleration rate or predicted acceleration rate range determined for job (y).

[0102] On the other hand, reference sign B shows an example where the predicted acceleration rate range determined for job (x) and the predicted acceleration rate or predicted acceleration rate range determined for all other jobs (y) are not consistent.

[0103] In this reference sign B, (a) and (b) respectively show the relationship between the predicted acceleration rate range determined for job (x) and the predicted acceleration rate range determined for all other jobs (y), and (c) shows the relationship between the predicted acceleration rate range determined for job (x) and the predicted acceleration rate determined for all other jobs (y).

[0104] In any of (a) to (c) of symbol B, there are portions where x < y and portions where x > y. That is, in (a) to (c) of symbol B, the magnitude relationship between the predicted acceleration rate range determined for job (x) and the predicted acceleration rate or predicted acceleration rate range determined for job (y) is indeterminate.

[0105] When the acceleration rate prediction unit 104 determines that there is consistency as a result of the consistency check performed as described above, the acceleration rate prediction unit 104 transmits the predicted acceleration rate and predicted value acceleration rate range of the batch size of the scheduling target user program 201 determined (predicted) to the calculation resource scheduling unit 103 described later. In the calculation resource scheduling unit 103, the predicted acceleration rate and predicted acceleration rate range determined by the acceleration rate prediction unit 104 are used to determine the priority of the job.

[0106] On the other hand, when the acceleration rate prediction unit 104 determines that there is no consistency as a result of the consistency check performed as described above, the acceleration rate prediction unit 104 discards the predicted acceleration rate and predicted value acceleration rate range of the batch size of the scheduling target user program 201 determined (predicted). For jobs for which it is determined that there is no consistency in the predicted acceleration rate and predicted value acceleration rate range, the priority is calculated by the priority calculation unit 102 described later.

[0107] The priority calculation unit 102 sets a priority for determining the priority for assigning to the GPU 10b for each job.

[0108] The priority calculation unit 102 may set priorities for jobs using various methods. For example, the priority calculation unit 102 may set priorities according to the dummy job execution status of the job information of the job.

[0109] Before the execution of the user program 201, the priority calculation unit 102 executes a dummy job to measure the performance of each calculation resource. The dummy job may be a part of the job executed by the user program 201, and for example, may be the processing of one epoch of machine learning.

[0110] If the acceleration rate prediction unit 104 checks the consistency and determines that the predicted acceleration rate is inconsistent, the priority calculation unit 102 calculates the priority of the job that is determined to be inconsistent.

[0111] The priority calculation unit 102 executes the dummy job on each of the CPU 10a and the GPU 10b, and acquires the execution times (CPU execution time and GPU execution time) of the dummy jobs. The CPU execution time and GPU execution time may be acquired from the deep learning framework 202, for example.

[0112] The priority calculation unit 102 calculates an acceleration rate based on the CPU execution time and the GPU execution time by dividing the CPU execution time by the GPU execution time.

[0113] The priority calculation unit 102 sets the job priority based on the calculated acceleration rate. For example, the priority calculation unit 102 may set the calculated acceleration rate as the job priority value. The priority calculation unit 102 also notifies the computing resource scheduling unit 103 of the set priority.

[0114] Furthermore, when the dummy job execution status of a job indicates that the CPU dummy job has not been executed, the priority calculation unit 102 may set a value indicating that the GPU 10b is not allocated to the job priority of the job. The value indicating that the GPU 10b is not allocated may be, for example, "-1."

[0115] Furthermore, if the dummy job execution status indicates that the GPU dummy job has not been executed, the priority calculation unit 102 may set the job priority of the job to a value representing the highest priority. The value representing the highest priority may be, for example, “INT_MAX.”

[0116] Furthermore, when the dummy job execution status indicates that both the CPU dummy job and the GPU dummy job have been executed, the priority calculation unit 102 may calculate an acceleration rate based on the CPU epoch execution time and the GPU epoch execution time. The priority calculation unit 102 calculates the acceleration rate by dividing the CPU execution time by the GPU execution time.

[0117] The priority calculation unit 102 notifies the computation resource scheduling unit 103 of the set priority.

[0118] The computational resource scheduling unit 103 schedules jobs based on the priority of the jobs, and allocates computational resources (GPU 10b) according to the priority of the jobs.

[0119] Specifically, the computing resource scheduling unit 103 determines the allocation order to computing resources according to priority. For example, the job with the highest priority is assigned the allocation order to computing resources as 1. Furthermore, for jobs with the same acceleration rate, the computing resource scheduling unit 103 prioritizes the job that was submitted earlier.

[0120] The computational resource scheduling unit 103 determines the order in which the user programs 201 are allocated to the GPU 10b (computational resource) based on the acceleration rate prediction results (predicted acceleration rate, predicted acceleration rate range).

[0121] The computational resource scheduling unit 103 arranges the jobs in a job queue 108 (see FIG. 6) according to the determined allocation order. In the job queue 108, the job stored at the top is preferentially allocated to the GPU 10b. Since the present scheduling system 1 is equipped with two GPUs 10b, the top two jobs in the job queue 108 are allocated to the GPU 10b.

[0122] FIG. 6 is a diagram for explaining the processing of the computation resource scheduling unit 103 of the scheduling system 1 according to an embodiment.

[0123] In the example shown in FIG. 6, two jobs J1 and J2 are stored in the job queue 108, and a case will be described in which jobs J3 and J4 are further allocated in this state.

[0124] The computing resource scheduling unit 103 refers to the acceleration rates (predicted acceleration rate, predicted acceleration rate range) of jobs J3 and J4 and decides to store job J3, which has a larger acceleration rate (predicted acceleration rate, predicted acceleration rate range), before job J4 in the job queue 108.

[0125] Furthermore, since the predicted acceleration rate range of job J3 (<5.6: less than 5.6) is smaller than the acceleration rate (5.6) of job J2, which is stored earlier in job queue 108, the computing resource scheduling unit 103 stores job J3 after job J2 in job queue 108.

[0126] Since the predicted acceleration rate range of job J4 (<1.5: less than 1.5) is smaller than the predicted acceleration rate range acceleration rate of job J3 (<5.6: less than 5.6), the computing resource scheduling unit 103 stores job J4 after job J3 in the job queue 108.

[0127] In addition, the computational resource scheduling unit 103 returns the GPU number as resource information to the deep learning framework 202 of the job to which the GPU 10b is allocated.

[0128] Furthermore, the computational resource scheduling unit 103 returns the GPU number "-1" as resource information to the deep learning framework 202 for jobs to which the GPU 10b has not been allocated or for which the priority is -1. The resource information is information that identifies the computational resources (GPU 10b, CPU 10a) used to execute the job, and may be, for example, a GPU number or a CPU number.

[0129] The deep learning framework 202 updates the dummy job execution status (CPU epoch execution time or GPU epoch execution time) and transmits the resource information and job information to the job control unit 106. The deep learning framework 202 also saves the updated dummy job execution status (CPU epoch execution time or GPU epoch execution time) in the job execution history information 107.

[0130] In addition, the deep learning framework 202 transmits resource information and job information about the executed job to the job control unit 106.

[0131] The job control unit 106 causes the user program 201 to execute a job according to the results of scheduling by the computing resource scheduling unit 103 .

[0132] The job control unit 106 starts a job. When starting this job, the job control unit 106 initializes job information for the job to be started. When initializing the job, the job control unit 106 generates a job ID and also initializes a dummy job execution status.

[0133] The job control unit 106 starts the user program 201 and transmits to the user program 201 job information about the job to be executed.

[0134] The job control unit 106 processes one or more jobs registered in the job queue 108 in order from the top. The job control unit 106 causes the machine learning process to be executed until the epoch ends, and waits while the machine learning process is being executed.

[0135] In addition, if the resource information of the job to be executed by the user program 201 differs from the resource information of the computational resource currently executing the user program 201, the job control unit 106 stops the user program 201 and restarts the job on the new computational resource.

[0136] (C) Operation The training process of the deep learning model in the scheduling system 1 according to one embodiment configured as described above will be described with reference to the flowchart (steps S1 to S11) shown in Fig. 7. The process shown in Fig. 7 is repeatedly executed until all epochs in the training of the deep learning model are completed.

[0137] In step S1, the job control unit 106 starts a job to be used for training a deep learning model. The job control unit 106 initializes job information for the job to be started. The job control unit 106 also starts a user program 201 and transmits job information about the job to be executed to the user program 201.

[0138] In step S2, the user program 201 calls the API of the deep learning framework 202. The deep learning framework 202 executes a function of transmitting job information and deep learning information to the acceleration rate prediction unit 104 via a hook built into the API.

[0139] In step S3, the acceleration rate prediction unit 104 predicts the acceleration rate (predicted acceleration rate, predicted acceleration rate range) for the job. Details of the processing in step S3 will be described later with reference to FIG.

[0140] In step S4, the acceleration rate prediction unit 104 refers to the job execution history information 107 based on the deep learning model object for the user program 201 to be scheduled, and confirms whether the deep learning model that is the same as the deep learning model to be trained for the user program 201 to be scheduled is stored in the job execution history information 107.

[0141] If the confirmation result shows that the same deep learning model as the deep learning model to be trained for the user program 201 to be scheduled is not stored in the job execution history information 107, i.e., if the prediction range cannot be identified (see the No route in step S4), proceed to step S6.

[0142] In step S6, the priority calculation unit 102 sets a priority for the job to determine the order of priority for allocation to the GPU 10b.

[0143] The priority calculation unit 102 executes the dummy job on each of the CPU 10a and the GPU 10b, and acquires the respective execution times (CPU execution time and GPU execution time). The priority calculation unit 102 calculates an acceleration rate based on the CPU execution time and the GPU execution time. The priority calculation unit 102 sets, for example, the calculated acceleration rate as the job priority value. The priority calculation unit 102 notifies the computation resource scheduling unit 103 of the set priority. Then, the process proceeds to step S7.

[0144] Furthermore, if the result of the confirmation in step S4 is that the same deep learning model as the deep learning model to be trained for the user program 201 to be scheduled is stored in the job execution history information 107, that is, if the prediction range can be identified (see the Yes route in step S4), proceed to step S5.

[0145] In step S5, the acceleration rate prediction unit 104 checks the consistency for each job with respect to the predicted acceleration rate and predicted acceleration rate range of the batch size determined (predicted) for the user program 201 to be scheduled. The acceleration rate prediction unit 104 checks whether the predicted acceleration rate range determined for one job (x) is consistent with the acceleration rates or predicted acceleration rate ranges determined for all other jobs (y).

[0146] If the result of the check is that the predicted acceleration rate range determined for one job (x) and the acceleration rates or predicted acceleration rate ranges determined for all other jobs (y) are inconsistent (see the No route in step S5), the process proceeds to step S6, where the priority calculation unit 102 sets a priority for the job.

[0147] On the other hand, if the predicted acceleration rate range determined for one job (x) and the acceleration rates or predicted acceleration rate ranges determined for all other jobs (y) are consistent with each other (see the Yes route in step S5), proceed to step S7.

[0148] In step S7, the computational resource scheduling unit 103 schedules the jobs based on the priority of the jobs, and allocates computational resources (GPU 10b, CPU 10a) based on the priority of the jobs.

[0149] In step S8, the job control unit 106 causes the user program 201 to execute a job in accordance with the result of scheduling by the computational resource scheduling unit 103. The job control unit 106 causes the machine learning process to be executed until the epoch ends, and waits until the machine learning process ends.

[0150] When the machine learning process is completed, in step S9, the job control unit 106 checks whether the resource information of the job to be executed by the user program 201 is different from the resource information of the computational resource currently executing the user program 201.

[0151] As a result of the check, if the resource information of the job to be executed by the user program 201 is different from the resource information of the computing resource currently executing the user program 201 (see the Yes route in step S9), the process proceeds to step S10.

[0152] In step S10, the job control unit 106 moves the job resources. Specifically, the job control unit 106 stops the user program 201 and restarts the job with the new computing resources. Then, the process returns to step S2.

[0153] If the resource information of the job to be executed by the user program 201 is the same as the resource information of the computing resource currently executing the user program 201 (see the No route in step S9), the process proceeds to step S11.

[0154] In step S11, the job control unit 106 checks whether the machine learning process of the deep learning model has been completed for all epochs. If all epochs have not been completed (see the No route in step S11), the process returns to step S2. If all epochs have been completed (see the Yes route in step S11), the process ends.

[0155] Next, the details of the process of step S3 in the flowchart shown in FIG. 7 will be described with reference to the flowchart (steps S21 to S25) shown in FIG.

[0156] In step S21, the acceleration rate prediction unit 104 reads the job execution history information 107 from the job history storage unit 105, thereby acquiring the job execution history information 107.

[0157] In step S22, the acceleration rate prediction unit 104 refers to the job execution history information 107 based on the deep learning model object for the user program 201 to be scheduled, and confirms whether the same deep learning model as the deep learning model to be trained for the user program 201 to be scheduled is stored in the job execution history information 107.

[0158] As a result of the check, if the same deep learning model is stored in the job execution history information 107 (see the Yes route in step S22), proceed to step S23.

[0159] In step S23, the acceleration rate prediction unit 104 checks whether an acceleration rate is stored for a batch size that is the same as the batch size of the user program 201 to be scheduled in the job execution history information 107 for a deep learning model that is the same as the deep learning model of the user program 201 to be scheduled.

[0160] As a result of the confirmation, if the job execution history information 107 stores an acceleration rate for a batch size that is the same as the batch size of the user program 201 to be scheduled for the deep learning model that is the same as the deep learning model of the user program 201 to be scheduled (see the Yes route in step S23), proceed to step S24.

[0161] In step S24, the acceleration rate prediction unit 104 determines the acceleration rate of the batch size with execution history as the predicted value (predicted acceleration rate) of the acceleration rate of the scheduling target user program 201. Thereafter, the process ends.

[0162] Also, if the result of the check in step S23 shows that the acceleration rate is not stored in the job execution history information 107 for the same deep learning model as the deep learning model of the user program 201 to be scheduled for the same batch size as the batch size of the user program 201 to be scheduled (see the No route in step S23), proceed to step S25.

[0163] In step S25, the acceleration rate prediction unit 104 determines the range (predicted acceleration rate range) of the acceleration rate value of the user program 201 to be scheduled, based on the acceleration rate of the batch size with execution history for the same deep learning model in the job execution history information 107. Then, the process ends.

[0164] Also, if the confirmation in step S22 shows that the same deep learning model as the deep learning model to be trained for the user program 201 to be scheduled is not stored in the job execution history information 107 (see the No route in step S22), the processing is terminated.

[0165] (D) Effects Thus, according to the scheduling system 1 as an example of an embodiment, the acceleration rate prediction unit 104 predicts the acceleration rate (predicted acceleration rate, predicted acceleration rate range) of the user program 201 based on deep learning information (deep learning model object and input tensor size (batch size)) about the user program 201 that executes the job and job execution history information 107 stored in the job history storage unit 105. Then, the computing resource scheduling unit 103 uses the predicted acceleration rate and predicted acceleration rate range determined by the acceleration rate prediction unit 104 to determine the priority of the job.

[0166] This eliminates the need for the priority calculation unit 102 to calculate the priority of the job, and eliminates the need to issue a dummy job to the user program 201, thereby reducing the number of unnecessary dummy executions. This reduces the execution time cost and the cost of switching computational resources (CPU 10a, GPU 10B). As a result, the system utilization rate can be improved.

[0167] Furthermore, the acceleration rate prediction unit 104 checks the consistency for each job with respect to the predicted acceleration rate and predicted acceleration rate range of the batch size of the determined (predicted) user program 201 to be scheduled. If the acceleration rate prediction unit 104 determines that there is no consistency, the priority calculation unit 102 calculates the priority for the job determined to be inconsistent.

[0168] This prevents the computational resource scheduling unit 103 from allocating computational resources (GPU 10b, CPU 10a) to jobs using inconsistent priorities. That is, the computational resource scheduling unit 103 schedules jobs using predicted acceleration rates or predicted acceleration rate ranges that are consistent with the acceleration rates or predicted acceleration rate ranges determined for all other jobs. This improves the reliability of job scheduling.

[0169] (E) Other The configurations and processes of this embodiment can be selected as needed, or can be combined as appropriate.

[0170] The disclosed technology is not limited to the above-described embodiment, and can be implemented in various modifications without departing from the spirit of the present embodiment.

[0171] In the above-described embodiment, the computer 10 is provided with two CPUs 10a, but this is not limiting. The number of CPUs 10a may be one, or three or more.

[0172] Furthermore, in the above-described embodiment, the computer 10 is equipped with a CPU 10a as a computational resource, but this is not limited to this. Other processors may be provided instead of the CPU 10a. Examples of processors include integrated circuits (ICs) such as a CPU, MPU, APU, DSP, ASIC, and FPGA. Note that a combination of two or more of these integrated circuits may also be used as a processor. MPU is an abbreviation for Micro Processing Unit. APU is an abbreviation for Accelerated Processing Unit. DSP is an abbreviation for Digital Signal Processor, ASIC is an abbreviation for Application Specific IC, and FPGA is an abbreviation for Field-Programmable Gate Array.

[0173] Furthermore, in the above-described embodiment, the computer 10 is equipped with two GPUs 10b, but this is not limiting. The number of GPUs 10b may be one, or three or more.

[0174] In the above-described embodiment, the computer 10 is equipped with a GPU 10b as a computational resource, but this is not limiting. Instead of the GPU 10b, other graphics processing devices may be provided. Examples of graphics processing devices include various arithmetic processing devices, APUs, DSPs, ASICs, FPGAs, and other integrated circuits (ICs).

[0175] Furthermore, the above disclosure will enable those skilled in the art to implement and manufacture the present embodiment.

[0176] (F) Notes The following additional notes are provided regarding the above-described embodiments.

[0177] (Appendix 1) Based on information about a job to be processed for a deep learning model, a program execution history that associates the type of deep learning model, the batch size, and the acceleration rate is referenced to predict an acceleration rate associated with the execution of the deep learning model; Based on the predicted acceleration rate, the order of programs to be allocated to the computing resources is determined. A scheduling program that causes a computer to execute a process.

[0178] (Appendix 2) The predicted acceleration rate is checked for consistency for each job. If it is determined that the predicted acceleration rate is inconsistent as a result of checking the consistency, a priority is calculated for the job that is determined to be inconsistent. 2. The scheduling program according to claim 1, wherein the scheduling program causes the computer to execute a process.

[0179] (Appendix 3) Based on information about a job to be processed for a deep learning model, a program execution history that associates the type of deep learning model, the batch size, and the acceleration rate is referenced to predict an acceleration rate associated with the execution of the deep learning model; Based on the predicted acceleration rate, the order of programs to be allocated to the computing resources is determined. An information processing device comprising a control unit that executes processing.

[0180] (Appendix 4) The control unit The predicted acceleration rate is checked for consistency for each job. If it is determined that the predicted acceleration rate is inconsistent as a result of checking the consistency, a priority is calculated for the job that is determined to be inconsistent. 4. The information processing device according to claim 3, wherein the information processing device executes processing.

[0181] (Appendix 5) Based on information about a job to be processed for a deep learning model, a program execution history that associates the type of deep learning model, the batch size, and the acceleration rate is referenced to predict an acceleration rate associated with the execution of the deep learning model; Based on the predicted acceleration rate, the order of programs to be allocated to the computing resources is determined. A scheduling method characterized in that the processing is executed by a computer.

[0182] (Appendix 6) The predicted acceleration rate is checked for consistency for each job. If it is determined that the predicted acceleration rate is inconsistent as a result of checking the consistency, a priority is calculated for the job that is determined to be inconsistent. 6. The scheduling method according to claim 5, wherein the processing is executed by the computer. [Explanation of symbols]

[0183] 1. Scheduling System 10. Computers 10a-1, 10a-2, 10a CPU 10b-, 10b-2, 10b GPU 10c memory 10d storage section 10e IF section 10f IO section 10g reading unit 10-hour program 10i recording media 10j bus 101 Scheduler 102 Priority calculation section 103 Computational Resource Scheduling Unit 104 Acceleration rate prediction section 105 Job history memory unit 106 Job control section 107 Job execution history information 108 Job Queue 201 User Program 202 Deep Learning Framework

Claims

1. Based on information about a job to be processed for a deep learning model, a program execution history that associates the type of deep learning model, the batch size, and the acceleration rate is referenced to predict an acceleration rate associated with the execution of the deep learning model; Based on the predicted acceleration rate, the order of programs to be allocated to the computing resources is determined. A scheduling program that causes a computer to execute a process.

2. The predicted acceleration rate is checked for consistency for each job. If it is determined that the predicted acceleration rate is inconsistent as a result of checking the consistency, a priority is calculated for the job that is determined to be inconsistent.

2. The scheduling program according to claim 1, wherein the scheduling program causes the computer to execute processing.

3. Based on information about a job to be processed for a deep learning model, a program execution history that associates the type of deep learning model, the batch size, and the acceleration rate is referenced to predict an acceleration rate associated with the execution of the deep learning model; Based on the predicted acceleration rate, the order of programs to be allocated to the computing resources is determined. An information processing device comprising a control unit that executes processing.

4. Based on information about a job to be processed for a deep learning model, a program execution history that associates the type of deep learning model, the batch size, and the acceleration rate is referenced to predict an acceleration rate associated with the execution of the deep learning model; Based on the predicted acceleration rate, the order of programs to be allocated to the computing resources is determined. A scheduling method characterized in that the processing is executed by a computer.

Images