Information processing program, information processing device, and information processing method

Abstract

To reduce the time it takes to execute the process. [Solution] The first memory and the second memory are managed as a single integrated memory. When the process to be processed is to be executed on the second computing resource 10b, a first predicted completion time is predicted when the process to be processed is executed using a first method in which the process is allocated by exceeding the memory capacity of the second memory, and a second predicted completion time is predicted when the process to be processed is executed using a second method in which the process is allocated with a delay so as not to exceed the memory capacity of the second memory. The process to be processed is then executed using the method corresponding to the earlier of the two predicted completion times.

Description

[Technical Field]

[0001] The present invention relates to an information processing program, an information processing device, and an information processing method. [Background technology]

[0002] It is known that using a GPU (Graphics Processing Unit) instead of a CPU (Central Processing Unit) to run deep learning applications (hereinafter referred to as deep learning applications) improves processing performance (for example, Patent Document 1).

[0003] Furthermore, the rapid development of AI in recent years has led to soaring prices and supply shortages of GPUs. Consequently, there is a demand to run multiple AI (Artificial Intelligence) processes on a small number of GPUs (for example, one), that is, to run multiple deep learning applications.

[0004] However, when AI processing is executed by multiple users or multiple processes, Out of Memory (OOM) errors occur with a high probability.

[0005] One known technology for suppressing OOM (Out of Memory) is Nvidia's CUDA Unified Memory (CUDA is a registered trademark).

[0006] CUDA Unified Memory is a technology that allows the memory used by the CPU (Central Processing Unit) (system memory) and the memory on the GPU (hereinafter sometimes referred to as GPU memory) to be accessed from a single memory space. This allows the physically separate CPU memory and GPU memory to be treated as if they were a single integrated memory, thereby suppressing the occurrence of Out-of-Memory (OOM). [Prior art documents] [Patent Documents]

[0007] [Patent Document 1] Japanese Patent Publication No. 2022-124765 [Patent Document 2] Japanese Patent Publication No. 2004-252983 [Patent Document 3] U.S. Patent Application Publication No. 2024 / 0211307 [Patent Document 4] U.S. Patent Application Publication No. 2016 / 0103715 [Overview of the project] [Problems that the invention aims to solve]

[0008] However, in CUDA Unified Memory, if more memory is used than the GPU's memory, a large number of page migrations may occur due to page faults (where pages do not exist in the GPU's memory), potentially leading to thrashing and a significant decrease in process execution speed.

[0009] In one aspect, the present invention aims to reduce the time required to execute a process. [Means for solving the problem]

[0010] This information processing program, in a computer comprising a first memory used by a first computing resource and a second memory used by a second computing resource, manages the first memory and the second memory as a single integrated memory, and when causing the second computing resource to execute a process to be processed: predicts a first predicted end time when the process to be processed is executed by a first method that allocates the process in an amount exceeding the memory capacity of the second memory, and predicts a second predicted end time when the process to be processed is executed by a second method that allocates the process by delaying execution so as not to exceed the memory capacity of the second memory; and based on the first predicted end time and the second predicted end time, causes the computer to execute processing for executing the process to be processed using one of the first method and the second method that corresponds to higher processing performance of the process to be processed. [Effects of the Invention]

[0011] According to one embodiment, the time taken to execute a process can be shortened. [Brief Description of Drawings]

[0012] [Figure 1] FIG. 1 is a diagram schematically illustrating the configuration of an information processing system according to one embodiment. [Figure 2] FIG. 2 is a block diagram showing a hardware (HW) configuration example of a computer that implements functions of the information processing system according to one embodiment. [Figure 3] FIG. 3 is a diagram schematically showing processes processed in the information processing system according to one embodiment. [Figure 4] FIG. 4 is a diagram for explaining over-memory execution and delayed execution in the information processing system according to one embodiment. [Figure 5] FIG. 5 is a diagram for explaining a process history in the information processing system according to one embodiment. [Figure 6] FIG. 6 is a diagram showing a scheduled end time of a target process during delayed execution in the information processing system according to one embodiment. [Figure 7] This figure shows the estimated termination time of the target process in memory overload execution in an information processing system according to one embodiment. [Figure 8] This is a flowchart illustrating the processing of an information processing system according to one embodiment. [Figure 9] This is a flowchart illustrating a scheduling method in an information processing system according to one embodiment. [Figure 10] This is a flowchart illustrating a scheduling method in an information processing system according to one embodiment. [Modes for carrying out the invention]

[0013] Embodiments relating to this information processing program, information processing apparatus, and information processing method will be described below with reference to the drawings. However, the embodiments shown below are merely illustrative, and there is no intention to exclude various modifications or applications of technologies not explicitly shown in the embodiments. In other words, these embodiments can be implemented with various modifications without departing from their spirit. Furthermore, each figure is not intended to represent only the components shown in the figure, but may include other functions, etc.

[0014] (A) Configuration Figure 1 is a schematic diagram showing the configuration of an information processing system 1 according to one embodiment, and Figure 2 is a block diagram showing an example of the hardware (HW) configuration of a computer 10 that realizes the functions of the information processing system 1 according to one embodiment.

[0015] (A-1) Example hardware configuration If multiple computers are used as hardware resources to implement the functions of information processing system 1, each computer may have the hardware configuration illustrated in Figure 2.

[0016] As shown in Figure 2, the computer 10 is an information processing device, and its hardware configuration may include, as an example, one or more (one in the example shown in Figure 2) CPUs 10a, one or more (one in the example shown in Figure 2) GPUs 10b, memory 10c, storage unit 10d, IF (Interface) unit 10e, IO (Input / Output) unit 10f, and read unit 10g.

[0017] The CPU 10a is an example of an arithmetic processing unit that performs various control and calculations, and is a control unit that executes various processes. The CPU 10a may be connected to each block in the computer 10 via a bus 10j so that they can communicate with each other. The bus 10j may include a PCIe (Peripheral Component Interconnect-Express) bus. The CPU 10a may be a multiprocessor containing multiple processors, a multicore processor having multiple processor cores, or a configuration having multiple multicore processors. The CPU 10a is an example of a first computing resource.

[0018] GPU10b may be an accelerator such as a GPGPU (General Purpose computing on Graphics Processing Unit). Furthermore, GPU10b may be used to control screen display for output devices such as monitors in the I / O unit 10f. GPU10b may be configured as an accelerator that performs machine learning processing and inference processing using machine learning models. Regarding machine learning processing and inference processing, GPU10b can be said to have higher processing performance than CPU10a. GPU10b is an example of a second computing resource. GPU10b is also equipped with VRAM (Video Random Access Memory), which is not shown in the diagram. The VRAM of GPU10b may be called GPU memory. GPU memory is an example of second memory used by GPU10b (the second computing resource).

[0019] These CPU 10a and GPU 10b are computing resources allocated to the user program 101, which will be described later. CPU 10a is an example of a first computing resource, and GPU 10b is an example of a second computing resource.

[0020] Memory 10c is an example of hardware that stores various data and program information. Examples of memory 10c include volatile memory such as DRAM (Dynamic Random Access Memory) and non-volatile memory such as PM (Persistent Memory), or both. Memory 10c is an example of first memory used by the CPU 10a (first computing resource). Memory 10c may also be called system memory or CPU memory.

[0021] The storage unit 10d is an example of hardware that stores various data and program information. Examples of storage units 10d include magnetic disk devices such as HDDs (Hard Disk Drives), semiconductor drive devices such as SSDs (Solid State Drives), and various storage devices such as non-volatile memory. Examples of non-volatile memory include flash memory, SCM (Storage Class Memory), and ROM (Read Only Memory).

[0022] The memory unit 10d may store a program 10h (user program 101, information processing program) that implements all or part of the various functions of the computer 10.

[0023] For example, the CPU 10a of the information processing system 1 can implement the functions of the user program 101, AI framework 102, memory control library 105, scheduler 107, scheduling method determination unit 108, and process history storage unit 109, which will be described later, by loading the program 10h stored in the storage unit 10d into memory 10c and executing it.

[0024] The IF unit 10e is an example of a communication interface that controls the connection and communication between this computer 10 and other computers. For example, the IF unit 10e may include an adapter compliant with LAN (Local Area Network) such as Ethernet®, or optical communication such as FC (Fibre Channel). The adapter may support wireless, wired, or both communication methods. The program 10h may be downloaded from the network to the computer 10 via the communication interface and stored in the storage unit 10d.

[0025] The I / O unit 10f may include either an input device or an output device, or both. Examples of input devices include keyboards, mice, and touch panels. Examples of output devices include monitors, projectors, and printers. The I / O unit 10f may also include a touch panel or the like that integrates the input and output devices. The output device may be connected to the GPU 10b. The I / O unit 10f may also be an input or output device of another information processing device remotely connected to this computer 10 via SSH (Secure Shell) or the like.

[0026] The reading unit 10g is an example of a reader that reads data and program information recorded on the recording medium 10i. The reading unit 10g may include a connection terminal or device to which the recording medium 10i can be connected or inserted. Examples of the reading unit 10g include an adapter compliant with USB (Universal Serial Bus), a drive device for accessing a recording disk, and a card reader for accessing flash memory such as an SD card. The recording medium 10i may store a program 10h, and the reading unit 10g may read the program 10h from the recording medium 10i and store it in the storage unit 10d.

[0027] Examples of recording media 10i include non-temporary computer-readable recording media such as magnetic / optical discs and flash memory. Examples of magnetic / optical discs include flexible discs, CDs (Compact Discs), DVDs (Digital Versatile Discs), Blu-ray discs, and HVDs (Holographic Versatile Discs). Examples of flash memory include semiconductor memory such as USB memory and SD cards.

[0028] The hardware configuration of computer 10 described above is illustrative. Therefore, the addition or deletion of hardware within computer 10 (for example, adding or deleting arbitrary blocks), division, integration in any combination, or addition or deletion of buses may be performed as appropriate.

[0029] (A-2) Example of Functional Configuration As shown in Figure 1, the information processing system 1 may, for example, include functions such as a user program 101, an AI framework 102, a memory control library 105, a scheduler 107, a scheduling method determination unit 108, and a process history storage unit 109. These functions may be implemented by the hardware of the computer 10 (see Figure 2).

[0030] User program 101 may be, for example, a program that implements the process of training (deep learning) a deep learning model (machine learning model) not shown in the diagram, and may execute jobs related to deep learning. User program 101 is, for example, a deep learning program.

[0031] Figure 3 is a schematic diagram showing the process processed in an information processing system 1 according to one embodiment.

[0032] In AI processing, similar processes are repeatedly executed, as seen in batch processing during the learning process and inference processing on inference servers.

[0033] For example, in the deep learning (training phase) process, preprocessing, machine learning (main process), and postprocessing are repeatedly executed. Similarly, in the inference phase process, preprocessing such as receiving requests, inference (main process), and postprocessing are repeatedly executed.

[0034] Of these, the pre-processing and post-processing may be performed using CPU 10a, and the main process may be performed using GPU 10b.

[0035] A series of processes including pre-processing, main processing, and post-processing may be referred to as a job. In this information processing system 1, one user program 101 may execute one process, and multiple processes may be executed by executing multiple user programs 101. Furthermore, one user program 101 may execute multiple processes.

[0036] In this information processing system 1, the scheduler 107, described later, allocates computing resources (CPU 10a, GPU 10b) to each of multiple processes and performs scheduling to execute jobs.

[0037] Furthermore, when the user program 101 has finished using the computing resources, it may send an output (release notification) to the scheduler 107 via the AI ​​framework 102 to notify it that it is releasing the computing resources. For example, when the user program 101 has finished processing using the GPU 10b, it sends a GPU release notification to the scheduler 107 via the AI ​​framework 102. Also, when the user program 101 has finished processing using the CPU 10a, it sends a CPU release notification to the scheduler 107 via the AI ​​framework 102.

[0038] The memory control library 105 is a library that controls data allocation in a single memory address space without distinction between CPU memory and GPU memory. The memory control library 105 also functions as a memory allocation control unit 106.

[0039] The memory allocation control unit 106 manages CPU memory and GPU memory as a single, undifferentiated memory. The memory allocation control unit 106 manages the storage area of ​​CPU memory and the storage area of ​​GPU memory as a single memory space. The memory allocation control unit 106 performs the function of managing CPU memory (first memory) and GPU memory (second memory) as a single integrated memory.

[0040] The memory allocation control unit 106 responds to the AI ​​framework 102 with a pointer representing a memory area in a single memory address space formed by integrating the memory address space of the CPU memory and the memory address space of the GPU memory, in response to an access request from the second library 104, which will be described later. Hereinafter, the memory formed by integrating the CPU memory and GPU memory into a single memory address space may be referred to as unified memory.

[0041] The memory allocation control unit 106 transfers data requested by the AI ​​framework 102 from CPU memory (system memory) to GPU memory if the data does not exist in GPU memory.

[0042] The memory control library 105 may be, for example, a CUDA library, and the memory allocation control unit 106 may perform a function such as cudaMallocManaged().

[0043] The AI ​​framework 102 is a framework for performing AI-related processing. The AI ​​framework 102 has a first library 103 and a second library 104. The AI ​​framework 102 may perform functions such as PyTorch or TensorFlow.

[0044] The first library 103 may output switching instructions to the scheduler 107 to switch computing resources (CPU 10a, GPU 10b) during the execution of the user program 101, for example, when transitioning from pre-processing to main processing and when transitioning from the end of main processing to post-processing. The first library 103 sends a request for GPU resources to the scheduler 107.

[0045] The first library 103 may be executed, for example, by the execution of a hook set in a specific GPU processing function in the user program 101. The first library 103 may be incorporated into the Interface layer.

[0046] The second library 104 performs processing to realize memory integration, which treats the storage area of ​​the CPU memory and the storage area of ​​the GPU memory as a single memory address space. The second library 104 replaces the memory allocation processing of the AI ​​framework 102 with the memory allocation control unit 106 of the memory control library 105. The second library 104 allocates memory via the memory allocation control unit 106 in response to the memory allocation processing of the AI ​​framework 102.

[0047] The second library 104 may be incorporated into the memory allocation layer.

[0048] The second library 104, for example, allows the memory allocation control unit 106 to perform memory allocation using the cudaMallocManaged function, thereby enabling processing by the GPU 10b without generating an OOM even if memory is exceeded.

[0049] This information processing system 1 has a function (first scheduling function) that performs delayed execution to delay the execution of processing by the GPU 10b in order to avoid a memory overload state in the single memory address space formed by the storage area of ​​the system memory used by the CPU 10a (hereinafter sometimes referred to as CPU memory) and the storage area of ​​the GPU memory owned by the GPU 10b.

[0050] Here, a memory overload state refers to a condition where, when a new process is introduced to run on GPU10b, the memory usage exceeds the amount of memory installed on the GPU.

[0051] In delayed execution, if running a new process on GPU10b would result in a memory overload, the system will wait to launch the new process until memory becomes available. In other words, the start of execution of the new process is delayed to avoid a memory overload.

[0052] Furthermore, this information processing system 1 also has a function (second scheduling function) that performs memory overload execution, which involves introducing a new process even if a memory overload condition occurs when a new process is executed on the GPU 10b.

[0053] Figure 4 is a diagram illustrating memory overload execution and delayed execution in an information processing system 1 according to one embodiment. Symbol A indicates the memory usage state due to memory overload execution, and symbol B indicates the memory usage state due to delayed execution.

[0054] Figure 4 shows an example where, at the current time (time t0), process #1 is running, and process #2 is then added to the process.

[0055] In memory overload execution, as shown by symbol A, process #2 is executed at the current time (time t0) (immediate execution) even though the memory usage U1 exceeds the amount of GPU memory installed (see symbol P0).

[0056] On the other hand, in delayed execution, as shown by symbol B, the execution of process #2 is delayed until time t1 when at least part of the processing of process #1 has finished and memory has been freed, and when process #2 is executed in addition, the memory usage U2 will not exceed the amount of GPU memory installed. In other words, process #2 is executed after waiting until time t1 when the memory overload condition does not occur (see symbol P1).

[0057] This allows GPU10b to be shared among multiple processes during delayed execution without exceeding the available GPU memory.

[0058] The scheduling method determination unit 108 determines the scheduling method used by the scheduler 107, which will be described later. Specifically, the scheduling method determination unit 108 determines, based on the processing history of the process by the GPU 10b, whether the scheduler 107 should use delayed execution or memory overload execution for scheduling the process. The processing history of the process by the GPU 10b is stored in the process history storage unit 109.

[0059] The process history storage unit 109 stores information (hereinafter referred to as process history) that indicates the execution history (execution record) of processes performed by the GPU 10b. The execution of a process by the GPU 10b can be referred to as GPU processing.

[0060] Figure 5 is a diagram illustrating the process history in an information processing system 1 according to one embodiment.

[0061] In Figure 5, arrow B indicates the execution time t0 of the GPU processing when delayed execution is performed. Since no memory overload condition occurs when delayed execution is performed, the execution time of the GPU processing when delayed execution is performed can be called the "time without memory overload".

[0062] Furthermore, in Figure 5, arrow A indicates the execution time of the GPU process when memory overload execution is performed. When memory overload execution occurs, the system is basically in a memory overload state, but there may be cases where the memory overload state is eliminated, for example, when other processes running in parallel terminate. The execution time of the GPU process in a memory overload state can be called "time with memory overload".

[0063] As shown in Figure 5, the execution time of GPU processing when memory overload occurs can be expressed as the sum of the time with memory overload t2 and the time without memory overload t1 (t1 + t2).

[0064] Furthermore, if the memory is exceeded, the scheduling method determination unit 108 calculates the delay rate α based on the following equation (1).

[0065] Delay rate α = t2 / (t0-t1) ... (1) The process history stored in the process history storage unit 109 may include the execution time of the GPU process when memory overload is performed (t1+t2), the execution time of the GPU process when delayed execution is performed (t0), the amount of GPU memory M used by the process during GPU processing, and the delay rate α in the case of memory overload, as determined as described above. Furthermore, the process history storage unit 109 stores the above process history for each process identification (PID: process ID).

[0066] The scheduling method determination unit 108 predicts the completion time of GPU processing for all currently running processes when the processes to be scheduled (hereinafter sometimes referred to as target processes) are processed using memory overload execution. The predicted completion time of GPU processing for all currently running processes may be called the predicted completion time.

[0067] The scheduling method determination unit 108 predicts a first predicted completion time when the target process is executed using a first method (memory overload execution) in which the amount of GPU memory (amount of installed GPU memory) is exceeded when the target process is executed on the GPU 10b (second computing resource).

[0068] Figure 6 shows the estimated completion time of the target process during delayed execution in an information processing system 1 according to one embodiment. In Figure 6, the vertical axis represents the amount of GPU memory used (GPU memory usage), and the horizontal axis represents the elapsed time.

[0069] The scheduling method determination unit 108, with regard to delayed execution, schedules the target process to run at the earliest possible time without causing a memory overload, based on the amount of GPU memory used and the scheduled end time of the currently running processes (processes #1 and #2 in the example shown in Figure 6), and the amount of GPU memory used by the target process.

[0070] The scheduling method determination unit 108 may determine the predicted end time of a target process by, for example, adding the execution time of the target process to the start time of the target process. Similarly, the scheduling method determination unit 108 determines the predicted end time of the GPU processing for all processes currently running based on the scheduling results.

[0071] The scheduling method determination unit 108 predicts a second predicted completion time when the target process is executed using a second method (delayed execution) that delays the process to allocate memory so as not to exceed the amount of GPU memory (amount of installed memory).

[0072] Furthermore, the scheduling method determination unit 108 determines the predicted completion time of GPU processing for all currently running processes when processing with memory overload, based on the scheduling results from the scheduler 107.

[0073] Figure 7 shows the estimated termination time of the target process in memory overload execution in an information processing system 1 according to one embodiment. In Figure 7, the vertical axis represents the amount of GPU memory used (GPU memory usage), and the horizontal axis represents the elapsed time.

[0074] In the example shown in Figure 7, the execution time of the target process tover_subscribed corresponds to the execution interval in which memory overage occurs. The scheduling method determining unit 108 calculates the time t required for the execution interval in which memory overage occurs over_subscribed using the following formula (2), where M p is the GPU memory usage amount by GPU processing of process p, α p is the delay rate of process p, and t end p is the scheduled end time of process p.

[0075] Also, in formula (2), min p (t end p -t now ) represents the time until the end time of the first terminating process, and can be said to represent the time in the state before a delay occurs. Also, in this formula (2), considering that a larger GPU memory usage amount has a greater influence on delay, it can be said that this min p (t end p -t now ) is multiplied by a coefficient that uses the GPU memory usage amount.

[0076]

Num.

[0077] Then, by adding the remaining execution time of each job to the t obtained using formula (2), the scheduling method determining unit 108 can calculate the scheduled end time of each process extended due to the occurrence of thrashing. over_subscribed

[0078] The scheduling method determining unit 108 may obtain the end time of process p using the following formula (3).

[0079] End time t p =t​over_subscribed +t end p -min p (t end p -t now )+t now ...(3) The scheduling method determination unit 108 then determines the scheduling method to be the one with better performance between memory overload execution and delayed execution, based on predetermined performance criteria.

[0080] The performance criterion could be, for example, the sum of the execution times of all GPU processes, or the time when the last process terminates. For example, if throughput is the priority in terms of performance, it is desirable to use the sum of the execution times of all GPU processes as the performance criterion. If latency is the priority in terms of performance, it is desirable to use the time when the last process terminates as the performance criterion. Other values ​​may also be used as performance criteria.

[0081] The scheduling method determination unit 108 executes the target process using the method that corresponds to the higher processing performance of the target process, among memory overload execution (first method) and delayed execution (second method), based on the first predicted completion time and the second predicted completion time.

[0082] The scheduler 107 allocates computing resources (GPU 10b) to the user program 101. The scheduler 107 also manages the job allocation status for computing resources in this information processing system 1. The scheduler 107 manages the allocation status of GPU 10b.

[0083] The scheduler 107 assigns a process to the GPU 10b using the scheduling method (delayed execution or memory overload execution) determined by the scheduling method determination unit 108.

[0084] For example, if running a process on GPU10b would result in a memory overload, the scheduler 107 will schedule the process to run immediately on GPU10b if the process is to be assigned to GPU10b using memory overload execution. Alternatively, if the process is to be assigned to GPU10b using delayed execution, the scheduler 107 will wait for the process to run until the GPU memory usage no longer exceeds the installed GPU memory. The scheduler 107 will then schedule the process to start processing on GPU10b at the earliest possible time, provided that the GPU memory usage does not exceed the installed GPU memory.

[0085] The scheduler 107 may, for example, instruct (respond to) the AI ​​framework 102 to execute a process on the GPU 10b at a time determined by scheduling.

[0086] The scheduler 107 may manage the allocation status of GPU 10b using, for example, a resource table (not shown). In the resource table, for example, if a job is assigned to GPU 10b, information identifying the GPU 10b may be associated with information identifying the assigned process. If no job is assigned to GPU 10b, the resource table may be associated with information indicating that GPU 10b is available.

[0087] When the scheduler 107 receives a GPU allocation request from the AI ​​framework 102, it checks whether GPU 10b is available for allocation to this process. If GPU 10b is available, the scheduler 107 allocates GPU 10b to the process.

[0088] (B) Operation The processing of the information processing system 1 according to one embodiment configured as described above will be explained according to the flowchart (steps A1 to A6) shown in Figure 8.

[0089] In step A1, user program 101 is launched. When user program 101 executes the GPU processing function of AI framework 102, a pre-configured hook for this GPU processing function is executed (step A2).

[0090] This hook causes the first library 103 to be executed in the AI ​​framework 102. The first library 103 sends a GPU resource request to the scheduler 107 and then waits (step A3).

[0091] The AI ​​framework 102 receives a response to the GPU resource request from the scheduler 107 (step A4) and executes the GPU processing (step A5). During the GPU processing, the second library 104 instructs the memory allocation control unit 106 to execute, for example, the cudaMallocManaged function to perform memory allocation. The memory allocation control unit 106 responds to the AI ​​framework 102 with a pointer to the unified memory address space.

[0092] AI framework 102 accesses memory using pointers in a unified memory address space. If the data to be accessed does not exist in GPU memory, the data is transferred from system memory.

[0093] In step A6, CPU processing (for example, post-processing or pre-processing for the next process) is performed. After that, the process returns to step A2.

[0094] Next, the scheduling method in the information processing system 1 according to one embodiment will be explained according to the flowcharts shown in Figures 9 and 10. Figure 9 is a flowchart (steps B1 to B3) for explaining the processing of the scheduler 107, and Figure 10 is a flowchart (steps C1 to C11) for explaining the processing of the scheduling method determination unit 108.

[0095] When the scheduler 107 receives a GPU resource request from the AI ​​framework 102 (step B1 in Figure 9), it schedules the start time for GPU processing (step B2 in Figure 9). The scheduler 107 schedules the GPU processing to start at the earliest possible time, provided that the GPU memory is not exceeded. The scheduler 107 also activates the scheduling method determination unit 108.

[0096] The scheduling method determination unit 108 checks whether a memory overload would occur if the GPU processing of the target process were started at the current time (step C1 in Figure 10). If the check does not result in a memory overload (see No. route in step C1), the scheduling method determination unit 108 terminates its processing.

[0097] In this case, if starting the GPU processing of the target process at the current time would not result in a memory overload, the start time of the GPU processing scheduled by scheduler 107 is the current time.

[0098] When the scheduled time for GPU processing arrives, the scheduler 107 responds to the AI ​​framework 102 with an instruction to start the process execution (step B3 in Figure 9). In other words, the scheduler 107 instructs the GPU 10b to immediately start the process execution.

[0099] On the other hand, if the verification in step C1 results in a memory overload (see the Yes route in step C1), proceed to step C2 in Figure 10.

[0100] In step C2, the scheduling method determination unit 108 accesses the process history stored in the process history storage unit 109.

[0101] In step C3 of Figure 10, the scheduling method determination unit 108 obtains information regarding the PID of the target program (target process) from the process history.

[0102] The information to be obtained may include the execution time of the GPU process when memory overload is performed (t1+t2), the execution time of the GPU process when delayed execution is performed (t0), the amount of GPU memory M used by the process during GPU processing, and the delay rate α in the event of memory overload.

[0103] In step C4 of Figure 10, the scheduling method determination unit 108 checks the process history to see if there is any GPU processing execution time corresponding to the PID of the target process when memory overload is performed.

[0104] If there is no GPU processing time in the process history when memory overload execution is performed (see the Yes route in step C4), the process proceeds to step C5 in Figure 10. In step C5, the scheduling method determination unit 108 determines memory overload execution as the scheduling method. After that, the process proceeds to step C11 in Figure 10.

[0105] If the process history contains execution time for GPU processing when memory overload occurs (see No. route in step C4), proceed to step C6 in Figure 10.

[0106] In step C6 of Figure 10, the scheduling method determination unit 108 checks the process history to see if there is no execution time for GPU processing at the time of delayed execution corresponding to the PID of the target process.

[0107] If there is no GPU processing time in the process history when delayed execution is performed (see the Yes route in step C6), the process proceeds to step C7 in Figure 10. In step C7, the scheduling method determination unit 108 determines delayed execution as the scheduling method. After that, the process proceeds to step C11 in Figure 10.

[0108] If the process history contains execution time for GPU processing during delayed execution (see No. route in step C6), proceed to step C8 in Figure 10.

[0109] In step C8, the scheduling method determination unit 108 uses information obtained from the process history to calculate the predicted completion time when processing is performed with memory overload.

[0110] In step C9 of Figure 10, the scheduling method determination unit 108 uses the scheduling result of the scheduler 107 to calculate the predicted completion time when processing is performed with delayed execution.

[0111] In step C10 of Figure 10, the scheduling method determination unit 108 compares the predicted completion times of GPU processing for all processes and determines the method that will terminate earlier, using either memory overload execution or delayed execution.

[0112] Subsequently, in step C11, the scheduling method determination unit 108 returns the GPU processing start time based on the determined method to the scheduler 107. After that, the process terminates.

[0113] (C) Effects Thus, according to the information processing system 1 as an example of the embodiment, the memory allocation control unit 106 manages the CPU memory and GPU memory as a single integrated memory. This makes it possible to avoid the occurrence of OOM.

[0114] Furthermore, when the target process is to be executed on the GPU 10b, the scheduling method determination unit 108 predicts the estimated completion time of the GPU processing when the target process is executed with memory overload (first estimated completion time) and the estimated completion time of the GPU processing when the target process is executed without memory overload (second estimated completion time).

[0115] The scheduling method determination unit 108 then identifies (determines) the method corresponding to the higher processing performance of the target process, either memory overload execution or delayed execution, and the scheduler 107 performs scheduling using this determined method. This reduces the time required for processing the target process and improves throughput. In other words, it improves the performance of processing the process.

[0116] Furthermore, the scheduling method determination unit 108 identifies (determines) for each process the method that corresponds to the one with higher processing performance for the target process, either memory overload execution or delayed execution. This allows each of multiple processes to be executed using the most advantageous method, thereby speeding up process execution for the entire system.

[0117] Furthermore, the scheduling method determination unit 108 determines the predicted completion time of GPU processing for all running processes in the case of delayed execution by scheduling the target process to be executed at the earliest possible time without exceeding the amount of GPU memory installed.

[0118] This makes it possible to accurately and easily determine the predicted completion time of GPU processing for all running processes when processing with delayed execution.

[0119] Furthermore, the scheduling method determination unit 108 determines the predicted completion time of GPU processing for all running processes when processing with memory overload, based on the execution time of processing by the GPU 10b when memory overload is performed, the execution time of processing by the GPU 10b when delayed execution is performed, and the amount of integrated memory used by the process for processing by the GPU 10b.

[0120] This makes it possible to accurately and easily determine the predicted completion time of GPU processing for all running processes when processing with memory overload.

[0121] The scheduling method determination unit 108 determines the scheduling method based on predetermined performance criteria, selecting the one with better performance between memory overload execution and delayed execution. This allows processes to be processed more efficiently, thereby improving processing performance.

[0122] (D) Other The disclosed technology is not limited to the embodiments described above and can be implemented in various modified forms without departing from the spirit of this embodiment.

[0123] For example, the embodiments described above show an example in which the computer 10 has one CPU 10a, but it is not limited to this, and the computer 10 may have two or more CPUs 10a. Similarly, the embodiments described above show an example in which the computer 10 has one GPU 10b, but it is not limited to this, and the computer 10 may have two or more GPUs 10b.

[0124] Furthermore, while the embodiments described above include examples where the memory control library 105 is a CUDA library and the memory allocation control unit 106 performs the function of cudaMallocManaged(), the invention is not limited to these examples. The functions of the memory control library 105 and the memory allocation control unit 106 may be implemented using other known methods.

[0125] Furthermore, the above disclosure makes it possible for those skilled in the art to implement and manufacture this embodiment.

[0126] (E) Note The following additional information is disclosed regarding the embodiments described above.

[0127] (Note 1) In a computer having a first memory used by a first computing resource and a second memory used by a second computing resource, The first memory and the second memory are managed as a single integrated memory. When the process to be processed is executed on the second computing resource, The system predicts a first predicted completion time when the process to be processed is executed using a first method in which the process is allocated to exceed the amount of memory in the second memory, and a second predicted completion time when the process to be processed is executed using a second method in which the process is allocated with a delay so as not to exceed the amount of memory in the second memory. Based on the first predicted completion time and the second predicted completion time, the process to be processed is executed using the method corresponding to the higher processing performance of the process to be processed among the first and second methods. An information processing program that causes the aforementioned computer to perform the processing.

[0128] (Note 2) The process for predicting the second predicted completion time obtains the first predicted completion time by scheduling the process to be executed at the earliest possible time without exceeding the memory capacity of the second memory. An information processing program as described in Appendix 1, characterized by including processing.

[0129] (Note 3) The process for predicting the first predicted completion time calculates the first predicted completion time based on the execution time of the processing by the second computing resource when the first method is implemented, the execution time of the processing by the second computing resource when the second method is implemented, and the amount of integrated memory used by the process for processing by the second computing resource. An information processing program as described in Appendix 1 or 2, characterized by including processing.

[0130] (Note 4) An information processing device having a first memory used by a first computing resource and a second memory used by a second computing resource, The first memory and the second memory are managed as a single integrated memory. When the process to be processed is executed on the second computing resource, The system predicts a first predicted completion time when the process to be processed is executed using a first method in which the process is allocated to exceed the amount of memory in the second memory, and a second predicted completion time when the process to be processed is executed using a second method in which the process is allocated with a delay so as not to exceed the amount of memory in the second memory. Based on the first predicted completion time and the second predicted completion time, the process to be processed is executed using the method corresponding to the higher processing performance of the process to be processed among the first and second methods. An information processing apparatus characterized by having a control unit that performs processing.

[0131] (Note 5) The control unit, In the process of predicting the second predicted completion time, the first predicted completion time is obtained by scheduling the process to be executed at the earliest possible time without exceeding the memory capacity of the second memory. An information processing apparatus according to Appendix 4, characterized by performing processing.

[0132] (Note 6) The control unit, In the process of predicting the first predicted completion time, the second predicted completion time is calculated based on the execution time of the processing by the second computing resource when the first method is implemented, the execution time of the processing by the second computing resource when the second method is implemented, and the amount of integrated memory used by the process for processing by the second computing resource. An information processing device according to Appendix 4 or 5, characterized by performing processing.

[0133] (Note 7) In a computer having a first memory used by a first computing resource and a second memory used by a second computing resource, The first memory and the second memory are managed as a single integrated memory. When the process to be processed is executed on the second computing resource, The system predicts a first predicted completion time when the process to be processed is executed using a first method in which the process is allocated to exceed the amount of memory in the second memory, and a second predicted completion time when the process to be processed is executed using a second method in which the process is allocated with a delay so as not to exceed the amount of memory in the second memory. Based on the first predicted completion time and the second predicted completion time, the process to be processed is executed using the method corresponding to the higher processing performance of the process to be processed among the first and second methods. An information processing method characterized in that the processing is performed by the computer.

[0134] (Note 8) The process for predicting the second predicted completion time obtains the first predicted completion time by scheduling the process to be executed at the earliest possible time without exceeding the memory capacity of the second memory. The information processing method described in Appendix 7, characterized by including processing.

[0135] (Note 9) The process for predicting the first predicted completion time calculates the second predicted completion time based on the execution time of the processing by the second computing resource when the first method is implemented, the execution time of the processing by the second computing resource when the second method is implemented, and the amount of integrated memory used by the process for processing by the second computing resource. The information processing method according to Appendix 7 or 8, characterized by including processing. [Explanation of Symbols]

[0136] 1. Information Processing System 10 Computers 10a CPU 10b GPU 10c memory 10d storage section 10e IF section 10f IO section 10g reading unit 10-hour program 10i recording media 10j bus 101 User Programs 102 AI Frameworks 103 Library 1 104 Second Library 105 Memory Control Library 106 Memory allocation control unit 107 Scheduler 108 Scheduling method determination unit 109 Process history storage unit

Claims

1. In a computer having a first memory used by a first computing resource and a second memory used by a second computing resource, The first memory and the second memory are managed as a single integrated memory. When the process to be processed is to be executed on the second computing resource, The system predicts a first predicted completion time when the target process is executed using a first method that allocates the process by exceeding the memory capacity of the second memory, and a second predicted completion time when the target process is executed using a second method that delays the allocation of the process so as not to exceed the memory capacity of the second memory. Based on the first predicted completion time and the second predicted completion time, the process to be processed is executed using the method corresponding to the higher processing performance of the process to be processed, among the first and second methods. An information processing program that causes the aforementioned computer to perform the processing.

2. The process for predicting the second predicted completion time obtains the first predicted completion time by scheduling the process to be executed at the earliest possible time without exceeding the memory capacity of the second memory. The information processing program according to claim 1, characterized by including processing.

3. The process for predicting the first predicted completion time calculates the second predicted completion time based on the execution time of the processing by the second computing resource when the first method is implemented, the execution time of the processing by the second computing resource when the second method is implemented, and the amount of integrated memory used by the process for processing by the second computing resource. An information processing program according to claim 1 or 2, characterized by including processing.

4. An information processing device having a first memory used by a first computing resource and a second memory used by a second computing resource, The first memory and the second memory are managed as a single integrated memory. When the process to be processed is to be executed on the second computing resource, The system predicts a first predicted completion time when the target process is executed using a first method that allocates the process by exceeding the memory capacity of the second memory, and a second predicted completion time when the target process is executed using a second method that delays the allocation of the process so as not to exceed the memory capacity of the second memory. Based on the first predicted completion time and the second predicted completion time, the process to be processed is executed using the method corresponding to the higher processing performance of the process to be processed, among the first and second methods. An information processing apparatus characterized by having a control unit that performs processing.

5. In a computer having a first memory used by a first computing resource and a second memory used by a second computing resource, The first memory and the second memory are managed as a single integrated memory. When the process to be processed is to be executed on the second computing resource, The system predicts a first predicted completion time when the target process is executed using a first method that allocates the process by exceeding the memory capacity of the second memory, and a second predicted completion time when the target process is executed using a second method that delays the allocation of the process so as not to exceed the memory capacity of the second memory. Based on the first predicted completion time and the second predicted completion time, the process to be processed is executed using the method corresponding to the higher processing performance of the process to be processed, among the first and second methods. An information processing method characterized in that the processing is performed by the computer.

Images