Job management method and information processing apparatus

Abstract

To suppress the occurrence of memory shortage in each calculation node when a plurality of jobs operate in parallel in a system having a plurality of calculation nodes.SOLUTION: In a system having a plurality of calculation nodes, a control unit causes at least a part of the plurality of calculation nodes to execute a plurality of jobs #A and #B by switching the plurality of jobs at a divided time in a state where data of the plurality of jobs are distributed and held on a memory in each calculation node. For each of jobs #A and #B input to the system, a control unit acquires the number of set nodes set to be used for execution of the job, and determines whether the number of set nodes is equal to or less than a threshold. The processor is configured to increase, within a range of a total number of computing nodes of the system, the number of computing nodes used to execute at least some of the plurality of jobs for which the set number of computing nodes is equal to or less than the threshold.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] The present invention relates to a job management program, a job management method, and an information processing apparatus. [Background technology]

[0002] 2. Description of the Related Art Computer systems are known in which a plurality of computing nodes, each of which is a computer equipped with a processor and a memory, cooperate to perform calculations.

[0003] In a system with multiple computing nodes, a time slice execution technique is known in which multiple jobs are executed in parallel, i.e., time-shared execution, in order to efficiently use the computing nodes. In time slice execution, the data of multiple jobs is temporarily stored in a distributed state in the memory of each computing node. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2023-164156 Summary of the Invention [Problem to be solved by the invention]

[0005] Therefore, when multiple jobs are executed in parallel in a system having multiple computing nodes, the amount of memory storage space used by each computing node (hereinafter, sometimes referred to as "memory usage") increases. If the memory usage in a computing node exceeds the available storage capacity of the memory storage space installed in the computing node (hereinafter, sometimes referred to as "memory capacity"), a memory shortage occurs. When a memory shortage occurs, a memory swap operation is performed to exchange data between areas of the same capacity in the memory and an external storage device, which may result in a decrease in system performance.

[0006] In one aspect, an object of the present invention is to prevent memory shortages from occurring in each computing node when multiple jobs run in parallel in a system having multiple computing nodes. [Means for solving the problem]

[0007] In one aspect, a job management program is used in a system having a plurality of computing nodes. The job management program distributes and stores data of a plurality of jobs in memory of each computing node, and switches between the plurality of jobs at timed intervals to execute the jobs on at least some of the plurality of computing nodes. The job management program causes a computer to perform the following processing. The processing may acquire, for each job submitted to the system, a number of set nodes set to be used for executing the job. The processing may determine whether the number of set nodes is equal to or less than a threshold. The processing may increase the number of computing nodes used for executing at least some of the plurality of jobs whose number of set nodes is equal to or less than the threshold, within the total number of computing nodes in the system. [Effects of the Invention]

[0008] In one aspect, the present invention can prevent memory shortages in each computing node when multiple jobs run in parallel in a system having multiple computing nodes. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram illustrating an example of a hardware (HW) configuration of a computing system according to an embodiment. [Figure 2] FIG. 2 is a block diagram showing an example of a hardware configuration of a calculation node shown in FIG. [Figure 3] FIG. 2 is a block diagram illustrating an example of the hardware configuration of a computer. [Figure 4] FIG. 10 illustrates an example of a job execution state in non-time slice execution. [Figure 5]FIG. 10 is a diagram illustrating an example of a job execution state in time slice execution according to a comparative example. [Figure 6] FIG. 10 illustrates an example of job switching during time slice execution. [Figure 7] FIG. 10 is a diagram illustrating an example of a job execution state in time slice execution according to an embodiment of the present invention. [Figure 8] FIG. 10 is a diagram illustrating another example of the execution state of a job in time slice execution according to an embodiment of the present invention. [Figure 9] FIG. 10 is a diagram illustrating the relationship between the number of computing nodes that execute a job and memory usage. [Figure 10] FIG. 10 is a diagram illustrating an example of a signal transmission process for executing a time slice. [Figure 11] 10 is an example of a correspondence table linking job IDs and process IDs in each computing node. [Figure 12] FIG. 10 is a diagram showing experimental results on the effect of time slice execution on processing time. [Figure 13] 2 is a block diagram showing an example of a functional configuration of a login server shown in FIG. 1. [Figure 14] 10 is a flowchart illustrating an example of a job management process according to an embodiment of the present invention. [Figure 15] 10 is a flowchart illustrating an example of processing when the number of set nodes is 1 according to the embodiment of the present invention. [Figure 16] FIG. 10 is a diagram illustrating an example of performance characteristics. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. However, the embodiment described below is merely an example, and is not intended to exclude various modifications or application of techniques not explicitly described below. For example, this embodiment can be implemented with various modifications within the scope of its spirit. In the drawings used in the following description, parts with the same reference numerals represent the same or similar parts unless otherwise specified.

[0011] [A] Hardware configuration example of one embodiment FIG. 1 is a block diagram showing an example of the hardware (HW) configuration of a computing system 1 according to an embodiment.

[0012] As shown in FIG. 1, the computing system 1 may illustratively include a computing server 2, a management server 3, a login server 4, a console 5, and a network device 6.

[0013] The computation server 2 executes the computations assigned to it. The computation server 2 may be, for example, a computer system that realizes the functions of HPC (High Performance Computing). The computation server 2 includes a plurality of computation nodes 20-1 to 20-N (N is the number of computation nodes) that are communicatively connected to each other, and performs computations by linking these plurality of computation nodes 20-1 to 20-N (sometimes referred to as computation nodes 20). The computation server 2 may include, for example, several thousand or more computation nodes 20.

[0014] Each computation node 20 is an example of a computer or calculator equipped with a processor and memory. Each computation node 20 may be connected to a network device 6. The network device 6 may be a network switch, such as a Layer 2 switch (L2 switch). The network switch may be multi-stage. Multiple computation nodes 20 may constitute an indirect network computer connected mainly via the network device 6. In other words, each computation node 20 may function as a server. However, the computation server 2 is not limited to an indirect network computer. For example, multiple computation nodes 20 may be connected via a direct connection network, and various modifications can be made. A program for executing a computation assigned to the computation server 2 is executed by one or more computation nodes 20. When executed by multiple computation nodes 20, the computation contents are communicated between the computation nodes 20 via the network device 6 to obtain the computation results.

[0015] The management server 3 is an example of a computer or information processing device that manages the order of multiple jobs (programs) to be executed by the calculation servers 2 and computer resources. The management server 3 executes a job scheduler. The job scheduler is, for example, Slurm. However, the job scheduler is not limited to Slurm.

[0016] When a user accesses the login server 4, the login server 4 verifies the user's identity and, if it determines that the user is legitimate, allows the user to use the system.

[0017] The console 5 is a terminal device operated by a user to cause the calculation server 2 to execute calculations, and is an example of a computer. The console 5 is connected to the login server 4 via the network 1b. The calculation system 1 may include multiple consoles 5.

[0018] In this embodiment, a user logs in to the login server 4 via the network 1b by operating the console 5. For example, the user submits a job to be executed by the calculation server 2 via the console 5. The console 5 transmits to the login server 4 a program, a job script, etc., related to the job to be executed by the calculation server 2.

[0019] The login server 4 receives the program, job script, etc. from the console 5. This causes the job to be submitted to the computing system 1. The login server 4 executes a process to increase the arguments in the program and the number of nodes in the job script. The login server 4 transmits the program, job script, etc. that reflect the increase in the number of nodes to the management server 3. This process will be described later.

[0020] The login server 4, the management server 3, and the calculation server 2 may be connected via a network device 6 to enable high-speed communication.

[0021] Fig. 2 is a block diagram showing an example of a hardware configuration of the calculation node 20 shown in Fig. 1. The calculation node 20 includes CPUs (Central Processing Units) 21-1 and 21-2 (sometimes referred to as CPU 21), a memory 22, a memory controller 23, and an IF device 24.

[0022] The CPU 21 is an example of a processing unit that performs various controls and calculations, and may execute jobs.

[0023] The memory 22 is an example of HW that stores various data, programs, and other information, and is an example of a main memory. The memory 22 may be, for example, one or both of a volatile memory such as a DRAM (Dynamic Random Access Memory) and a non-volatile memory such as a PM (Persistent Memory).

[0024] The memory controller 23 is a controller that controls access between the CPU 21 and the memory 22, and is, for example, an integrated circuit (IC).

[0025] The IF device 24 is a communication device used for communication between the calculation nodes 20 .

[0026] In a computing system 1 having a plurality of computing nodes 20, data of a plurality of jobs is distributed and stored in the memory 22 of each computing node 20. While the data of the plurality of jobs is distributed and stored, and the plurality of jobs are switched at delimited time intervals and executed on at least some of the plurality of computing nodes 20, memory shortages in the memory 22 are prevented by processing in the login server 4, etc. Details of the processing will be described later.

[0027] Next, an example of the hardware configuration of the login server 4 shown in FIG. 1 will be described.

[0028] Fig. 3 is a block diagram showing an example of the hardware configuration of the login server 4. As shown in Fig. 3, the login server 4 may include, as its HW configuration, a CPU 4a, a memory 4b, an IF device 4c, a graphics processing device 4d, a storage device 4e, an IO (Input / Output) device 4f, and a reading device 4g, for example.

[0029] The CPU 4a is an example of an arithmetic processing unit or processor that performs various controls and calculations. The CPU 4a may be connected to each block in the login server 4 via a bus 4j so that they can communicate with each other. The CPU 4a may be a multiprocessor including multiple processors, a multi-core processor having multiple processor cores, or a configuration having multiple multi-core processors.

[0030] Instead of the CPU 4a, a processor such as an integrated circuit (IC) such as an MPU, APU, DSP, ASIC, or FPGA may be provided. Note that a combination of two or more of these integrated circuits may be used as a processor. MPU is an abbreviation for Micro Processing Unit. APU is an abbreviation for Accelerated Processing Unit. DSP is an abbreviation for Digital Signal Processor, ASIC is an abbreviation for Application Specific IC, and FPGA is an abbreviation for Field-Programmable Gate Array.

[0031] The memory 4b is an example of HW that stores various data, programs, and other information. The memory 4b may be, for example, a volatile memory such as a DRAM, or a non-volatile memory such as a PM. The memory 4b is an example of a main storage device.

[0032] The IF device 4c is an example of a communication IF that controls connections and communications between the compute nodes 20, the management server 3, the login server 4, and the console 5. The IF device 4c may include an adapter that complies with a high-speed interconnect via a network device 6, a LAN (Local Area Network) such as Ethernet (registered trademark), or optical communications such as FC. The adapter may support either or both of wireless and wired communication methods.

[0033] The program 4h may be downloaded from the network to the login server 4 via the communication IF and stored in the storage device 4e.

[0034] The graphics processing device 4d is an example of a processing device that controls screen display for an output device such as a monitor among the IO devices 4f. The graphics processing device 4d may also be configured as an accelerator that executes various calculations, such as machine learning processing and estimation processing using a machine learning model. Examples of the graphics processing device 4d include various processing devices, such as a GPU (Graphics Processing Unit), an APU, a DSP, an ASIC, an FPGA, or other integrated circuits (ICs).

[0035] The storage device 4e is an example of HW that stores various types of data, programs, and other information. The storage device 4e may be used as local storage for each of the computation node 20, the management server 3, the login server 4, and the console 5. Examples of the storage device 4e include various storage devices such as a magnetic disk device such as a hard disk drive (HDD), a semiconductor drive device such as a solid state drive (SSD), and a nonvolatile memory. Examples of the nonvolatile memory include a flash memory, a storage class memory (SCM), and a read-only memory (ROM).

[0036] The storage device 4e may store a program 4h. The program 4h is a program executed by the CPU 4a or the graphics processing device 4d. The program 4h stored in the login server 4 may include a job management program that increases the number of nodes that execute a job. The program 4h stored in the storage device 4e may also include a program to be executed by the computation server 2 (computation node 20), for example.

[0037] For example, the CPU 4a of the login server 4 can implement the function of the login server 4 (for example, the control unit 110 shown in FIG. 13) by loading the program 4h stored in the storage device 4e into the memory 4b and executing it.

[0038] The IO device 4f may include one or both of an input device and an output device. Examples of input devices include a keyboard, a mouse, and a touch panel. Examples of output devices include a monitor, a projector, and a printer. The IO device 4f may also include a display device such as a touch panel that combines an input device and an output device. The output device may be connected to the graphics processing device 4d.

[0039] The reading device 4g is an example of a reader that reads data or program information recorded on the recording medium 4i. The reading device 4g may include a connection terminal or device to which the recording medium 4i can be connected or inserted. Examples of the reading device 4g include an adapter that complies with USB (Universal Serial Bus) or the like, a drive device that accesses a recording disk, and a card reader that accesses a flash memory such as an SD card. Note that the recording medium 4i may store a program 4h, and the reading device 4g may read the program 4h from the recording medium 4i and store it in the storage device 4e.

[0040] Examples of the recording medium 4i include non-transitory computer-readable recording media such as magnetic / optical disks and flash memories. Examples of magnetic / optical disks include flexible disks, CDs (Compact Discs), DVDs (Digital Versatile Discs), Blu-ray Discs, and HVDs (Holographic Versatile Discs). Examples of flash memories include semiconductor memories such as USB memories and SD cards.

[0041] The above-described HW configuration of the login server 4 is an example. Therefore, the HW in the login server 4 may be increased or decreased (for example, adding or deleting any block), divided, integrated in any combination, or added or deleted buses, as appropriate. The HW configuration of the computation nodes 20, management server 3, console 5, etc. may also be the same as that shown in FIG. 3.

[0042] [B] Description of one embodiment FIG. 4 is a diagram showing an example of the execution state of job 7 in non-time slice execution. In FIG. 4, jobs #1 to #9 are shown as multiple jobs 7. The length of the rectangles representing each of jobs #1 to #9 in the horizontal direction on the paper indicates the time during which each of jobs #1 to #9 is being executed. The length of the rectangles representing each of jobs #1 to #9 in the vertical direction on the paper indicates the number of nodes among the computation nodes 20 that execute each of jobs #1 to #9. In particular, the length of each rectangle in FIG. 4 in the vertical direction on the paper is the number of set nodes that are set to be used to execute each of jobs #1 to #9.

[0043] The management server 3 executes a scheduler to determine the order of operations (execution order) of the jobs 7 according to the calculation program. The order of operations may be determined based on the number of set nodes set to be used for the execution of the jobs 7 requested from the console 5, the priority of the jobs 7, the number of nodes currently in use in the calculation server 2, and the number of nodes scheduled to be released in the future. The order of operations may be determined using an existing scheduler function.

[0044] The scheduler reserves computer resources to be used in the calculation servers 2 according to the determined operation order, and causes the calculation servers 2 to execute the calculation programs using the reserved computer resources.

[0045] As shown in Figure 4, in non-time slice execution, the scheduler basically executes jobs 7 in the order in which they were submitted. Once started, job 7 continues to run with computer resources reserved until it is finished. Even if job 7 is submitted before other jobs 7, if it is not in a state where it can be executed at that time, job 7 will be made to wait until it becomes in a state where it can be executed.

[0046] For large-scale jobs like job #7, which use more than a predetermined number of nodes, it becomes difficult to start other processes before or during execution. Specifically, in order to secure nodes to execute job #7, the execution of other jobs #8 and #9 is suppressed before job #7 is executed. As a result, even though jobs #7, #8, and #9 are queued, an area 8 is created where computer resources cannot be used effectively.

[0047] FIG. 5 illustrates an example of job execution status in time-sliced ​​execution according to a comparative example. Job #7, a large-scale job, and other jobs #6, #8, and #9 are executed in a fine-grained time-sliced ​​manner (time slice). Job #7 is divided into jobs #7-1, #7-2, and #7-3 and executed. Similarly, job #6 is divided into jobs #6-1, #6-2, and #6-3 and executed. Job #8 is divided into jobs #8-1 and #8-2 and executed, and job #9 is divided into jobs #9-1 and #9-2 and executed. As a result, job #7, a large-scale job, can be executed immediately, and the utilization rate of the computing nodes is improved. Large-scale jobs, such as jobs executed by all computing nodes 20 (all systems) of the computing server 2, can be executed smoothly at any time.

[0048] FIG. 6 is a diagram illustrating an example of switching of job 7 in time slice execution. In time slice execution, a computing server 2 having multiple computing nodes 20 executes a group of jobs, job #A and job #B, by switching between them at time intervals on the same computer resource. In the example of FIG. 4, job #7 is job #A, and jobs #6, #8, and #9 are job #B. As shown in FIG. 5, job #7 and jobs #6, #8, and #9 are executed by switching between them at time intervals. In FIG. 5, job #7 is executed as jobs #7-1, #7-2, and #7-3 by time slice execution. Furthermore, jobs #8 and #9 are executed as jobs #8-1 and #8-2 and jobs #9-1 and #9-2 by time slice execution.

[0049] Switching time t c1 is the time when job #A is paused and job #B is executed, and the switching time t c2 is the time during which job #B is paused and job #A is executed. c1 and t c2 may be 0.1 seconds or more and 1 second or less, and t c1 and t c2 is equal to the time t c In the following, t c The time slice execution is performed by dividing the jobs 7 into groups by a switching time t c This is a process of switching between jobs and executing them on at least some of the multiple computation nodes 20. The switching between job #A and job #B is executed in synchronization across the entire system (i.e., all computation nodes 20). This makes it possible to prevent performance degradation due to communication waits caused by synchronization loss.

[0050] In order to time-slice execute multiple jobs #A and #B on a certain resource, those jobs #A and #B must be left in the memory 22 of each computing node 20. Therefore, if jobs #A and #B that use a large amount of memory are executed by time slicing, they may not be able to be executed or performance may be degraded.

[0051] FIG. 7 is a diagram showing an example of the execution state of job 7 in time slice execution in the embodiment of the present invention.

[0052] The CPU 4a of the login server 4 obtains the number of set nodes for each job 7 submitted to the computing system 1 through the console 5 from information such as the program and job script received from the console 5. The number of set nodes is the number of nodes set to be used to execute job 7. The CPU 4a determines whether the number of set nodes is equal to or less than a threshold. In one example, the threshold is half the total number of computing nodes 20 in the computing system 1. In the case shown in FIG. 4, jobs #6, #8, #9, etc. are determined to have the number of set nodes (i.e., the length of the vertical axis direction of the corresponding rectangle) equal to or less than the threshold. Job #7, which is a large-scale job, is determined to have the number of set nodes greater than the threshold. Based on the determination result, the CPU 4a increases the number of nodes used to execute jobs #8 and #9. In FIG. 7, job #8 is switched at a switching time t c The jobs #8-1 and #8-2, which are separated by , and the job #9 are switched at a time t c The number of nodes used to execute jobs #9-1 and #9-2 separated by is increased to twice the set number of nodes.

[0053] To increase the number of nodes used to execute job 7 in this way, a wrapper program (i.e., a conversion program) may be provided in the job submission program provided in the login server 4. The CPU 4a executes the wrapper program to increase the arguments in the program and the number of nodes in the job script. The CPU 4a may transmit the program, job script, etc. that reflect the increased number of nodes to the management server 3 via the network device 6, etc.

[0054] In this example, the login server 4 that executes the login process executes processes such as obtaining the number of set nodes, determining the threshold value, and increasing the number of nodes, but the embodiment of the present invention is not limited to this case. At least a part of these processes may be performed by any information processing device provided between the console 5 and the management server 3. For example, depending on the specifications of the computing system 1, at least a part of these processes may be executed by one function of the management server 3. Furthermore, a server that performs at least a part of these processes may be provided separately from the login server 4 and management server 3 that execute the login process.

[0055] 8 is a diagram showing another example of the execution state of job 7 in time slice execution in the embodiment of the present invention. In the above-mentioned FIG. 7, job #6 is switched at switching time t c 8, job #6-1, which is a large-scale job subject to time slice execution, has already started execution before job #7, which is a large-scale job subject to time slice execution, starts. Therefore, the login server 4 does not increase the number of nodes executing jobs #6-2 and #6-3, from the viewpoint of having the computation node 20 executing job #6-1 take charge of executing the remaining jobs #6-2 and #6-3. However, the login server 4 may increase the number of nodes used to execute jobs #6-1, #6-2, and #6-3, as shown in FIG.

[0056] In the process shown in FIG. 8, when the CPU 4a of the login server 4 receives jobs #6 and #7, it may determine that job #6 is subject to time-slicing execution. For example, if the maximum execution times for jobs #6 and #7 are specified by information from the console 5, the CPU 4a may determine that job #6 is subject to time-slicing execution based on that information and the number of nodes set. In this case, the CPU 4a may divide job #6, which is subject to time-slicing execution, into jobs #6-1, #6-2, and #6-3 and increase the number of nodes. Job #6-1 cannot be executed in parallel with job #5 due to the increased number of nodes, so it is executed after job #5 is completed. Specifically, to prevent jobs #5 and #6-1 from running in parallel, a restriction is imposed so that job #6-1 does not execute until the preceding job #5 is completed. This restriction may be implemented, for example, by creating a dependency relationship between jobs #7 using the afterany option in Slurm.

[0057] 9 is a diagram showing the relationship between the number of computing nodes that execute job 7 and memory usage. In the embodiment shown in Fig. 7 and Fig. 8, for job 7 that is the target of time slice execution, the number of computing nodes that execute job 7 is increased by changing and arranging the number of nodes so that the memory usage does not exceed the memory capacity of memory 22.

[0058] As described above, in a parallel application in which multiple computation nodes 20 are operated in parallel to perform calculations, data is distributed and stored in the memory 22 of each computation node 20. When the amount of data in a job 7 is constant, increasing the number of computation nodes can reduce the memory usage per node. The processes shown in FIGS. 7 and 8 are suitable for parallel applications in which a computation target domain is divided into multiple partial domains and each computation node 20 performs calculations on its assigned partial domain. Examples of such parallel applications include meteorological simulations and quantum simulations. Quantum simulation is a technique for simulating the state of quantum bits by storing them as state vectors in the memory 22. FIG. 9 will be used to explain the quantum simulation as an example.

[0059] In the example of Figure 9, the required memory usage doubles for each additional quantum bit. Also, the example shows a case where the total number of computation nodes 20 in the entire computation system 1 is 1024 + α. α may correspond to the number of redundant computation nodes provided in case a computation node 20 fails. For example, the total number of computation nodes 20 is 1056.

[0060] In one example, when a simulation of 40 qubits is executed using all of the computation nodes 20, the total memory usage across all of the computation nodes 20 is 16 TiB. When the total number of computation nodes 20 is 1024+α, the memory 22 per node holds 16 GiB of data, which is 16 TiB divided by 1024+α. In this case, the number of qubits to be simulated per node is 30 qubits, which is the value obtained by dividing 40 qubits by 1024+α.

[0061] If the number of computation nodes 20 executing job 7 is doubled while maintaining the number of qubits to be simulated, the memory usage per node is halved. For example, if a simulation calculation of 30 qubits of target qubits is performed per node, and then a simulation calculation of 30 qubits of target qubits is performed per two nodes, then the memory 22 per node only needs to hold 8 GiB of data. A simulation calculation of 30 qubits of target qubits per two nodes is equivalent to a simulation calculation of 29 qubits of target qubits, which is 30 qubits divided by 2, per node.

[0062] In FIG. 9, job #A group and job #B are input as jobs subject to time slice execution. Job #A group includes job #A-1 and job #A-2. In one example, job #A-1 and job #A-2 are input after job #6 (switching time t c , and #8 (switching time t c In one example, job #B corresponds to job #7 (which is divided into jobs #8-1 and #8-2 by switching time t c This may correspond to the case where the job is divided into jobs #7-1 to #7-3 by the job number.

[0063] In the example shown in FIG. 9, as shown in the left diagram, Job #A-1 and Job #A-2 are each 38-qubit jobs (38 Qubit Jobs) with 38 qubits as the number of qubits to be simulated, and the number of configured nodes is 256. The 38-qubit jobs (38 Qubit Jobs) are distributed to 256 nodes. In this case, the number of qubits to be simulated per node is the same as when 40 qubits are simulated across the total number of computation nodes 20 (1024 nodes + α), so the memory 22 per node holds 16 GiB of data. Similarly, for Job #B, the memory 22 per node holds 16 GiB of data.

[0064] Therefore, when job #A group and job #B are executed in a time slice manner, memory 22 must hold 32 GiB of data per node. As a result, memory usage exceeds memory capacity (see arrow P1), and a memory swap operation (SWAP) is performed to exchange data between areas of the same capacity in memory 22 and the external storage device, resulting in a decrease in system performance.

[0065] The CPU 4a determines whether the number of set nodes for each of jobs #A-1 and #A-2 is equal to or less than a threshold. The threshold may be half the total number of computing nodes 20 in the computing system 1. The number of set nodes for jobs #A-1 and #A-2 is determined to be equal to or less than the threshold. The number of set nodes for job #B is determined to be greater than the threshold. Therefore, based on the determination result, the CPU 4a increases the number of nodes used to execute jobs #A-1 and #A-2. In the right diagram of FIG. 9, the number of nodes used to execute jobs #A-1 and #A-2 is increased to twice the number of set nodes. As a result, a 38-qubit job (38 Qubit Job) is distributed to 512 nodes, and the memory usage of the memory 22 per node for jobs #A-1 and #A-2 is halved to 8 GiB. Therefore, even when jobs #A and #B are executed in a time-sliced ​​manner, the memory 22 per node only needs to hold 24 GiB of data. As a result, the memory usage becomes equal to or less than the memory capacity (see arrow P2), enabling memory operations on the memory 22. Since memory swap operations (SWAP) are avoided, degradation of system performance is suppressed.

[0066] However, if the number of set nodes for job #A-1 etc. is 1, doubling the number of nodes executing job #A-1 to 2 will result in communication between multiple calculation nodes 20, so separate processing may be performed. The processing when the number of nodes for job 7 is 1 will be described later.

[0067] In FIG. 9, if both Job #A and Job #B are large-scale jobs with a set number of nodes greater than the threshold (for example, the 40-qubit job in FIG. 9), CPU 4a executes processing to prevent two large-scale jobs from running in parallel. In one example, a restriction is imposed so that Job #B is not executed until the preceding Job #A is completed. This restriction may be implemented, for example, by creating a dependency between Jobs 7 using the afterany option in Slurm.

[0068] Fig. 10 is a diagram showing an example of a signal transmission process for executing a time slice. Fig. 11 is an example of a correspondence table 25 that links job IDs and process IDs in each computation node 20. Fig. 11 shows the correspondence table 25 in computation node 20-1. Similar correspondence tables 25 are created in the other computation nodes 20.

[0069] In one example, when time slice execution is performed, the management server 3 executes the processing shown in FIG. 10 based on the program arguments and the contents of the job script modified by a wrapper program provided on the login server 4 or the like.

[0070] The management server 3 may include a switching signal transmitter 31. The switching signal transmitter 31 transmits a switching signal to the computation nodes 20-1 to 20-n by broadcasting a packet, thereby stopping the job with job ID 30 and executing the next job with job ID 31. That is, the switching signal transmitter 31 notifies each computation node 20 of the job ID to be executed next. The switching signal transmitter 31 has a job list 32 listing jobs 7 to be executed. Each computation node 20 creates a correspondence table 25 that associates each job ID in the job list 32 with a process ID. Upon receiving a switching signal indicating the job ID to be executed next, the processor in the computation node 20 refers to the correspondence table 25 and transmits a STOP signal to each process to stop it or a CONT signal to resume it. The STOP and CONT signals are software interrupt functions of the kernel that notify processes and process groups of various events. 10 and 11 are examples of signal transmission processing for executing time slices, and the processing is not limited to the processing shown in FIGS.

[0071] FIG. 12 shows the experimental results on the effect of time-slicing on processing time. The left diagram in FIG. 12 shows the processing time when two 32-qubit jobs are time-sliced ​​in a computing system 1 with a total of eight computing nodes 20. Compared to twice the processing time when a 32-bit job is executed alone (i.e., when two 32-bit jobs are executed sequentially), the switching time t c When time slicing was performed with the time set to 1, 5, or 10 seconds, the processing time did not increase.

[0072] Similarly, the right diagram of FIG. 12 shows the processing time when a 32-qubit job and a 33-qubit job are executed in time slices in a computing system 1 having a total of eight computing nodes 20. Compared to the processing time when a 32-qubit job and a 33-qubit job are executed sequentially, the switching time t cWhen time slicing was performed with the interval set to 1, 5, or 10 seconds, the processing time did not increase. Therefore, time slicing did not result in a performance degradation due to overhead.

[0073] [C] Functional configuration example of one embodiment Fig. 13 is a block diagram showing an example of the functional configuration of the login server 4 shown in Fig. 1. The login server 4 may include a control unit 110. The control unit 110 is configured with a focus on the process of increasing the number of computation nodes by the login server 4. For example, the control unit 110 may be provided in the login server 4 as a function realized by executing a wrapper program (i.e., a conversion program), or may be provided in the management server 3 as part of the functions of a scheduler.

[0074] The control unit 110 may, for example, include a set node number acquisition unit 130, a time slice execution determination unit 132, a judgment unit 134, a one-node job processing unit 140, an increase unit 150, and an output unit 160. The one-node job processing unit 140 may, for example, include a memory usage acquisition unit 141, a performance characteristic acquisition unit 142, a node allocation information acquisition unit 143, and an execution time measurement unit 144.

[0075] Hereinafter, the functional blocks 130 to 160 included in the control unit 110 shown in FIG. 13 will be described with reference to operation examples shown in FIGS.

[0076] [D] Operation example 14 is a flowchart showing an example of job management processing according to an embodiment of the present invention. The processing in FIG. 14 may be executed by the login server 4. However, instead of the login server 4, another device may execute the processing.

[0077] The control unit 110 waits until it receives a job 7 from the console 5 (see the NO route in step S1). For example, the control unit 110 receives a program and a job script related to the job 7 from the console 5. When the control unit 110 receives the job 7 from the console 5 (see the YES route in step S1), the time slice execution determination unit 132 determines whether or not to perform time slice execution on the received multiple jobs 7 (step S2). For example, if the maximum execution time for each job 7 is specified by information from the console 5, the time slice execution determination unit 132 may determine that each job is subject to time slice execution based on that information and information on the number of set nodes. However, if a maximum execution time is not specified, the processing in step S2 may be omitted.

[0078] In the time slice execution, the data of the jobs 7 is distributed and stored in the memories 22 of the computing nodes 20, and the jobs 7 are executed at a time interval t c This is a process in which the processing is switched between the plurality of computing nodes 20 and executed by at least some of the computing nodes 20.

[0079] If time slice execution is not performed (see the NO route from step S2), the process proceeds to step S8, whereas if time slice execution is performed (see the YES route from step S2), the process proceeds to step S3.

[0080] The set node number acquisition unit 130 acquires, for each job 7 submitted to the computing system 1, the set node number that is set to be used for executing the job 7 (step S3).

[0081] The determination unit 134 determines whether the number of set nodes is more than one (step S4). If the number of set nodes is not more than one, that is, if it is 1 (see the NO route in step S4), the process proceeds to step S5. The one-node job processing unit 140 executes the process when the number of set nodes is 1 (step S5), and the process proceeds to step S8.

[0082] If the result of the determination in step S4 is that the number of set nodes is plural (see the YES route in step S4), the determination unit 134 determines whether the number of set nodes is equal to or less than a threshold (step S6). In one example, the threshold may be 1 / N (N is a natural number equal to or greater than 2) of the total number of calculation nodes 20 in the calculation system 1. In particular, the threshold may be half of the total number of calculation nodes 20 in the calculation system 1.

[0083] If the number of set nodes is equal to or less than the threshold (see the YES route in step S6), the increasing unit 150 increases the number of computation nodes used for execution of the job 7 within the range of the total number of computation nodes 20 in the computation system 1 (step S7). If the threshold is 1 / N of the total number of computation nodes 20, the increasing unit 150 may multiply the number of set nodes by N. In particular, if the threshold is 1 / 2 of the total number of computation nodes 20, the increasing unit 150 may double the number of set nodes.

[0084] The increasing unit 150 may perform a process of increasing the number of arguments in a program and the number of nodes in a job script.

[0085] If the number of set nodes is greater than the threshold value (see the NO route from step S6), the process proceeds to step S8.

[0086] The output unit 160 instructs the management server 3 to cause the computing nodes 20 to execute multiple jobs 7 (step S8). When the increase unit 150 creates a program, a job script, etc. that reflects the increase in the number of nodes, the output unit 160 transmits the changed program and job script to the management server 3. When step S8 ends, the process returns to step S1.

[0087] If job 7, which runs on one node, is run on two nodes, communication overhead occurs due to parallelization, and the extent of performance degradation may be significant. For example, if job 7 is changed from being run on four nodes to being run on eight nodes, communication between multiple nodes is already occurring when it is run on four nodes, and the amount of communication simply increases. On the other hand, if job 7 is changed from being run on one node to being run on two nodes, there is a big difference in that the presence or absence of inter-node communication is changed.

[0088] FIG. 15 is a flowchart showing an example of processing when the number of set nodes is 1 in the embodiment of the present invention. If the determining unit 134 determines that the number of set nodes for job 7 is 1, that is, if it determines that job 7 is a job using one node, the process of FIG. 15 may be executed.

[0089] When a job 7 is submitted, the memory usage acquisition unit 141 determines whether there is information from the user as to whether the memory usage of the job 7 is less than half of the memory capacity (memory capacity) of the memory 22 of the calculation node 20, i.e., the available memory capacity.

[0090] If the information from the user, i.e., the information from the console 5, indicates that the memory usage of the job 7 is less than half the memory capacity of the memory 22 of the calculation node 20 (see the YES route in step S10), the process proceeds to step S11. The one-node job processing unit 140 maintains the set node number of the job 7 as 1. In other words, the control unit 110 causes one calculation node 20 to execute the job 7 as a one-node using job (step S11). Thereafter, the control unit 110 ends the process.

[0091] If the information provided by the user indicates that the memory usage of the job 7 is not equal to or less than half the memory capacity of the memory 22 of the computing node 20 (see the NO route in S10), the process proceeds to step S12.

[0092] In step S12, the performance characteristic acquisition unit 142 determines whether the performance characteristic 9 (scalability information) of the program that executes the job 7 is known. Furthermore, if the performance characteristic 9 of the program that executes the job 7 can be acquired, the performance characteristic acquisition unit 142 acquires the performance characteristic 9. The performance characteristic 9 includes performance information when the number of computing nodes 20 is increased or decreased. The performance information may be processing speed and processing time.

[0093] If the performance characteristics 9 for the program executing job 7 are known (see the YES route in step S12), the performance characteristics acquisition unit 142 determines whether executing job 7 on two nodes will result in lower performance than when executed on one node (step S13).

[0094] FIG. 16 is a diagram showing an example of performance characteristics 9. As shown in FIG. 16, performance information characteristics 9 may be, for example, processing speeds for different numbers of nodes running job 7. When job 7 that runs on one node runs on two nodes, communication overhead occurs due to parallelization, and the extent of the decrease in processing speed may become significant. Performance characteristics 9 include performance information when job 7 runs on one computation node 20 and performance information when job 7 runs on two computation nodes 20.

[0095] If executing job 7 on two nodes results in lower performance than when executed on one node (see the YES route from step S13), the control unit 110 causes one computation node 20 to execute job 7 as a one-node job (step S14). Then, the control unit 110 ends the process. If executing job 7 on two nodes does not result in lower performance than when executed on one node (see the NO route from step S13), the control unit 110 causes two computation nodes 20 to execute job 7 as a two-node job (step S15). Then, the control unit 110 ends the process.

[0096] The processing in steps S12 to S15 is an example of processing for changing the number of computing nodes in accordance with the performance information for each number of computing nodes of the target job 7.

[0097] If the performance characteristics 9 of the program that executes the job 7 are not known (see the NO route in step S12), the process proceeds to step S16.

[0098] The node allocation information acquisition unit 143 acquires node allocation information. The node allocation information may include information on whether the job 7 has been assigned to all of the computation nodes 20. The control unit 110 determines whether there is an unused computation node 20 (step S16). If the job 7 has been assigned to all of the computation nodes 20 (see the NO route from step S16), the control unit 110 causes one computation node 20 to execute the job 7 as a one-node-using job (step S17). If there is an unused computation node 20 (see the YES route from step S16), the control unit 110 causes two computation nodes 20 to execute the job 7 as a two-node-using job (step S18).

[0099] In conjunction with the processing of steps S17 and S18, the execution time measurement unit 144 records the identification information of the program running the job 7, the number of nodes used, and the execution time (processing time) (step S19). The execution time measurement results by the execution time measurement unit 144 are stored in the execution time storage unit 124. The control unit 110 may calculate the performance characteristic 9 based on the execution time measurement results. Thereafter, the control unit 110 ends the processing.

[0100] [E] Effects of one embodiment According to a method according to one embodiment, when the control unit 110 causes the computing system 1 to execute a time slice, the control unit 110 acquires, for each job 7 submitted to the computing system 1, the number of set nodes set to be used for executing the job 7. The control unit 110 determines whether the number of set nodes is equal to or less than a threshold. The control unit 110 increases the number of computing nodes used for execution of at least some of the jobs 7 whose number of set nodes is equal to or less than the threshold, within the total number of computing nodes 20 of the computing system 1.

[0101] This allows the amount of data held by one computing node to be reduced by increasing the number of computing nodes that execute job 7, thereby preventing memory shortages even when time-slicing is performed.

[0102] The threshold value is half the total number of computing nodes 20, and in the process of increasing the number of computing nodes within the range of the total number of computing nodes, the number of computing nodes is doubled.

[0103] This allows job 7, which fits within half the system size, to be executed on twice the number of nodes, thereby maintaining a state in which time slice execution is always possible.

[0104] The process of increasing the number of computation nodes within the range of the total number of computation nodes 20 is executed by the control unit 110 when the number of set nodes is equal to or less than the threshold and is plural.

[0105] This makes it possible to suppress the occurrence of communication overhead caused by running job 7, which runs on one node, on two nodes.

[0106] The number of computing nodes is changed according to the performance characteristics 9 for each number of computing nodes of the target job 7.

[0107] This allows the number of nodes that execute the job 7 to be dynamically changed according to the performance characteristics 9 of the program, thereby maintaining an optimal situation.

[0108] [F] Notes The following additional notes are provided regarding the above-described embodiments.

[0109] (Appendix 1) A job management program for a system having a plurality of computing nodes, in which data of a plurality of jobs is distributed and stored in a memory of each computing node, and the plurality of jobs are switched at a time interval to be executed by at least some of the computing nodes, comprising: For each job submitted to the system, obtain the number of nodes set to be used for executing the job; determining whether the number of set nodes is equal to or less than a threshold; a job management program that causes a computer to execute a process of increasing the number of computing nodes used to execute at least some of a plurality of jobs for which the set number of nodes is equal to or less than the threshold, within the range of the total number of computing nodes in the system;

[0110] (Appendix 2) the threshold is half the total number of the computing nodes, In the process of increasing the number of computing nodes within the range of the total number of computing nodes, the number of computing nodes is doubled. 1. A job management program as described in Appendix 1.

[0111] (Appendix 3) The process of increasing the number of computing nodes within the range of the total number of computing nodes is executed by the computer when the number of set nodes is equal to or less than the threshold and is plural. 1. A job management program as described in Appendix 1.

[0112] (Appendix 4) changing the number of computing nodes according to performance characteristics of the target job for each number of computing nodes; 2. The job management program according to claim 1, which causes the computer to execute a process.

[0113] (Appendix 5) A job management method in a system having a plurality of computing nodes, in which data of a plurality of jobs is distributed and stored in a memory of each computing node, and the plurality of jobs are switched at a delimited time and executed by at least some of the plurality of computing nodes, comprising: For each job submitted to the system, obtain the number of nodes set to be used for executing the job; determining whether the number of set nodes is equal to or less than a threshold; A job management method in which a computer executes a process to increase the number of computing nodes used to execute at least some of a plurality of jobs for which the set node number is equal to or less than the threshold, within the range of the total number of computing nodes in the system.

[0114] (Appendix 6) the threshold is half the total number of the computing nodes, In the process of increasing the number of computing nodes within the range of the total number of computing nodes, the number of computing nodes is doubled. 6. A job management method as set forth in claim 5.

[0115] (Appendix 7) The process of increasing the number of computing nodes within the range of the total number of computing nodes is executed by the computer when the set number of nodes is equal to or less than the threshold and is plural. 6. A job management method as set forth in claim 5.

[0116] (Appendix 8) changing the number of computing nodes according to performance characteristics of the target job for each number of computing nodes; 6. The job management method according to claim 5, wherein the processing is executed by the computer.

[0117] (Appendix 9) In a system having a plurality of computing nodes, an information processing apparatus is provided which distributes and stores data of a plurality of jobs in a memory of each computing node, and switches the plurality of jobs at a delimited time interval to execute the jobs on at least some of the plurality of computing nodes, For each job submitted to the system, obtain the number of nodes set to be used for executing the job; determining whether the number of set nodes is equal to or less than a threshold; an information processing apparatus including a processor configured to increase the number of computing nodes used for execution of at least some of a plurality of jobs for which the set number of nodes is equal to or less than the threshold, within the range of the total number of computing nodes in the system;

[0118] (Appendix 10) the threshold is half the total number of the computing nodes, the processor doubles the number of computation nodes in the process of increasing the number of computation nodes within the range of the total number of computation nodes; 10. The information processing device according to claim 9.

[0119] (Appendix 11) the processor increases the number of computing nodes within the total number of computing nodes when the number of set nodes is equal to or less than the threshold and is a plurality of nodes; 10. The information processing device according to claim 9.

[0120] (Appendix 12) 10. The information processing device according to claim 9, wherein the processor changes the number of computing nodes according to performance characteristics for each number of computing nodes of a target job. [Explanation of symbols]

[0121] 1. Computing System 2. Calculation Server 3 Management Server 4 Login Server 4a,21,21-1,21-2,21a,21b,21c CPU 4b, 22, 22-1 to 22-3, 22b memory 4c,24 IF device 4d,28 Graphics Processing Unit 4e storage device 4f IO device 4g reader 4-hour program 4i recording media 4j bus 5 Console 6. Network devices 7 Jobs 8 areas 9 Performance characteristics 20, 20-0, 20-1 to 20-15, 20-n computing nodes 23 Memory Controller 24 IO device 25 Correspondence Table 31 Switching signal transmission unit 32 Job List 110 control section 130 Number of configured nodes acquisition unit 132 Time slice execution decision unit 134 Judgment section 140 1-node job processing unit 141 Memory usage acquisition unit 142 Performance information acquisition section 143 Node Allocation Information 144 Execution Time Measurement Unit 150 Increased 160 Output section

Claims

1. A job management program for a system having a plurality of computing nodes, in which data of a plurality of jobs is distributed and stored in a memory of each computing node, and the plurality of jobs are switched at a time interval to be executed by at least some of the computing nodes, comprising: For each job submitted to the system, obtain the number of nodes set to be used for executing the job; determining whether the number of set nodes is equal to or less than a threshold; increasing the number of computing nodes used to execute at least some of the jobs for which the set number of nodes is equal to or less than the threshold, within the range of the total number of computing nodes in the system; A job management program that causes a computer to execute a process.

2. the threshold is half the total number of the computing nodes, In the process of increasing the number of computing nodes within the range of the total number of computing nodes, the number of computing nodes is doubled. The job management program according to claim 1 .

3. The process of increasing the number of computing nodes within the range of the total number of computing nodes is executed by the computer when the number of set nodes is equal to or less than the threshold and is plural. The job management program according to claim 1 .

4. changing the number of computing nodes according to performance characteristics of the target job for each number of computing nodes; 2. The job management program according to claim 1, wherein the program causes the computer to execute a process.

5. 1. A job management method in a system having a plurality of computing nodes, in which data of a plurality of jobs is distributed and stored in a memory of each computing node, and the plurality of jobs are switched at a delimited time period to be executed by at least some of the computing nodes, For each job submitted to the system, obtain the number of nodes set to be used for executing the job; determining whether the number of set nodes is equal to or less than a threshold; increasing the number of computing nodes used to execute at least some of the jobs for which the set number of nodes is equal to or less than the threshold, within the range of the total number of computing nodes in the system; A job management method in which processing is executed by a computer.

6. In a system having a plurality of computing nodes, an information processing apparatus is provided which switches between a plurality of jobs at a time interval and executes the jobs on at least some of the computing nodes in a state where data of the jobs is distributed and held in a memory of each computing node, For each job submitted to the system, obtain the number of nodes set to be used for executing the job; determining whether the number of set nodes is equal to or less than a threshold; increasing the number of computing nodes used to execute at least some of the jobs for which the set number of nodes is equal to or less than the threshold, within the range of the total number of computing nodes in the system; An information processing device having a processor.

Images