Resource scheduling method, device, platform, scheduling system and computer program product

Abstract

This disclosure relates to the field of GPU resource management technology, and in particular to a resource scheduling method, apparatus, platform, scheduling system, and computer program product. By obtaining a user-submitted job task and a selected resource pool, wherein the resource pool includes at least one computing node; applying for computing resources in the resource pool based on the job task; and allocating the job task to the computing node corresponding to the computing resource for computation, it is possible to achieve unified access to different GPU resources.

Description

Technical Field

[0001] This disclosure relates to the field of GPU resource management technology, and in particular to a resource scheduling method, apparatus, platform, scheduling system and computer program product. Background Technology

[0002] Seismic processing system software is responsible for processing seismic data. With the increasing scale of seismic data and the growing computational demands of operational algorithms, the computing power of conventional CPUs is no longer sufficient for computationally intensive tasks such as pre-stack (post-stack) time migration, pre-stack (post-stack) depth migration, reverse-time migration, and full-waveform inversion. To address these issues, these geophysical algorithms have been developed into multiple GPU-based modules, significantly improving computational efficiency. With the rise of domestically produced GPUs, these modules have also been ported to domestic GPU platforms. However, how to achieve unified management of GPU resources from different manufacturers and a unified system for discovering, requesting, scheduling, and releasing GPU resources within the platform remains a pressing problem. Summary of the Invention

[0003] This disclosure provides a resource scheduling method, apparatus, platform, scheduling system, and computer program product that can achieve unified scheduling of different GPU resources.

[0004] Firstly, this disclosure provides a resource scheduling method, including:

[0005] Obtain the user-submitted job task and the selected resource pool, wherein the resource pool includes at least one computing node;

[0006] Based on the job task, request computing resources from the resource pool;

[0007] The job tasks are assigned to the computing nodes corresponding to the computing resources for computation.

[0008] In some embodiments, the method further includes:

[0009] Get environment variable configuration information;

[0010] The configuration information is distributed to each computing node. Each computing node has an independent installation directory for its dependency library files. When a computing node obtains the configuration information, it stores it in the installation directory and loads the installation directory when the operation starts.

[0011] In some embodiments, the method further includes:

[0012] Obtain resource information for each computing node;

[0013] Based on the resource information, each computing node is managed hierarchically.

[0014] In some embodiments, the hierarchical management of each computing node based on the resource information includes:

[0015] Output resource information for each computing node;

[0016] Obtain partition information for each resource and partition the computing nodes corresponding to each resource.

[0017] Resource pools are determined based on each partition;

[0018] The various resource pools are displayed for users to choose from.

[0019] In some embodiments, determining the resource pool based on each partition includes:

[0020] Output the resource information of the compute nodes corresponding to each partition;

[0021] Obtain the compute node selected by the user, and create a queue based on the selected compute node;

[0022] Each resource pool is determined based on each queue.

[0023] In some embodiments, the resource information includes: resource type, and the method further includes:

[0024] Unify the resource types of all computing nodes to the target type;

[0025] The output of resource information for each computing node includes:

[0026] Output the resource type of each computing node as the target type.

[0027] Secondly, this disclosure provides a resource scheduling device, comprising:

[0028] The acquisition module is used to acquire the job tasks submitted by the user and the selected resource pool, wherein the resource pool includes at least one computing node;

[0029] The computing module is used to request computing resources from the resource pool based on the job task;

[0030] The allocation module is used to allocate the job tasks to the computing nodes corresponding to the computing resources for computation.

[0031] Thirdly, this disclosure provides a scheduling platform, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described above.

[0032] Fourthly, this disclosure provides a scheduling system, which includes a scheduling platform and multiple computing nodes, wherein the scheduling platform is communicatively connected to each computing node.

[0033] Fifthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the methods described in the preceding aspects.

[0034] In a sixth aspect, this disclosure provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the methods described in the foregoing aspects.

[0035] This disclosure provides a resource scheduling method that obtains a user-submitted job task and a selected resource pool, wherein the resource pool includes at least one computing node; requests computing resources from the resource pool based on the job task; and allocates the job task to the computing node corresponding to the computing resource for computation, thereby enabling unified scheduling of different GPU resources. Attached Figure Description

[0036] The present disclosure will be described in more detail below based on embodiments and with reference to the accompanying drawings:

[0037] Figure 1 This is a flowchart illustrating a resource scheduling method provided in an embodiment of the present disclosure.

[0038] Figure 2 A partitioning diagram provided for an embodiment of this application;

[0039] Figure 3 A schematic diagram of a queue provided for an embodiment of this application;

[0040] Figure 4 A schematic diagram of a resource pool provided in an embodiment of this application;

[0041] Figure 5 A schematic diagram of a resource scheduling device provided in an embodiment of this application;

[0042] In the accompanying drawings, the same parts are referred to by the same reference numerals, and the drawings are not drawn to scale. Detailed Implementation

[0043] To enable those skilled in the art to better understand the technical solutions of this disclosure, and to fully understand and implement the process of how this disclosure applies technical means to solve technical problems and achieve corresponding technical effects, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, not all embodiments. The embodiments of this disclosure and the various features within them can be combined with each other without conflict, and the resulting technical solutions are all within the protection scope of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort should fall within the protection scope of this disclosure.

[0044] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0045] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0046] Example 1

[0047] The resource scheduling method provided in this embodiment can be applied to a scheduling platform. Figure 1 This is a flowchart illustrating a resource scheduling method provided in an embodiment of the present disclosure, as shown below. Figure 1 As shown, the resource scheduling method includes:

[0048] Step S101: Obtain the job task submitted by the user and the selected resource pool, wherein the resource pool includes at least one computing node.

[0049] In this embodiment, a job task typically refers to a specific computational or processing task that the user wants the system to perform. These tasks may include data analysis, model training, simulation, etc. Job tasks are usually submitted to the system by the user through a specific interface. Data analysis can be seismic data analysis, which may include: pre-stack (post-stack) time migration, pre-stack (post-stack) depth migration, reverse time migration, and full waveform inversion.

[0050] In this embodiment, a resource pool refers to a collection of computing resources in the system that can be used to execute job tasks. These resources typically include multiple computing nodes, each of which may contain a certain number of GPU cores, memory, storage space, etc. Resource pools are predefined. The scheduling platform can provide information on multiple resource pools to the user, who can then select the appropriate resource pool. Computing nodes can be GPU nodes.

[0051] In this embodiment, a compute node is the basic unit in the resource pool, used to execute job tasks. Each compute node typically contains one or more GPU cores, memory, network interfaces, etc., and may run a specific operating system and application.

[0052] In this embodiment, the user-submitted job task and selected resource pool can be obtained through a user interface (such as a web interface, command-line interface, etc.). The user submits the job task and selects the desired resource pool through the interface. The system receives the user's input and stores it in an internal data structure.

[0053] Step S102: Request computing resources from the resource pool based on the job task.

[0054] In this embodiment, computing resources refer to the hardware and software resources on a computing node that can be used to execute job tasks. These resources may include GPUs, memory space, disk I / O capabilities, etc.

[0055] In this embodiment, a match can be made between the job's requirements and available resources in the resource pool (such as idle computing nodes). If suitable resources are found, the system will allocate these resources to the job and prepare to run it on the corresponding computing node.

[0056] Step S103: Assign the job task to the computing node corresponding to the computing resource for computation.

[0057] In this embodiment, after resources are successfully allocated, the scheduling platform needs to send the job task to the corresponding computing node for execution. This typically involves transferring relevant data of the job task (such as input files, code, etc.) to the computing node and starting the job task on that node.

[0058] This disclosure provides a resource scheduling method that obtains a user-submitted job task and a selected resource pool, wherein the resource pool includes at least one computing node; requests computing resources from the resource pool based on the job task; and allocates the job task to the computing node corresponding to the computing resource for computation, thereby enabling unified scheduling of different GPU resources.

[0059] In some embodiments, the method further includes:

[0060] Obtain the configuration information of environment variables; distribute the configuration information to each computing node, wherein each computing node has an independent installation directory for dependency library files, and each computing node stores the configuration information in the installation directory when it obtains the configuration information, and loads the installation directory when the operation starts.

[0061] In this embodiment, environment variables are global values ​​in the operating system used to specify runtime environment parameters. These parameters can affect program behavior, such as path settings and memory allocation limits. In a distributed computing environment, environment variables are typically used to configure dependencies required by job tasks, library file paths, etc. Dependency library files are external libraries and modules required for program execution. These library files may contain functions, classes, interfaces, etc., for the program to call at runtime. In a distributed computing environment, each computing node may need to install different dependency library files to meet the needs of different job tasks.

[0062] In this embodiment, after obtaining the configuration information, the scheduling platform needs to distribute it to each computing node. Each computing node, upon receiving the configuration information, stores it in a separate dependency library file installation directory. This directory is typically pre-configured and stores all the dependency library files and configuration information required by the node. The purpose of storing the configuration information is to facilitate its loading and use when the operation starts.

[0063] In this embodiment, each computing node, upon receiving the configuration information, stores it in a separate dependency library file installation directory. This directory is typically pre-configured and stores all the dependency library files and configuration information required by the node. The purpose of storing the configuration information is to facilitate its loading and use during computation startup. At computation startup, the computing node loads the configuration information from its dependency library file installation directory. This typically involves setting environment variables and updating path settings. The purpose of loading the configuration information is to ensure that the job tasks can be executed in the correct runtime environment and correctly call the required dependency library files.

[0064] In this embodiment, runtime environment variables are configured and take effect on the scheduling platform. When the job runs, the scheduling platform distributes them uniformly to the computing nodes, avoiding the disaster of configuration file management caused by the increase of nodes.

[0065] In this embodiment, the dependency libraries of different GPU nodes are also inconsistent. Forcibly placing all the dependency library files of all GPU nodes in one folder can easily lead to dependency conflicts. To solve this problem, each GPU node is configured with a dependency library file (GPU_HOME), and the lib folders of different GPUs are placed under GPU_HOME. In this way, GPU_HOME is loaded when the job starts, and different GPU nodes will load their respective dependency libraries, thereby completing the decoupling and separation of the dependency environment of different GPU cards.

[0066] The method provided in this application embodiment enables unified management of different GPU resources.

[0067] In some embodiments, prior to step S101, the method further includes:

[0068] Step S1011: Obtain resource information for each computing node.

[0069] In this embodiment of the application, the resource information may include: card number, primary device number, secondary device number, busid, device devpath settings, resource type (paigpu), etc.

[0070] In this embodiment, each computing node is equipped with a GPU plugin, and each GPU plugin must implement the interface in a common plugin interface parent class. Through the implementation of the interface, the definition and registration of its own GPU plugin are completed, and the acquisition and reporting of the GPU information list are accomplished.

[0071] The pseudocode for the interface definition is as follows:

[0072]

[0073]

[0074] The getRegisterGPURequestInfo interface is responsible for registering and discovering each GPU, while the getGPUDevices interface is responsible for returning a list of GPU devices for each node.

[0075] The implementation details of these two interfaces are described below:

[0076] The getRegisterGPURequestInfo interface is relatively simple; you only need to set the GPU resource type name (paigpu) and the plugin version number.

[0077] The implementation principle of getGPUDevices is as follows:

[0078] Obtain device information using the command-line management tools provided by various GPU manufacturers (such as ixsmi from Tianshu Zhixing). The complete command line is as follows: ixsmi

[0079] The following information was obtained using the query-gpu-index,pci.bus_id--format-csv,noheader:

[0080] 0,00000000:3B:00.0;

[0081] 1,00000000:AF:00.0;

[0082] Where 0 and 1 represent the card number, and the following 00000000:3B:00.0 and 00000000:AF:00.0 represent the GPU card's busid.

[0083] Obtain the major device number and minor device number for each device driver, which are then used to identify and manage different devices. Major Device Number: The major device number identifies the device driver. Each major device number corresponds to a specific driver responsible for communicating with the device. When the kernel receives a request for a device, it uses the major device number to find the corresponding driver and handle the request. Minor Device Number: The minor device number distinguishes different instances or functions under the same major device. For example, a major device may have multiple minor devices. Minor device numbers are typically used to represent multiple devices of the same type, or to indicate different functions or interfaces of a device. For example, multiple GPU cards may share the same major device number, but each card has a different minor device number.

[0084] The major device number can be obtained using the following command: stat -c %t:%T devpath( / dev / dri / xxx), where xxx represents the device number of different devices. The return value is a hexadecimal major device number, which needs to be converted to decimal before use.

[0085] The value of the secondary device number is the card number obtained in the first step.

[0086] This completes the process of obtaining the device ID information. Finally, simply input the card number, major device ID, minor device ID, busid, device devpath, and resource type (paigpu) into the device's initial information to complete the GPU card registration.

[0087] In this embodiment of the application, each computing node can send relevant resource information to the scheduling platform, thereby enabling the scheduling platform to obtain the resource information of the computing nodes.

[0088] In this embodiment of the application, the scheduling platform can uniformly discover resource information.

[0089] Step S1012: Perform hierarchical management of each computing node based on the resource information.

[0090] In this embodiment, resource information of each computing node can be output; partition information for each resource information can be obtained, and the computing nodes corresponding to each resource information can be partitioned; resource pools can be determined based on each partition; and each resource pool can be output for the user to select.

[0091] In this embodiment, for the platform, a cluster may contain multiple types of GPU cards. However, in terms of hardware compatibility and driver software stability, currently only one type of GPU card will exist in a single computing node. Therefore, the cluster will contain N nodes with GPU cards from vendor A, and M nodes with GPU cards from vendor B.

[0092] In this embodiment, a three-tiered management mechanism of partitions, queues, and resource pools is adopted. Each partition displays the hostname, memory, CPU, and GPU information (excluding vendor information) of each node. The number of nodes included in a partition can be freely selected.

[0093] In some embodiments, the resource information of the computing nodes corresponding to each partition is output; the computing nodes selected by the user are obtained, and queues are created based on the computing nodes selected by the user; and each resource pool is determined based on each queue.

[0094] In this embodiment, the administrator can select different partitions to create queues. Queues can be further subdivided into resource pools. The resource pool information includes the queue name and the number of GPUs contained in the queue. When a user submits a job, they select a different resource pool to complete the job.

[0095] The method provided in this application embodiment enables unified management of different GPU resources.

[0096] In this embodiment of the application, when a job is submitted, it will apply for computing resources (memory size, number of CPU cores, number of GPU cards). When a job is submitted, it will uniformly apply for GPU resources as the resource type of the GPU.

[0097] In some embodiments, the resource information includes: resource type, and the method further includes:

[0098] Unify the resource types of all computing nodes to the target type;

[0099] For example, different resource types can be unified as paigpu resources.

[0100] The method provided in this application embodiment automatically identifies different types of GPU resources as GPU resources, thus achieving seamless integration at the user level.

[0101] Example 2

[0102] Based on the above embodiments, this application provides another resource scheduling method. This method is applied to a GPU resource scheduling system. The GPU resource scheduling system will automatically identify different types of GPU resources as GPU resources, so that it is imperceptible to the user. The user can choose to submit modules of different types of GPUs, thereby realizing the unified use of different GPU resources in the same cluster.

[0103] In this embodiment, each GPU plugin implements a unified GPU scheduling plugin interface, and each plugin discovers and registers GPU resource information uniformly on each node. In the resource management and job management of the processing system, these GPU resources are managed uniformly. During the job submission stage, pigpu resources are uniformly requested as GPU resources for use by the GPU module.

[0104] In this embodiment, GPU plugins from different manufacturers must implement the interface in a common plugin interface parent class. Through this interface implementation, they can define and register their own GPU plugins, and obtain and report GPU information lists.

[0105] The pseudocode for the interface definition is as follows:

[0106]

[0107] The getRegisterGPURequestInfo interface is responsible for registering and discovering each GPU, while the getGPUDevices interface is responsible for returning a list of GPU devices for each node.

[0108] The implementation details of these two interfaces are described below:

[0109] The getRegisterGPURequestInfo interface is relatively simple; you only need to set the GPU resource type name (paigpu) and the plugin version number.

[0110] The implementation principle of getGPUDevices is as follows:

[0111] 1. Obtain device information using the command-line management tools provided by various GPU manufacturers (such as ixsmi from Tianshu Zhixing). The complete command line is as follows: ixsmi

[0112] The following information was obtained using the query-gpu-index,pci.bus_id--format-csv,noheader:

[0113] 0,00000000:3B:00.0;

[0114] 1,00000000:AF:00.0;

[0115] Where 0 and 1 represent the card number, and the following 00000000:3B:00.0 and 00000000:AF:00.0 represent the GPU card's busid.

[0116] 2. Obtain the major device number and minor device number of each device driver, which will be used to identify and manage different devices later.

[0117] Major Device Number: The major device number is used to identify a device driver. Each major device number corresponds to a specific driver, which is responsible for communicating with the device. When the kernel receives a request for a device, it uses the major device number to find the corresponding driver and then handles the request.

[0118] Minor Device Number: The minor device number is used to distinguish different instances or functions under the same major device. For example, a major device may have multiple minor devices. Minor device numbers are typically used to represent multiple devices of the same type, or to indicate different functions or interfaces of a device. For example, multiple GPU cards may share the same major device number, but each card may have a different minor device number.

[0119] The major device number can be obtained using the following command: stat -c %t:%T devpath( / dev / dri / xxx), where xxx represents the device number of different devices. The return value is a hexadecimal major device number, which needs to be converted to decimal before use.

[0120] The value of the secondary device number is the card number obtained in the first step.

[0121] This completes the process of obtaining the device ID information. Finally, simply input the card number, major device ID, minor device ID, busid, device devpath, and resource type (paigpu) into the device's initial information to complete the GPU card registration.

[0122] When the cluster service starts, each node will start a resource collection process. The process will load GPU information, and the information collected by each GPU node will eventually be reported to the scheduling platform for unified resource management.

[0123] For the scheduling platform, a cluster may contain multiple types of GPUs, but in terms of hardware compatibility and driver software stability, currently only one type of GPU will exist in a single node. Therefore, the cluster will contain N nodes with GPUs from vendor A and M nodes with GPUs from vendor B.

[0124] During the platform's resource management process, the processing system establishes GPU queues for users to submit jobs. The π platform's processing system adopts a three-level management mechanism of partitions, queues, and resource pools. In the partitions, as shown in the figure below, the hostname of each node, as well as the node's memory, CPU, and GPU information (excluding vendor information) are displayed. Figure 2 A partitioning diagram provided for an embodiment of this application, such as Figure 2 As shown, the number of nodes a partition can contain can be freely chosen. After the partition selection is complete, a queue can be created. Figure 3 A schematic diagram of a queue provided in an embodiment of this application, such as... Figure 3 As shown, the administrator selects different partitions to create queues. The NVIDIA GPU queue selects the nvgpu partition, while the day-based GPU queue selects the tsgpu partition. Jobs run on the queues, and the partitions isolate the nodes between queues. Therefore, a module generated based on the day-based development environment will not run on a node containing NVIDIA GPUs, leading to an error.

[0125] When users submit jobs, they need to submit them to the resource pool. Figure 4 This is a schematic diagram of a resource pool provided in an embodiment of this application, such as... Figure 4 As shown, the resource pool information includes the queue name and the number of GPUs in the queue. When submitting a job, users select different resource pools to complete the job. Modules generated based on NVIDIA GPUs are submitted to NVIDIA's GPU resource pool, while modules generated based on domestically produced GPUs are submitted to their respective GPU resource pools.

[0126] When a job is submitted, it will request computing resources (memory size, number of CPU cores, number of GPU cards). When a job is submitted, it will request paigpu resources as the GPU resource type.

[0127] In this embodiment, after the job is submitted, the computing process will be started on different GPU nodes to perform the module's computation. Runtime environment variables are only configured on the master node and are uniformly distributed to the computing nodes by the master node during job execution, avoiding the configuration file management disaster that can occur as the number of nodes increases.

[0128] Different GPU nodes have different GPU dependency libraries, and forcibly putting all GPU dependency library files into a single folder can easily lead to dependency conflicts. To solve this problem, each GPU node is configured with an environment variable GPU_HOME, and the lib folders for different GPUs are placed under GPU_HOME. This way, GPU_HOME is loaded when the job starts, and different GPU nodes will load their respective dependency libraries, thus decoupling and separating the dependency environments of different GPU cards.

[0129] The method provided in this application embodiment automatically identifies different types of GPU resources as GPU resources, thus achieving seamless user experience. Users can choose to submit modules of different types of GPUs, thereby enabling the unified use of different GPU resources within the same cluster.

[0130] Example 3

[0131] This application provides a resource scheduling device. Figure 5 This is a schematic diagram of a resource scheduling device provided in an embodiment of this application, such as... Figure 5 As shown, it includes:

[0132] The acquisition module is used to acquire the job tasks submitted by the user and the selected resource pool, wherein the resource pool includes at least one computing node;

[0133] The computing module is used to request computing resources from the resource pool based on the job task;

[0134] The allocation module is used to allocate the job tasks to the computing nodes corresponding to the computing resources for computation.

[0135] In some embodiments, the resource scheduling device is further configured to:

[0136] Get environment variable configuration information;

[0137] The configuration information is distributed to each computing node. Each computing node has an independent installation directory for its dependency library files. When a computing node obtains the configuration information, it stores it in the installation directory and loads the installation directory when the operation starts.

[0138] In some embodiments, the resource scheduling device is further configured to:

[0139] Obtain resource information for each computing node;

[0140] Based on the resource information, each computing node is managed hierarchically.

[0141] In some embodiments, the hierarchical management of each computing node based on the resource information includes:

[0142] Output resource information for each computing node;

[0143] Obtain partition information for each resource and partition the computing nodes corresponding to each resource.

[0144] Resource pools are determined based on each partition;

[0145] The various resource pools are displayed for users to choose from.

[0146] In some embodiments, determining the resource pool based on each partition includes:

[0147] Output the resource information of the compute nodes corresponding to each partition;

[0148] Obtain the compute node selected by the user, and create a queue based on the selected compute node;

[0149] Each resource pool is determined based on each queue.

[0150] In some embodiments, the resource information includes: resource type, and the resource scheduling device is further configured to:

[0151] Unify the resource types of all computing nodes to the target type;

[0152] The output of resource information for each computing node includes:

[0153] Output the resource type of each computing node as the target type.

[0154] Example 4

[0155] Based on the above embodiments, this embodiment provides a scheduling platform, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the above embodiments.

[0156] Example 5

[0157] Based on the above embodiments, this embodiment provides a scheduling system, which includes a scheduling platform and multiple computing nodes, wherein the scheduling platform is communicatively connected to each computing node.

[0158] In some embodiments of this example, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the method described in the above embodiments.

[0159] In some embodiments of this example, a computer program product is provided, including a computer program / instructions, which, when executed by a processor, implements the steps of the method described in the above embodiments.

[0160] The processor may include, but is not limited to, one or more processors or microprocessors. Each processor may be implemented as an Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic component, for executing the methods in the above embodiments.

[0161] Computer-readable storage media can be implemented by any type of volatile or non-volatile storage device or a combination thereof. Computer-readable storage media may include, but are not limited to, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, and computer storage media (e.g., hard disks, floppy disks, solid-state drives, removable disks, CD-ROMs, DVD-ROMs, Blu-ray discs, etc.).

[0162] Computer-readable storage media may also store at least one computer-executable program / instruction, such as computer-readable instructions. Computer-readable storage media include, but are not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Computer-readable storage media may include, for example, read-only memory (ROM), hard disk, flash memory, etc. For example, a non-transitory computer-readable storage medium may be connected to a computing device such as a computer, and then, when the computing device executes the computer-readable instructions stored on the computer-readable storage medium, the various methods described above can be performed.

[0163] In addition, the computer device may include (but is not limited to) a data bus, an input / output (I / O) bus, a display, and input / output devices (e.g., keyboard, mouse, speakers, etc.).

[0164] The processor can communicate with external devices via the I / O bus through wired or wireless networks.

[0165] In one embodiment, the at least one computer-executable instruction may also be compiled into or comprise a software product / computer program product, wherein one or more computer-executable instructions are executed by a processor to perform the steps of the various functions and / or methods in the embodiments described herein.

[0166] In the embodiments provided in this disclosure, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0167] It should be noted that, in this disclosure, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element limited by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0168] While the embodiments disclosed herein are as described above, the foregoing content is merely for the purpose of facilitating understanding of this disclosure and is not intended to limit this disclosure. Any person skilled in the art to which this disclosure pertains may make any modifications and changes in form and detail of the implementation without departing from the spirit and scope of this disclosure; however, the scope of patent protection of this disclosure shall still be determined by the scope defined in the appended claims.

Claims

1. A resource scheduling method, characterized in that, include: Obtain the user-submitted job task and the selected resource pool, wherein the resource pool includes at least one computing node; Based on the job task, request computing resources from the resource pool; The job tasks are assigned to the computing nodes corresponding to the computing resources for computation.

2. The method according to claim 1, characterized in that, The method further includes: Get environment variable configuration information; The configuration information is distributed to each computing node. Each computing node has an independent installation directory for its dependency library files. When a computing node obtains the configuration information, it stores it in the installation directory and loads the installation directory when the operation starts.

3. The method according to claim 1, characterized in that, The method further includes: Obtain resource information for each computing node; Based on the resource information, each computing node is managed hierarchically.

4. The method according to claim 3, characterized in that, The hierarchical management of each computing node based on the resource information includes: Output resource information for each computing node; Obtain partition information for each resource and partition the computing nodes corresponding to each resource. Resource pools are determined based on each partition; The various resource pools are displayed for users to choose from.

5. The method according to claim 4, characterized in that, The determination of resource pools based on each partition includes: Output the resource information of the compute nodes corresponding to each partition; Obtain the compute node selected by the user, and create a queue based on the selected compute node; Each resource pool is determined based on each queue.

6. The method according to claim 3, characterized in that, The resource information includes: resource type, and the method further includes: Unify the resource types of all computing nodes to the target type; The output of resource information for each computing node includes: Output the resource type of each computing node as the target type.

7. A resource scheduling device, characterized in that, include: The acquisition module is used to acquire the job tasks submitted by the user and the selected resource pool, wherein the resource pool includes at least one computing node; The computing module is used to request computing resources from the resource pool based on the job task; The allocation module is used to allocate the job tasks to the computing nodes corresponding to the computing resources for computation.

8. A scheduling platform, characterized in that, A memory, a processor, and a computer program stored on the memory, characterized in that the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A scheduling system, characterized in that, The scheduling system includes: the scheduling platform as described in claim 7 and multiple computing nodes, wherein the scheduling platform is communicatively connected to each computing node.

10. A computer program product comprising a computer program / instructions, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 6.

Images