A heterogeneous computing power scheduling method, device and storage medium

Abstract

The embodiment of the present invention provides a heterogeneous computing power scheduling method, device and storage medium, the method comprising: in response to a heterogeneous computing power scheduling task sent by a user, classifying the heterogeneous computing power resources of the scheduling system according to the applicable scenarios and computing power resource types of the computing power resources, and obtaining resource adaptation scenarios and resource adaptation manufacturers; dividing the heterogeneous computing power resources and the heterogeneous computing power scheduling tasks into resource pools and task pools respectively; constructing a computing power scheduling objective function; solving the computing power scheduling objective function to obtain a resource allocation result; monitoring the task pool, obtaining the current task from the task polling queue, and allocating the computing power resources in the resource pool to the current task according to the resource allocation result, the resource adaptation scenario and the resource adaptation manufacturer.

Description

Technical Field

[0001] The present invention relates to the field of computing power scheduling technology, and in particular to, but not limited to, a heterogeneous computing power scheduling method, device, and storage medium. Background Art

[0002] In recent years, with the rapid development of artificial intelligence technology, people's demand for heterogeneous computing power has also increased sharply, and heterogeneous computing power scheduling has become an increasingly important topic. The unified management and scheduling of various computing resources face many challenges. Different types of computing resources have different performance and power consumption characteristics. How to reasonably allocate and schedule them according to task requirements is a complex issue. Secondly, heterogeneous computing power scheduling needs to support flexible task and resource orchestration capabilities to cope with complex and changing application scenarios.

[0003] Therefore, building a comprehensive heterogeneous computing resource management and scheduling method is of great significance for improving computing performance, reducing resource waste, and promoting digital transformation. Summary of the invention

[0004] In order to solve the above-mentioned technical problems existing in the prior art, the present invention provides a heterogeneous computing power scheduling method, device and storage medium.

[0005] The technical method of the embodiment of the present invention is implemented as follows:

[0006] In a first aspect, an embodiment of the present invention provides a heterogeneous computing power scheduling method, which is applied to a heterogeneous computing power scheduling system, wherein the system includes a heterogeneous computing power management platform and a heterogeneous computing power scheduling platform; the method includes:

[0007] The heterogeneous computing power management platform responds to the heterogeneous computing power scheduling task sent by the user, classifies the heterogeneous computing power resources of the scheduling system according to the applicable scenarios and computing power resource types of the computing power resources, obtains resource adaptation scenarios and resource adaptation manufacturers, and sends the heterogeneous computing power scheduling task and the heterogeneous computing power resources to the heterogeneous computing power scheduling platform;

[0008] The heterogeneous computing power scheduling platform receives the heterogeneous computing power resources and the heterogeneous computing power scheduling tasks sent by the heterogeneous computing power management platform, and divides the heterogeneous computing power resources and the heterogeneous computing power scheduling tasks into a resource pool and a task pool respectively;

[0009] The heterogeneous computing power scheduling platform constructs a computing power scheduling objective function; the computing power scheduling objective function is expressed as: , the constraint condition of the computing power scheduling objective function is expressed as: , ;

[0010] in, Indicates the number of resources required for the current task; represents the number of resources in the resource pool. If the current task is an inference task, it represents the inference resource pool. If the current task is a training task, it represents the training resource pool. Indicates the video memory of the jth resource in the resource pool; Indicates that the jth resource in the resource pool has occupied the video memory; Indicates the video memory usage of the i-th resource requested by the current task; Indicates whether the i-th resource requested by the current task is allocated to the j-th device in the resource pool; represents the allocation weight of the jth resource in the resource pool;

[0011] The heterogeneous computing power scheduling platform solves the computing power scheduling objective function to obtain a resource allocation result;

[0012] The heterogeneous computing power scheduling platform monitors the task pool, obtains the current task from the task polling queue, and allocates the computing power resources in the resource pool to the current task according to the resource allocation result, the resource adaptation scenario and the resource adaptation manufacturer.

[0013] In some embodiments, the resource pool includes: a training resource pool, an inference resource pool, a test resource pool and a cache pool; the training resource pool stores computing power equipment used for training; the inference resource pool stores computing power equipment used for inference; the test resource pool stores a small amount of computing power equipment for trial operation; the cache pool is used to recycle idle computing power resources in the training resource pool and the inference resource pool according to preset rules.

[0014] In some embodiments, the test resource pool is also used to allocate corresponding computing resources according to user needs and test-run tasks to obtain operating parameters; the operating parameters include: video memory occupancy rate and model throughput.

[0015] In some embodiments, the method further includes: when the resource adaptation manufacturer does not exist in the resource allocation result, it indicates that the resources are insufficient, the computing power resources in the cache pool are added to the resource pool, and the resource allocation result is re-solved.

[0016] In some embodiments, the method also includes: monitoring the resource status of the resource pool in real time to obtain resource idle time and resource retention time; determining resource utilization based on the resource idle time and resource retention time; when the resource utilization does not exceed a preset utilization threshold, recycling the idle computing resources to the heterogeneous computing management platform.

[0017] In some embodiments, after allocating the heterogeneous computing resources to the current task according to the resource allocation result, the method further includes: updating the resource status of the computing resources allocated in the resource pool to allocated resources, and updating the task status in the task pool to executed.

[0018] In a second aspect, an embodiment of the present invention provides an electronic device, comprising: a memory for storing executable instructions; and a processor for implementing the above-mentioned heterogeneous computing power scheduling method when executing the executable instructions stored in the memory.

[0019] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium that stores executable instructions for causing a processor to execute the executable instructions to implement the above-mentioned heterogeneous computing power scheduling method.

[0020] The heterogeneous computing power scheduling method, device and storage medium provided by the embodiment of the present invention include: responding to the heterogeneous computing power scheduling task sent by the user, classifying the heterogeneous computing power resources of the scheduling system according to the applicable scenario and computing power resource type of the computing power resource, and obtaining the resource adaptation scenario and resource adaptation manufacturer; dividing the heterogeneous computing power resources and the heterogeneous computing power scheduling tasks into a resource pool and a task pool respectively; constructing a computing power scheduling objective function; solving the computing power scheduling objective function to obtain a resource allocation result; monitoring the task pool, obtaining the current task from the task polling queue, and allocating the computing power resources in the resource pool to the current task according to the resource allocation result, resource adaptation scenario and resource adaptation manufacturer. In this way, the heterogeneous computing power scheduling method provided by the present invention has a more flexible resource scheduling capability, can achieve better computing power resource utilization effect by maximizing resource utilization and resource recovery mechanism, and can also reduce resource waste and power consumption waste to a certain extent, and improve computing power resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a flowchart of a heterogeneous computing power scheduling method provided by an embodiment of the present invention;

[0022] Figure 2 It is a schematic diagram of the composition structure of the heterogeneous computing power scheduling device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present invention. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present invention.

[0024] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. Unless otherwise defined, all technical and scientific terms used in the embodiments of the present invention have the same meaning as those commonly understood by those skilled in the art to which the embodiments of the present invention pertain. The terms used in the embodiments of the present invention are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0025] The following describes an exemplary application of a heterogeneous computing power scheduling device according to an embodiment of the present invention. The heterogeneous computing power scheduling device provided by the embodiment of the present invention can be implemented as a terminal or a server. In one implementation, the heterogeneous computing power scheduling device provided by the embodiment of the present invention can be implemented as various types of terminals such as laptops, tablet computers, desktop computers, mobile devices, etc.; in another implementation, the heterogeneous computing power scheduling device provided by the embodiment of the present invention can also be implemented as a server, wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery Network), and big data and artificial intelligence platforms. The terminal and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiment of the present invention. The following describes an exemplary application of the heterogeneous computing power scheduling device when it is implemented as a server.

[0026] The embodiment of the present invention provides a heterogeneous computing power scheduling method, see Figure 1 , Figure 1 is a flow chart of the heterogeneous computing power scheduling method provided by an embodiment of the present invention, which will be combined with Figure 1 The steps shown are explained.

[0027] Step S110, the heterogeneous computing power management platform responds to the heterogeneous computing power scheduling task sent by the user, classifies the heterogeneous computing power resources of the scheduling system according to the applicable scenarios and computing power resource types of the computing power resources, obtains resource adaptation scenarios and resource adaptation manufacturers, and sends the heterogeneous computing power scheduling task and the heterogeneous computing power resources to the heterogeneous computing power scheduling platform.

[0028] In some embodiments, the heterogeneous computing power management platform is mainly used to receive users' heterogeneous computing power scheduling tasks, classify the heterogeneous computing power resources in the system according to the applicable scenarios and types of computing power resources, and send the task and resource information to the heterogeneous computing power scheduling platform.

[0029] In some embodiments, the heterogeneous computing power scheduling platform is used to receive heterogeneous computing power resources and heterogeneous computing power scheduling tasks from the heterogeneous computing power management platform, divide them into resource pools and task pools respectively, and is responsible for monitoring the task pools, allocating resources through a preset scheduling algorithm, and constructing the constraints of the algorithm to solve the resource allocation results.

[0030] In some embodiments, heterogeneous computing power resources are allocated to the current task based on the resource allocation results, resource adaptation scenarios and resource adaptation manufacturers solved by the heterogeneous computing power scheduling platform.

[0031] In step S120, the heterogeneous computing power scheduling platform receives the heterogeneous computing power resources and the heterogeneous computing power scheduling tasks sent by the heterogeneous computing power management platform, and divides the heterogeneous computing power resources and the heterogeneous computing power scheduling tasks into a resource pool and a task pool respectively.

[0032] In some embodiments, heterogeneous computing resources refer to computing power resources of different types, different manufacturers, and different architectures, such as CPU, GPU, FPGA, etc.

[0033] In some embodiments, resource adaptation scenarios refer to categories of heterogeneous computing resources classified according to applicable scenarios of computing resources, such as image recognition scenarios, data processing scenarios, etc.

[0034] In some embodiments, the resource adaptation manufacturer refers to the category of heterogeneous computing resources classified according to the computing resource type, such as computing resources from different manufacturers such as NVIDIA and AMD.

[0035] Step S130: The heterogeneous computing power scheduling platform constructs a computing power scheduling objective function; the computing power scheduling objective function is expressed as: , the constraint condition of the computing power scheduling objective function is expressed as: , .

[0036] in, Indicates the number of resources required for the current task; represents the number of resources in the resource pool. If the current task is an inference task, it represents the inference resource pool. If the current task is a training task, it represents the training resource pool. Indicates the video memory of the jth resource in the resource pool; Indicates that the jth resource in the resource pool has occupied the video memory; Indicates the video memory usage of the i-th resource requested by the current task; Indicates whether the i-th resource requested by the current task is allocated to the j-th device in the resource pool; Represents the allocation weight of the jth resource in the resource pool.

[0037] Step S140: the heterogeneous computing power scheduling platform solves the computing power scheduling objective function to obtain a resource allocation result.

[0038] Step S150, the heterogeneous computing power scheduling platform monitors the task pool, obtains the current task from the task polling queue, and allocates the computing power resources in the resource pool to the current task according to the resource allocation result, the resource adaptation scenario and the resource adaptation manufacturer.

[0039] The heterogeneous computing power scheduling method, device and storage medium provided by the embodiment of the present invention include: responding to the heterogeneous computing power scheduling task sent by the user, classifying the heterogeneous computing power resources of the scheduling system according to the applicable scenario and computing power resource type of the computing power resource, and obtaining the resource adaptation scenario and resource adaptation manufacturer; dividing the heterogeneous computing power resources and the heterogeneous computing power scheduling tasks into a resource pool and a task pool respectively; constructing a computing power scheduling objective function; solving the computing power scheduling objective function to obtain a resource allocation result; monitoring the task pool, obtaining the current task from the task polling queue, and allocating the computing power resources in the resource pool to the current task according to the resource allocation result, resource adaptation scenario and resource adaptation manufacturer. In this way, the heterogeneous computing power scheduling method provided by the present invention has a more flexible resource scheduling capability, can achieve better computing power resource utilization effect by maximizing resource utilization and resource recovery mechanism, and can also reduce resource waste and power consumption waste to a certain extent, and improve computing power resource utilization.

[0040] In some embodiments, the resource pool includes: a training resource pool, an inference resource pool, a test resource pool and a cache pool; the training resource pool stores computing power equipment used for training; the inference resource pool stores computing power equipment used for inference; the test resource pool stores a small amount of computing power equipment for trial operation; the cache pool is used to recycle idle computing power resources in the training resource pool and the inference resource pool according to preset rules.

[0041] In some embodiments, the test resource pool is also used to allocate corresponding computing resources according to user needs and test-run tasks to obtain operating parameters; the operating parameters include: video memory occupancy rate and model throughput.

[0042] In some embodiments, the method further includes: when the resource adaptation manufacturer does not exist in the resource allocation result, it indicates that the resources are insufficient, the computing power resources in the cache pool are added to the resource pool, and the resource allocation result is re-solved.

[0043] In some embodiments, the method also includes: monitoring the resource status of the resource pool in real time to obtain resource idle time and resource retention time; determining resource utilization based on the resource idle time and resource retention time; when the resource utilization does not exceed a preset utilization threshold, recycling the idle computing resources to the heterogeneous computing management platform.

[0044] Here, resource utilization can be expressed as: ,in, Indicates the idle time of the resource. Indicates the resource retention time.

[0045] In some embodiments, after allocating the heterogeneous computing resources to the current task according to the resource allocation result, the method further includes: updating the resource status of the computing resources allocated in the resource pool to allocated resources, and updating the task status in the task pool to executed.

[0046] It should be noted that when a user sends a heterogeneous computing power scheduling task, the heterogeneous computing power management platform starts working first. It will classify the heterogeneous computing power resources in the system according to the pre-set computing power resource application scenarios and computing power resource types. For example, according to different usage scenarios, computing power resources suitable for different scenarios such as high-performance computing, machine learning training, and deep learning reasoning are separated; according to different types, resources from different manufacturers and different architectures are distinguished to obtain resource adaptation scenarios and resource adaptation manufacturers. Then the heterogeneous computing power scheduling tasks and classified heterogeneous computing power resources are sent to the heterogeneous computing power scheduling platform to prepare for subsequent resource allocation.

[0047] The heterogeneous computing power scheduling platform receives heterogeneous computing power resources and heterogeneous computing power scheduling tasks from the heterogeneous computing power management platform, and divides them into resource pools and task pools. The resource pool stores various heterogeneous computing power resources, and the task pool stores user tasks.

[0048] Monitoring and scheduling phase: Continuously monitor the task pool. When there are tasks to be processed, use the preset scheduling algorithm to allocate computing resources in the resource pool to heterogeneous computing scheduling tasks. When using the preset scheduling algorithm, factors such as the video memory status of different resources (occupied and unoccupied), the video memory requirements of tasks for resources, and the allocation weight of resources are considered, and the optimal allocation plan is calculated through a formula.

[0049] Build constraints:

[0050] for This constraint, when calculating allocations, checks whether each resource allocation will cause the usage of that resource to exceed 98%, to avoid performance degradation or system crashes due to over-allocation.

[0051] for This constraint ensures that the number of resources allocated is exactly what the task requires, no more and no less.

[0052] Solve the resource allocation results: Based on the above scheduling algorithm and constraints, use mathematical methods (such as linear programming, etc.) to solve the resource allocation results, and obtain information such as which resource pools should be allocated to each task and the allocated quantity.

[0053] The heterogeneous computing resource scheduling platform accurately allocates heterogeneous computing resources to the current task based on the resource allocation results solved by the heterogeneous computing scheduling platform and the resource adaptation scenarios and resource adaptation manufacturer information classified by the heterogeneous computing management platform. For example, according to the requirements of the task, computing resources suitable for specific manufacturers and specific scenarios are allocated to the task to ensure that the task can run in a suitable environment, while taking into account the heterogeneity of resources to ensure efficient use of resources.

[0054] The following describes an exemplary application of an embodiment of the present invention in a practical application scenario.

[0055] Step 1: Build a heterogeneous computing power management platform.

[0056] The platform implements the following functions: receiving tasks submitted by users; managing heterogeneous computing resources.

[0057] Step 2: The management platform classifies computing resources according to computing resource manufacturers and applicable scenarios, such as:

[0058] Manufacturers: NVIDIA, Ascend, etc.; Applicable scenarios: inference, training.

[0059] Also, record device parameter information, including: device name (such as A100, 910p, 310p, etc.); video memory; computing performance (FLOPS);

[0060] Step 3: The management platform collects user tasks.

[0061] It mainly includes the following information: task type (inference or training); some necessary information such as the type of resource applied for (manufacturer, device name); and the number of resources applied for (for example, 1 A100).

[0062] Step 4: Build a heterogeneous computing power scheduling platform.

[0063] The scheduling platform receives the heterogeneous computing resources and tasks pushed by the management platform, and divides the heterogeneous computing resources and tasks into:

[0064] Test resource pool: stores a small amount of computing power equipment for trial operation;

[0065] Training resource pool: stores computing devices used for training;

[0066] Reasoning resource pool: stores computing devices used for reasoning;

[0067] Cache pool: Idle resources in the training / inference resource pool will be recycled to the cache pool according to certain rules;

[0068] Task pool: stores tasks submitted by users;

[0069] Test resource pool: allocates corresponding computing resources according to user needs, and obtains running parameters by trial running tasks, as follows: video memory usage; model throughput: the number of samples that the model can train / infer per second, used to describe model performance; users are required to set batchSize to 1, and this parameter is obtained after the model is trial run for a certain period of time;

[0070] Training / inference resource pool: records heterogeneous computing resource usage information, such as: video memory usage; whether tasks have been assigned;

[0071] Resource Idle Time: The accumulated idle time of a computing device in the resource pool;

[0072] Resource retention time: the cumulative time a computing device is stored in the resource pool;

[0073] Priority: ; (decimal between 0-1, such as A100=0.9, P4=0.5);

[0074] Task pool: records user task information and execution status, as follows:

[0075] Task type (training or inference); name of the device applied for; running status: executed or not executed; actual video memory usage; number of occupied resources; throughput: obtained from the test pool.

[0076] Step 5: The scheduling platform monitors the task pool and allocates computing resources in the training / inference resource pool to tasks through the scheduling algorithm.

[0077] Optimization goal: (According to the user-specified resources, maximize resource utilization, and solve the resource allocation results ).

[0078] Constraints: (Video memory usage is less than or equal to 0.98, this value is set by the scheduling platform between 0-1).

[0079] Constraints: (The solution result must be equal to the number of resources requested by the user).

[0080] The variables have the following meanings: Indicates the number of resources required for the current task; The number of resources in the resource pool is represented by . If it is an inference task, it represents the inference resource pool; if it is a training task, it represents the training resource pool; Indicates the video memory of the jth resource in the resource pool; Indicates that the jth resource in the resource pool has occupied the video memory; Indicates the video memory usage of the i-th resource requested by the current task; (obtained from the test pool) Indicates whether the i-th resource requested by the current task is allocated to the j-th device in the resource pool; Indicates the allocation weight of the jth resource in the resource pool. This value is used to control the device specified by the user. When the resources of the specified device are insufficient, other resources of the same manufacturer will be allocated, and will not be allocated to different manufacturers. The calculation method is as follows:

[0081]

[0082] Step 6: The solver solves step 5.

[0083] Distribute resources to the current task based on the solution results, update the resource status in the resource pool to "allocated resources", and update the task status in the task pool to "executed".

[0084] If there are no user-specified resources or resources from the same manufacturer in the solution results, it means that there are insufficient resources. At this time, try to add the resources in the cache pool to the allocatable resources and solve step 5 again.

[0085] It should be noted that whether to accept resources is decided by the user, and the scheduling platform only calculates the results of allocable resources and gives them to the user. If the demand is not met (insufficient resources), the scheduling platform will apply for relevant resources from the management platform to expand the resource pool.

[0086] Step 7: The scheduling platform monitors the resource pool status in real time and recycles resources when there are too many idle resources; resource utilization rate ( Indicates the idle time of the resource. Indicates the resource retention time).

[0087] when When the resource is recycled to the cache pool; when The resource is completely recovered to the management platform when needed, and then pushed to the scheduling platform for use when there is a related demand.

[0088] It should be noted that in the embodiment of the present invention, if the above-mentioned heterogeneous computing power scheduling method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions for a terminal to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read Only Memory), disk or optical disk, etc. Various media that can store program codes. In this way, the embodiment of the present invention is not limited to any specific combination of hardware and software.

[0089] Correspondingly, an embodiment of the present invention provides a heterogeneous computing power scheduling device, Figure 2 is a schematic diagram of the composition structure of the heterogeneous computing power scheduling device provided by an embodiment of the present invention, such as Figure 2 As shown, the heterogeneous computing power scheduling device 200 at least includes: a processor 201 and a computer-readable storage medium 202 configured to store executable instructions, wherein the processor 201 generally controls the overall operation of the heterogeneous computing power scheduling device. The computer-readable storage medium 202 is configured to store instructions and applications executable by the processor 201, and can also cache data to be processed or processed by each module in the processor 201 and the heterogeneous computing power scheduling device 200, which can be implemented through flash memory (FLASH) or random access memory (RAM, Random Access Memory).

[0090] An embodiment of the present invention provides a storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will be caused to execute the method provided by the embodiment of the present invention, for example, Figure 2 The method shown.

[0091] In some embodiments, the storage medium can be a computer-readable storage medium, for example, a ferroelectric random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disk, or a compact disk read-only memory (CD-ROM); it can also be various devices including one or any combination of the above memories.

[0092] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0093] As an example, executable instructions may, but need not correspond to, a file in a file system, may be stored as part of a file storing other programs or data, such as one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions). As an example, executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed at multiple locations and interconnected by a communication network.

[0094] The above description is only an embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement and improvement made within the spirit and scope of the present invention are included in the protection scope of the present invention.

[0095] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present invention, the size of the serial number of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention. The serial numbers of the above-mentioned embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments.

[0096] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method or device. In the absence of further restrictions, an element defined by the statement "comprises one..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed.

[0097] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A heterogeneous computing power scheduling method, characterized in that: Applied to a heterogeneous computing power scheduling system, the system includes a heterogeneous computing power management platform and a heterogeneous computing power scheduling platform; the method includes: The heterogeneous computing power management platform responds to the heterogeneous computing power scheduling task sent by the user, classifies the heterogeneous computing power resources of the scheduling system according to the applicable scenarios and computing power resource types of the computing power resources, obtains resource adaptation scenarios and resource adaptation manufacturers, and sends the heterogeneous computing power scheduling task and the heterogeneous computing power resources to the heterogeneous computing power scheduling platform; The heterogeneous computing power scheduling platform receives the heterogeneous computing power resources and the heterogeneous computing power scheduling tasks sent by the heterogeneous computing power management platform, and divides the heterogeneous computing power resources and the heterogeneous computing power scheduling tasks into a resource pool and a task pool respectively; The heterogeneous computing power scheduling platform constructs a computing power scheduling objective function; the computing power scheduling objective function is expressed as: , the constraint condition of the computing power scheduling objective function is expressed as: , ; in, Indicates the number of resources required for the current task; represents the number of resources in the resource pool. If the current task is an inference task, it represents the inference resource pool. If the current task is a training task, it represents the training resource pool. Indicates the video memory of the jth resource in the resource pool; Indicates that the jth resource in the resource pool has occupied the video memory; Indicates the video memory usage of the i-th resource requested by the current task; Indicates whether the i-th resource requested by the current task is allocated to the j-th device in the resource pool; represents the allocation weight of the jth resource in the resource pool; The heterogeneous computing power scheduling platform solves the computing power scheduling objective function to obtain a resource allocation result; The heterogeneous computing power scheduling platform monitors the task pool, obtains the current task from the task polling queue, and allocates the computing power resources in the resource pool to the current task according to the resource allocation result, the resource adaptation scenario and the resource adaptation manufacturer.

2. The method according to claim 1, characterized in that: The resource pool includes: a training resource pool, a reasoning resource pool, a test resource pool and a cache pool; The training resource pool stores computing devices used for training; The inference resource pool stores computing devices used for inference; The test resource pool stores a small amount of computing power equipment for trial operation; The cache pool is used to reclaim idle computing resources in the training resource pool and the reasoning resource pool according to preset rules.

3. The method according to claim 2, characterized in that The test resource pool is also used to allocate corresponding computing resources according to user needs and test run tasks to obtain operating parameters; The operating parameters include: video memory occupancy rate and model throughput.

4. The method according to claim 3, characterized in that: The method further comprises: When the resource adaptation manufacturer does not exist in the resource allocation result, it indicates that the resources are insufficient. The computing power resources in the cache pool are added to the resource pool, and the resource allocation result is re-solved.

5. The method according to claim 4, characterized in that The method further comprises: Monitor the resource status of the resource pool in real time to obtain resource idle time and resource retention time; Determining a resource utilization rate based on the resource idle time and the resource retention time; When the resource utilization rate does not exceed a preset utilization rate threshold, the idle computing resources are recycled to the heterogeneous computing power management platform.

6. The method according to claim 5, characterized in that After allocating the heterogeneous computing resources to the current task according to the resource allocation result, the method further includes: The resource status of the computing resources allocated in the resource pool is updated to allocated resources, and the task status in the task pool is updated to executed.

7. An electronic device, characterized in that: include: A memory for storing executable instructions; A processor, used to implement the heterogeneous computing power scheduling method described in any one of claims 1 to 6 when executing the executable instructions stored in the memory.

8. A computer-readable storage medium, characterized in that: Executable instructions are stored, which are used to cause the processor to execute the executable instructions to implement the heterogeneous computing power scheduling method described in any one of claims 1 to 6.

Images