Intelligent calculation center cloud scheduling system

Abstract

The invention provides an intelligent calculation center cloud scheduling system, and relates to the technical field of artificial intelligence and big data, and the system comprises a GPU cluster which is formed by the mixed deployment of at least a first type of GPU cards and a second type of GPU cards; and the scheduler is connected with the GPU cluster, a plurality of scheduling strategies are configured in the scheduler, and the scheduler is used for carrying out hybrid scheduling on the first type of GPU cards and the second type of GPU cards based on the scheduling strategies. The method has the beneficial effects that multi-strategy hybrid scheduling of different types of GPU cards is effectively realized, the use efficiency of computing power resources is improved, the operation cost is reduced, the operation efficiency of a system is improved, and the system risk is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and big data technology, and in particular to a kind of intelligence calculation center cloud scheduling system. BACKGROUND

[0002] With the rapid development of artificial intelligence (AI) and big data technology, the demand for computing resources in various sectors of society has shown explosive growth. This trend is not only reflected in large technology companies and research institutions, but also gradually permeates into the daily operations of small and medium-sized enterprises and various industries. In order to cope with this growing demand for computing, a special cloud scheduling system for intelligence calculation has emerged as an important tool for optimizing computing services. The special cloud scheduling system for intelligence calculation is a computing resource management platform for specific application scenarios. It can effectively allocate and manage computing resources through intelligent scheduling algorithms and resource management strategies to meet the needs of enterprises in different time and different scenarios. The main advantage of this system is to provide on-demand computing services, helping enterprises avoid unnecessary hardware investment and subsequent maintenance costs.

[0003] Currently, existing special cloud scheduling systems for intelligence calculation are mainly developed and optimized around Nvidia's computing cards. These computing cards, with their powerful graphics processing capabilities and computing performance, are widely used in fields such as deep learning and image recognition. Therefore, many enterprises tend to choose Nvidia's products when building computing platforms. Although Nvidia's computing cards have excellent performance, their market dominance also poses certain risks to enterprises. Especially in the context of increasingly complex international situation, the uncertainty of the supply chain makes dependence on a single manufacturer particularly vulnerable. Once there is a delay in supply or price fluctuations, the enterprise's computing tasks may be severely affected. Therefore, improving the support capability of the special cloud scheduling system for intelligence calculation for domestic computing cards has become a problem to be solved. SUMMARY

[0004] In view of the problems in the prior art, the present application provides a special cloud scheduling system for intelligence calculation, comprising: a GPU cluster formed by mixed deployment of at least first and second GPU cards; a scheduler connected to the GPU cluster, the scheduler is configured with multiple scheduling strategies, and is used for mixed scheduling of the first and second GPU cards based on the scheduling strategies.

[0005] Preferably, the scheduling strategy comprises a group scheduling strategy, and the scheduler comprises a first scheduling module configured to perform mixed scheduling on the first GPU cards and the second GPU cards when all resource requirements of tasks in a job can be met after receiving a service request for executing the job, or otherwise, configure the job to a processing state until all resource requirements of the tasks can be met.

[0006] Preferably, the scheduling strategy comprises a network topology scheduling strategy, and the scheduler comprises a second scheduling module configured to schedule tasks of a same job in a same switch supporting an IB network and an Ethernet network.

[0007] Preferably, the scheduling strategy comprises a shared scheduling strategy, and the scheduler comprises a third scheduling module configured to schedule multiple tasks to a same GPU card according to a number of GPU cards required by the tasks and a required amount of GPU memory required by each of the GPU cards.

[0008] Preferably, the scheduling strategy comprises a specified host scheduling strategy, and the scheduler comprises a fourth scheduling module configured to provide a specified port for specifying a group of hosts when creating a task, so that the task can only be scheduled in the specified hosts.

[0009] Preferably, the scheduling strategy comprises an emergency task scheduling strategy, and the scheduler comprises a fifth scheduling module configured to put a task corresponding to an emergency attribute of a user into a pre-configured emergency task queue when the task is submitted by the user, and configure the task in the emergency task queue to a highest scheduling priority.

[0010] Preferably, the scheduling strategy comprises a fair scheduling strategy, and the scheduler comprises a sixth scheduling module configured to create a corresponding scheduling queue for each user group created by a service layer based on the user group, put tasks of a same user group into a same scheduling queue, and cyclically select tasks of each user group for scheduling.

[0011] Preferably, the scheduling strategy comprises a fine-grained scheduling strategy, and the scheduler comprises a seventh scheduling module configured to schedule a task to a CPU card with a suitable GPU memory granularity slice according to a required GPU memory granularity size and a required GPU memory granularity slice number specified by the task in a service request.

[0012] Preferably, the scheduling strategy comprises a unified memory scheduling, and the scheduler comprises an eighth scheduling module configured to allocate a part of memory of a host to a GPU task when the unified memory scheduling is enabled.

[0013] Preferably, the scheduling strategy comprises dataset affinity scheduling, and the scheduler comprises a ninth scheduling module configured to preferentially schedule a task contained in a service request to a computing node corresponding to a GPU card having a required dataset resource cached.

[0014] The above technical solution has the following advantages or beneficial effects: effectively implementing multi-strategy hybrid scheduling of different types of GPU cards, improving the efficiency of computing resource use and reducing operation costs, while improving the operation efficiency of the system and reducing system risks. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 In a preferred embodiment of the present application, a structural diagram of a cloud scheduling system of an intelligent computing center is provided. DETAILED DESCRIPTION

[0016] The present application will be described in detail below with reference to the accompanying drawings and specific embodiments. The present application is not limited to this embodiment, and other embodiments can also fall within the scope of the present application as long as they comply with the spirit of the present application.

[0017] In a preferred embodiment of the present application, based on the above-mentioned problems existing in the prior art, a cloud scheduling system of an intelligent computing center is provided, as shown in Figure 1 The cloud scheduling system of the intelligent computing center comprises: a GPU cluster 1 comprising at least a first type of GPU card 11 and a second type of GPU card 12 mixedly deployed; a scheduler 2 connected to the GPU cluster 1, wherein a plurality of scheduling strategies are configured in the scheduler 2, and the scheduler 2 is used to perform hybrid scheduling on the first type of GPU card 11 and the second type of GPU card 12 based on the scheduling strategies.

[0018] Specifically, in the present embodiment, the first type of GPU card 11 is a domestic chip, and the second type of GPU card 12 is a NVIDIA chip. The domestic chip and the NVIDIA chip are mixedly deployed to form a GPU cluster, and the domestic chip and the NVIDIA chip are scheduled in parallel through a plurality of scheduling strategies, so as to meet the requirements of heterogeneous resource management capability of an intelligent computing cluster, support mixed deployment and parallel scheduling of different types of domestic chips and NVIDIA chips, realize the ability of flexible quota management of multi-tenant queues, and provide basic services such as data storage, governance, and analysis, as well as pre-trained models and model capabilities in specific fields, such as voice, image, and natural language processing, to accelerate the development of intelligent computing application scenarios and provide high-quality training and computing power services for vertical large model application scenarios.

[0019] Further, the intelligent computing center cloud scheduling system of the present application supports large-scale distributed parallel training of multi-task GPU level, supports domestic computing power resources, supports parallel operation of thousands of reasoning tasks, and supports Pod scheduling throughput of >5K / s. It supports domestic computing power resources; supports x86 and ARM architecture CPUs, including domestic CPUs (Haiguang, Kunpeng processors, etc.); supports multiple operating systems, including CentOS, Debian, Readhat, Kylin, etc. Support multiple intelligent scheduling strategies, fully utilize GPU computing power, and scheduling strategies include but are not limited to Tor-aware scheduling, Gang scheduling, Binpack scheduling, etc.

[0020] Specifically, the intelligent computing center cloud scheduling system of the present application is a domestic special cloud scheduling system based on business requests, which is realized by the following ways: In a preferred embodiment of the present application, the scheduling strategy includes a group scheduling strategy, and the scheduler 2 includes a first scheduling module 21 for performing mixed scheduling of the first GPU card 11 and the second GPU card 12 when it is determined that the resource requirements of all tasks in the job can be met after receiving a business request to execute the job, otherwise the job is configured to a pending state until the resource requirements of all tasks can be met.

[0021] Specifically, in this embodiment, based on the group scheduling strategy (Gang scheduling), the submitted job Job will only be successfully scheduled when the requirements of all tasks Task in the job Job are met, otherwise all Task will be in the pending state, and when the resources are sufficient, all Task will complete the scheduling.

[0022] In a preferred embodiment of the present application, the scheduling strategy includes a network topology scheduling strategy, and the scheduler 2 includes a second scheduling module 22 for scheduling each task of the same job in the same switch, and the switch supports IB network and Ethernet network.

[0023] Specifically, in this embodiment, the network topology scheduling strategy supports two network types for cluster management: IB network and Ethernet network, and supports scheduling according to access switches, and tries to schedule tasks in one switch to avoid communication loss across switches.

[0024] In a preferred embodiment of the present application, the scheduling strategy includes a shared scheduling strategy, and the scheduler 2 includes a third scheduling module 23 for scheduling multiple tasks to the same GPU card according to the number of required GPU cards specified by the tasks in the business request and the amount of video memory required by each GPU card.

[0025] Specifically, in this embodiment, a fine-grained GPU scheduling is provided based on a shared scheduling strategy, allowing multiple tasks to specify GPU memory and be scheduled to the same GPU card, thereby achieving GPU card reuse and improving GPU card utilization. When submitting a task, the number of GPU cards required and the amount of memory each GPU card needs to occupy are specified.

[0026] In a preferred embodiment of the present invention, the scheduling strategy includes a specified host scheduling strategy, and the scheduler 2 includes a fourth scheduling module 24 for providing a specified port for specifying a set of hosts when creating a task, thereby allowing the task to be scheduled only within the specified hosts.

[0027] In a preferred embodiment of the present invention, the scheduling strategy includes an emergency task scheduling strategy, and the scheduler 2 includes a fifth scheduling module 25, which is used to put the corresponding task into a pre-configured emergency task queue when the task submitted by the user has an emergency attribute, and to configure the task in the emergency task queue as the highest scheduling priority.

[0028] Specifically, in this embodiment, by using a built-in emergency task queue, when a user-submitted training task has an emergency task queue attribute, the emergency task will be placed in the emergency queue. Tasks in the emergency task queue have the highest scheduling priority, and the scheduler will only process other tasks after processing all the emergency tasks.

[0029] In a preferred embodiment of the present invention, the scheduling strategy includes a fair scheduling strategy, and the scheduler 2 includes a sixth scheduling module 26, which is used to create a corresponding scheduling queue for each user group created based on the business layer, so as to put tasks belonging to the same user group into the same scheduling queue, and to cyclically select tasks of each user group for scheduling.

[0030] Specifically, in this embodiment, a user group-based fair scheduling mechanism is provided. The business layer creates different user groups, and the scheduler creates a corresponding scheduling queue for each user group. Training tasks submitted by users in the same user group will enter the same queue, and the scheduler will cyclically select tasks from each user group for scheduling.

[0031] In a preferred embodiment of the present invention, the scheduling strategy includes a fine-grained scheduling strategy, and the scheduler 2 includes a seventh scheduling module 27, which is used to schedule the task to a CPU card with a suitable memory granularity slice according to the required memory granularity size and the number of memory granularity slices specified by the task contained in the service request.

[0032] Specifically, in the embodiment, based on the fine-grained scheduling strategy, the whole GPU card memory is divided into multiple granularity slices according to the preset memory granularity size, that is, the GPU card memory is sliced and isolated. When submitting a task, the memory granularity size (such as 4G or 8G) and the number of memory granularity slices that need to be sliced are specified. The job is scheduled to the GPU card with appropriate memory granularity slices.

[0033] In the preferred embodiment of the application, the scheduling strategy includes unified memory scheduling, and the scheduler 2 includes an eighth scheduling module 28 for allocating part of the host memory to the GPU task when the unified memory scheduling is started.

[0034] Specifically, in the embodiment, the unified memory scheduling UM provided by Cuda can use part of the host memory as GPU memory. When the UM is started, part of the host memory is allocated to the GPU task. When the GPU task is submitted, the GPU card memory is expanded and calculated to find a GPU card with appropriate job memory granularity slices for scheduling.

[0035] In the preferred embodiment of the application, the scheduling strategy includes data set affinity scheduling, and the scheduler 2 includes a ninth scheduling module 29 for preferentially scheduling the task contained in the service request to the computing node corresponding to the GPU card that caches the required data set resource.

[0036] Specifically, in the embodiment, based on the data set node caching capability of the AIStation, when a training task applies for using a data set, the training task is preferentially scheduled to the node that caches the data set, thereby avoiding the re-download of the data set cache and reducing the data set preparation time of the training task. During data set affinity scheduling, the node that completely matches the data set required by the job pod is selected as much as possible, and the data set affinity strategy is ignored for the node that does not match or partially matches.

[0037] In summary, the intelligent computing center cloud scheduling system of the application can realize multi-strategy hybrid scheduling of domestic GPU and NVIDIA GPU, improve the efficiency of computing resource use, and reduce the operation cost.

[0038] At the same time, the running efficiency of the system is improved, and the system risk is reduced.

[0039] As a preferred embodiment of the present application, the intelligent center cloud scheduling system of the present application can be applied to the field of large model fine-tuning. Large model fine-tuning is a further training process based on a pre-trained model for a specific task. This process involves adjusting some or all parameters of the model to adapt to new task requirements. The core of fine-tuning is transfer learning, which utilizes the general feature representation ability learned by the pre-trained model on large-scale datasets and applies it to specific tasks to improve the performance of the model. An efficient intelligent proprietary cloud scheduling system can fully guarantee the utilization rate of task computing power, shorten the time and save the cost.

[0040] The steps of fine-tuning usually include: Select a pre-trained large model, such as BERT, GPT-2, etc.

[0041] Prepare a dataset for a specific task and perform necessary preprocessing.

[0042] Load the pre-trained model and freeze the parameters of some layers as needed to preserve the general features learned by the model.

[0043] Train the model on the new dataset through an intelligent optimized scheduling strategy, usually using a smaller learning rate to avoid destroying the knowledge of the pre-trained model.

[0044] Evaluate the performance of the model on the new task and make adjustments as needed.

[0045] The above is only a preferred embodiment of the present application, and does not limit the implementation and protection scope of the present application. For those skilled in the art, it should be realized that any equivalent replacement and obvious change made by applying the contents of the present application and drawings should be included in the protection scope of the present application.

Claims

1. A cloud scheduling system for an intelligent computing center, characterized in that, include: GPU clusters include those formed by a mixed deployment of at least Type I GPU cards and Type II GPU cards; A scheduler is connected to the GPU cluster. The scheduler is configured with multiple scheduling policies for mixed scheduling of the first type of GPU cards and the second type of GPU cards based on the respective scheduling policies.

2. The intelligent computing center cloud scheduling system according to claim 1, characterized in that, The scheduling strategy includes a group scheduling strategy. The scheduler includes a first scheduling module, which, upon receiving a business request to execute a job, determines whether the resource requirements of all tasks in the job can be met, and then performs mixed scheduling on each of the first type of GPU cards and the second type of GPU cards; otherwise, it configures the job as pending until the resource requirements of all tasks in the job can be met.

3. The intelligent computing center cloud scheduling system according to claim 1, characterized in that, The scheduling strategy includes a network topology scheduling strategy, and the scheduler includes a second scheduling module for scheduling tasks belonging to the same job within the same switch. The switch supports IB networks and Ethernet networks.

4. The intelligent computing center cloud scheduling system according to claim 1, characterized in that, The scheduling strategy includes a shared scheduling strategy, and the scheduler includes a third scheduling module, which is used to schedule multiple tasks to the same GPU card according to the number of GPU cards required by the tasks contained in the service request and the amount of video memory required by each GPU card.

5. The intelligent computing center cloud scheduling system according to claim 1, characterized in that, The scheduling strategy includes a specified host scheduling strategy, and the scheduler includes a fourth scheduling module for providing a specified port for specifying a set of hosts when creating a task, thereby enabling the task to be intelligently scheduled within the specified hosts.

6. The intelligent computing center cloud scheduling system according to claim 1, characterized in that, The scheduling strategy includes an emergency task scheduling strategy. The scheduler includes a fifth scheduling module, which is used to place the corresponding task into a pre-configured emergency task queue when the task submitted by the user has an emergency attribute, and to configure the tasks in the emergency task queue as the highest scheduling priority.

7. The intelligent computing center cloud scheduling system according to claim 1, characterized in that, The scheduling strategy includes a fair scheduling strategy, and the scheduler includes a sixth scheduling module, which is used to create a corresponding scheduling queue for each user group created based on the business layer, so as to put tasks belonging to the same user group into the same scheduling queue, and to cyclically select tasks of each user group for scheduling.

8. The intelligent computing center cloud scheduling system according to claim 1, characterized in that, The scheduling strategy includes a fine-grained scheduling strategy. The scheduler includes a seventh scheduling module, which is used to schedule the task to a CPU card with a suitable memory granularity slice according to the required memory granularity size and the number of memory granularity slices specified by the task in the service request.

9. The intelligent computing center cloud scheduling system according to claim 1, characterized in that, The scheduling strategy includes unified memory scheduling, and the scheduler includes an eighth scheduling module, which is used to allocate part of the host memory to GPU tasks when unified memory scheduling is enabled.

10. The intelligent computing center cloud scheduling system according to claim 1, characterized in that, The scheduling strategy includes dataset affinity scheduling, and the scheduler includes a ninth scheduling module, which is used to prioritize scheduling tasks contained in business requests to the computing nodes corresponding to GPU cards that cache the required dataset resources.

Images