Separation decoupling resource allocation method and device, equipment and medium

Abstract

The invention provides a separation and decoupling resource allocation method and device, equipment and a medium, and relates to the technical field of resource management. The method comprises the following steps: acquiring resource demand information of a target application for at least one resource; according to the resource demand information, corresponding resources are scheduled from at least one resource pool constructed in advance and allocated to the target application, and the at least one resource pool is obtained after the to-be-scheduled resources are separated and decoupled. According to the invention, the resource utilization rate of the super server can be improved, and the flexibility of resource allocation and service can be ensured.

Description

Background Art

[0002] Against the backdrop of the rapid development of artificial intelligence (AI), AI applications are placing increasing demands on computing resources. Especially when processing large-scale data and complex algorithms, traditional servers face many challenges, including but not limited to processing power bottlenecks, low energy efficiency, and system scalability limitations.

[0003] The servers used in AI intelligent computing clusters are mostly in the form of a single machine with 8 cards. Some manufacturers use self-developed interconnection protocols such as high-performance computing clusters (HCCS, High-Performance Computing Cluster Service) and big data computing platforms BLINK to build a high-speed network architecture GPU Fabric between 8 GPUs in a single machine, but it does not support expansion outside the machine and provides GPU interconnection technology similar to NVIDIA (nvlink-network capability). Each server can only build a large-scale cluster through ROCE (Remote Direct Memory Access over Converged Ethernet, remote direct memory access through converged Ethernet) / IB (InfiniBand, infinite bandwidth) technology, and the efficiency of tensor parallel computing is low. NVIDIA can achieve 576 GPU scale-up (GPU scale expansion) interconnection to form a GPU fabric (GPU interconnection network) through NVLINK.

[0004] In related technologies, AI super servers are servers with higher computing power density compared to traditional single-machine 8-card servers. NVIDIA first provided NVL72 integrated rack AI super server products, supporting 72-GPU nvlink scale-up interconnection, but the products are highly tightly coupled and bundled with 36 grace CPUs (Grace central processing unit, a high-performance CPU architecture developed by Arm and NVIDIA). Original equipment manufacturers (OEMs) lack flexibility, and customers do not need so many CPUs in intelligent computing scenarios, and the cost of machine deployment and use is high. The launch of NVL72 indicates that AI super servers will become the main form of computing power product delivery in the future, and will drive manufacturers to build their own AI super server products based on self-developed scale-up interconnection solutions. However, there are currently no standard specifications for such products in the industry, and cloud vendors / operators and other customers cannot customize them on demand.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] The present disclosure provides a separated and decoupled resource allocation method, apparatus, device and medium, which at least to a certain extent overcome the problems of low resource utilization and lack of allocation flexibility in related technologies.

[0007] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0008] According to one aspect of the present disclosure, a separated and decoupled resource allocation method is provided, comprising: obtaining resource demand information of a target application for at least one resource; and scheduling corresponding resources from at least one pre-built resource pool and allocating them to the target application based on the resource demand information, wherein the at least one resource pool is obtained after separating and decoupling the resources to be scheduled.

[0009] In some embodiments, the resources to be scheduled are resources provided by a cluster service system; the cluster service system includes: multiple servers; wherein, before scheduling corresponding resources from at least one pre-constructed resource pool according to the resource demand information and allocating them to the target application, the method also includes: obtaining resource information of at least one of the following resources provided by each server in the cluster service system: central processing resources, computing power resources, storage resources and communication interface resources; based on the resource information provided by each server in the cluster service system, separating and decoupling the resources provided by each server in the cluster service system, and constructing at least one resource pool corresponding to the cluster service system.

[0010] In some embodiments, after separating and decoupling the resources provided by each server in the cluster service system based on the resource information provided by each server in the cluster service system, and constructing at least one resource pool corresponding to the cluster service system, the method also includes: monitoring resource change information provided by each server in the cluster service system; and updating at least one resource pool corresponding to the cluster service system based on the resource change information provided by each server in the cluster service system.

[0011] In some embodiments, based on the resource change information provided by each server in the cluster service system, at least one resource pool corresponding to the cluster service system is updated, including: when information about resources to be added in the cluster service system is monitored, the information about resources to be added is identified, and at least one resource pool in the cluster service system related to the information about resources to be added is updated; when information about resources to be removed in the cluster service system is monitored, running resources in the cluster service system are migrated, and at least one resource pool in the cluster service system related to the information about resources to be removed is updated.

[0012] In some embodiments, the target application is an AI application, the central processing resource is a CPU resource, the computing power resource is a GPU resource, the storage resource is a memory resource, and the communication interface resource is a switch resource; the resource pool corresponding to the cluster service system includes at least one of the following: a CPU pool, a GPU pool, a memory pool, and an I / O pool; the CPU pool includes: at least one CPU node; the GPU pool includes: at least one GPU node; the memory pool includes: at least one memory sharing node; the I / O pool includes: at least one switching aggregation node.

[0013] In some embodiments, the method also includes: obtaining resource status information of each GPU node; constructing a network topology of the cluster service system based on the resource status information of each GPU node; and dynamically adjusting a communication strategy based on the resource status information of each GPU node and the network topology of the cluster service system to ensure the communication performance of the cluster service system.

[0014] According to another aspect of the present disclosure, a separated and decoupled resource allocation device is also provided, including: an information acquisition module, used to obtain resource demand information of a target application for at least one resource; a resource allocation module, used to schedule corresponding resources from at least one pre-built resource pool according to the resource demand information, and allocate them to the target application, wherein the at least one resource pool is obtained after separating and decoupling the resources to be scheduled.

[0015] According to another aspect of the present disclosure, an electronic device is also provided, which includes: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned separation and decoupled resource allocation methods by executing the executable instructions.

[0016] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the separation and decoupling resource allocation method described above is implemented.

[0017] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the computer program implements any one of the above-mentioned separation and decoupled resource allocation methods.

[0018] The separated and decoupled resource allocation method, device, equipment and medium provided in the embodiments of the present disclosure obtain resource demand information of the target application for at least one resource, and then, according to the obtained resource demand information, schedule the corresponding resources from at least one resource pool obtained by separating and decoupling the pre-built resources to be scheduled, and allocate them to the target application. The embodiments of the present disclosure can improve the resource utilization of the super server while ensuring the flexibility of resource allocation and service.

[0019] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0021] Figure 1 A schematic diagram of a traditional server system architecture in the related art is shown;

[0022] Figure 2 A schematic diagram of an NVIDIA server system architecture in the related art is shown;

[0023] Figure 3 A schematic diagram of a separated and decoupled super server architecture in an embodiment of the present disclosure is shown;

[0024] Figure 4 A schematic diagram of a distributed system for separating and decoupling super servers in an embodiment of the present disclosure is shown;

[0025] Figure 5 A flow chart of a separation and decoupling resource allocation method in an embodiment of the present disclosure is shown;

[0026] Figure 6 A flow chart of a resource allocation method based on separation and decoupling of a resource pooling scheduling module in an embodiment of the present disclosure is shown;

[0027] Figure 7 A flow chart of a resource allocation method based on separation and decoupling of an adaptive communication library module in an embodiment of the present disclosure is shown;

[0028] Figure 8 A schematic diagram of a separated and decoupled resource allocation device in an embodiment of the present disclosure is shown;

[0029] Fig. 9 A block diagram of an electronic device in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0030] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0031] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0032] The specific implementation of the embodiment of the present disclosure is described in detail below with reference to the accompanying drawings.

[0033] Figure 1 FIG. 1 shows a schematic diagram of a traditional server system architecture in the related art. Figure 1 As shown, the system architecture may include a traditional server 101 , Ethernet 102 , and Remote Direct Memory Access over Converged Ethernet (ROCE) 103 .

[0034] In one embodiment of the present disclosure, the traditional server 101 may include a first traditional server 1011, a second traditional server 1012, and a third traditional server 1013. Taking the third traditional server 1013 as an example, the third traditional server 1013 may include a central processing unit (CPU) and a graphics processing unit (GPU).

[0035] In one embodiment of the present disclosure, the CPU and 8-GPU can be connected point-to-point based on the peripheral component interconnect express standard PCIE, which provides higher bandwidth and lower latency and can support a variety of different device types and speed levels.

[0036] In one embodiment of the present disclosure, the CPUs in each traditional server are connected via Ethernet, and the 8-GPUs in each traditional server are connected via a ROCE network.

[0037] However, in the related technology, a single server can only provide a high-bandwidth Internet network between 8 GPUs in a single machine through self-developed interconnection protocols such as high-performance computing clusters (HCCS) and big data computing platforms BLINK, but does not support high-bandwidth interconnection networking outside the machine. Each server can only build a large-scale cluster through ROCE (Remote Direct Memory Access over Converged Ethernet) / IB (InfiniBand), etc., and cannot break through the 8-GPU scale to form a larger GPU interconnection network, and the tensor computing efficiency is low.

[0038] It should be noted that the number of traditional servers in the embodiments of the present disclosure is only exemplary, and the embodiments of the present disclosure do not specifically limit the number of traditional servers.

[0039] Figure 2 FIG. 1 shows a schematic diagram of an NVIDIA server system architecture in the related art. Figure 2 As shown, the system architecture may include an NVIDIA server 201 , Ethernet 102 , and Remote Direct Memory Access over Converged Ethernet (ROCE) 103 .

[0040] In one embodiment of the present disclosure, the NVIDIA server 201 may include a first NVIDIA server 2011, a second NVIDIA server 2012, and a third NVIDIA server 2013. Taking the third NVIDIA server 2013 as an example, the third NVIDIA server 2013 may include 36 Grace CPUs, support 72-GPU nvlink scale-up interconnection, and the CPU and GPU are connected through nvlink, a high-speed interconnection technology developed by NVIDIA. However, the products are highly tightly coupled, OEM manufacturers lack flexibility, and customers do not need so many CPUs in the intelligent computing scenario. The cost of machine deployment and use is high, and customers do not have demand dominance.

[0041] It should be noted that the number of traditional servers in the embodiments of the present disclosure is only exemplary, and the embodiments of the present disclosure do not specifically limit the number of traditional servers.

[0042] Figure 3 FIG. 2 shows a schematic diagram of a separated and decoupled super server architecture in an embodiment of the present disclosure. Figure 3As shown, the server architecture may include a CPU scheduling node 301 , a switching aggregation node 302 , a GPU computing power node 303 and a memory sharing node 304 .

[0043] In one embodiment of the present disclosure, the CPU scheduling node 301 serves as an internal master control node and is composed of ordinary servers. Each server is connected to the memory sharing node and the GPU computing power node via a PCIE cable and a retimer, retains a traditional Ethernet port, and communicates externally with the CPU scheduling nodes of other super servers to form a larger cluster.

[0044] In one embodiment of the present disclosure, the switching aggregation node 302 lays out switches for the ROCE network, the computing fast link CXL / peripheral component interconnect high-speed standard PCIE network, and the Ethernet network. The ROCE network is connected to each GPU of the GPU computing node through the leaf node switch in the downstream, and is connected to other AI super servers through the external root node switch in the upstream; the CXL / PCIE network connects the CPU scheduling node 301, the GPU computing node 303, and the memory sharing node 304 inside the super server through the CXL / PCIE switch to provide service internal bus capabilities. If the supplier of the GPU computing node 303 does not provide a GPU interconnection network, it is constructed through a CXL / PCIE switch; the Ethernet is connected to the CPU scheduling node 301, and is connected to the Ethernet of other super servers to form a management plane network. Overall, the number of overall input / output I / O devices is reduced by several times through the switch and the capacity of the network card, thereby reducing overall power consumption.

[0045] In one embodiment of the present disclosure, the GPU computing power node serves as the computing power subject. It communicates with the CPU and accesses the shared memory through CXL / PCIE, and is connected to the leaf node switch through the standard ROCE interface. It forms a 1,000-card / 10,000-card cluster with the GPU computing power nodes in other super servers, and mainly carries the data parallelism and pipeline parallelism network traffic of the parameter plane. It supports GPU scale-up interconnection of various specifications such as 16, 32, 48, 64, and 128 internally to form a GPU interconnection network, providing high-bandwidth and low-latency communication capabilities, and mainly carries the tensor parallelism network traffic of the parameter plane.

[0046] In one embodiment of the present disclosure, the memory sharing node 304 is shared as a memory resource for the CPU and GPU to use, and provides unified system memory, memory consistency, and memory sharing capabilities through the standard CXL protocol. A GPU that supports CXL can directly access the node memory resources as the second-level cache of the GPU (i.e., L2 memory), and cooperate with high-bandwidth memory HBM to load model data, thereby improving the training capability and utilization of the GPU. At the same time, the L2 memory can bring the total memory capacity of different GPUs together. For example, GPU A (40G HBM+40G L2 memory) and GPU B (80HBM) can be calculated under the same model framework. The consistent capacity on both sides simplifies the model algorithm design, unloads tensor data, and improves computing power utilization.

[0047] Specifically, in one embodiment of the present disclosure, the CPU scheduling node 301 may include a first CPU server, a second CPU server, and a third CPU server; the switching aggregation node 302 may include an Ethernet switch, a first ROCE leaf node switch, a second ROCE leaf node switch, a third ROCE leaf node switch, a first CXL / PCIE switch, a second CXL / PCIE switch, and a third CXL / PCIE switch; the GPU scheduling node 303 may include a first GPU computing server (8 GPUs), a second GPU computing server (16 GPUs), a third GPU computing server (32 GPUs), and a GPU interconnection network; the memory sharing node 304 may include a first CXL memory expansion box and a second CXL memory expansion box.

[0048] In one embodiment of the present disclosure, the first super server 30 and the second super server 40 communicate with each other through an external Ethernet switch and a ROCE root node switch, respectively, wherein the Ethernet switch of the first super server 30 and the Ethernet switch of the second super server 40 are respectively connected to the external Ethernet switch, and the ROCE leaf node switch of the first super server 30 and the ROCE leaf node switch of the second super server 40 are respectively connected to the ROCE root node switch.

[0049] In one embodiment of the present disclosure, the management plane network is often composed of ordinary servers, constructed through traditional Ethernet, and communicates externally with the CPU scheduling nodes of other super servers and is responsible for CPU scheduling traffic. Specifically, the Ethernet switch in the switching aggregation node 302 is respectively connected to the first CPU server, the second CPU server, the third CPU server and the external Ethernet switch, and the external Ethernet switch is connected to the Ethernet switch of the second super server 40 to enable the first super server 30 and the second super server 40 to communicate.

[0050] In one embodiment of the present disclosure, the parameter plane network often includes a ROCE network and a GPU interconnection network, which carries data parallelism, pipeline parallelism, and tensor parallelism data traffic. Specifically, with respect to the ROCE network, each GPU computing server is connected to a leaf node switch via a standard ROCE interface, and forms a thousand-card / ten-million-card cluster with GPU computing nodes in other super servers. For example, the ROCE leaf node switch can be connected to the first GPU computing server, the second ROCE leaf node switch can be connected to the second GPU computing server, and the third ROCE leaf node switch can be connected to the third GPU computing server. The first ROCE leaf node switch, the second ROCE leaf node switch, and the third ROCE leaf node switch are respectively connected to the ROCE root node switch, and the ROCE root node switch is connected to the ROCE leaf node switch of the second super server 40, so that the first super server 30 and the second super server 40 communicate. With respect to the GPU interconnection network, the first GPU computing server, the second GPU computing server, and the third GPU computing server are respectively connected to the GPU interconnection network to form a larger GPU interconnection network, breaking through the limitation that only 8 GPU cards are installed inside the traditional computing node, and providing a larger high-bandwidth domain for tensor parallel computing.

[0051] It should be noted that the connection relationship between the above-mentioned ROCE leaf node switch and the GPU computing server is only for illustrative purposes. Different ROCE leaf node switches can be connected to different GPU computing servers in any order. The embodiment of the present disclosure does not specifically limit the connection order between the ROCE leaf node switch and the GPU computing server.

[0052] It should be noted that in the embodiment of the present disclosure, the server internally supports 16, 32, 48, 64, 128 GPU scale-up interconnections to form a GPU interconnection network. The connection relationship between the first GPU computing server, the second GPU computing server, and the third GPU computing server and the GPU interconnection network is only an exemplary relationship. The embodiment of the present disclosure does not specifically limit the specific connection order.

[0053] In one embodiment of the present disclosure, the internal bus capability of the server is provided by a CXL / PCIE switch on the switching aggregation node 302. Specifically, the CXL / PCIE network connects the CPU scheduling node 301, the GPU computing node 303, and the memory sharing node 304 inside the first super server 30 through the first CXL / PCIE switch, the second CXL / PCIE switch, and the third CXL / PCIE switch.

[0054] In one embodiment of the present disclosure, if the supplier of the GPU computing node 303 does not provide a GPU interconnection network, it can be constructed by using the first CXL / PCIE switch, the second CXL / PCIE switch and the third CXL / PCIE switch.

[0055] It should be noted that the number of CPU servers included in the above-mentioned CPU scheduling node, the number of ROCE leaf node switches and CXL / PCIE switches included in the switching aggregation node, the number of GPU computing servers included in the GPU computing power node, and the number of CXL memory expansion boxes included in the memory sharing node are only exemplary and can be set according to actual conditions. The embodiments of the present disclosure do not make specific limitations on this.

[0056] In one embodiment of the present disclosure, in the era of large models, model training requires the mixed use of multiple parallel strategies such as data parallelism, pipeline parallelism, and tensor parallelism, and the flow of tensor parallelism accounts for 75% of the overall flow of the AI ​​cluster, and the tensor parallel flow is distributed in the 8-GPU fabric of each server in the cluster. It can be seen that increasing the scale of the GPU fabric can effectively improve the efficiency of model calculation. This method separates and decouples the CPU, memory, and GPU, and supports GPU scale-up interconnection of multiple specifications (such as 16, 32, 48, 64, 128, etc.) in the GPU computing power node to form a GPU interconnection network, breaking through the limitation of 8 cards on a single machine. At the same time, the supporting CPU and memory can be flexibly configured on demand to achieve the best cost-effective solution for customers.

[0057] From the above, we can see that the separated and decoupled super server uses standard interconnection protocols such as CXL / PCIE and ROCE to separate CPU, GPU, and memory resources, improve the density of each resource node, support flexible customization, and provide customers with the best cost-effective solution.

[0058] Figure 4 FIG. 1 shows a schematic diagram of a distributed system for separating and decoupling super servers in an embodiment of the present disclosure. Figure 4 As shown, the server system may include a resource pooling scheduling module 501 and an adaptive communication library module 502 .

[0059] Among them, the resource pooling scheduling module 501 is used to provide pooling management functions of resource identification, resource abstraction, and resource segmentation and isolation. Each CPU, GPU, and memory are aggregated into a CPU abstraction, a GPU abstraction, and a memory abstraction, respectively, to provide high-concurrency job applications with a method similar to running on an ordinary server without the need for excessive adaptation. The resource pooling scheduling module automatically completes resource coordination and allocation, and supports multi-tenant use, online resource allocation, and real-time monitoring.

[0060] The adaptive communication library module 502 is used to realize automatic perception of resources and interconnection topology, and automatically match the communication strategy of AI model calculation according to the scale-up size and network type within the GPU computing power node.

[0061] From the above, we can see that in view of the characteristics of the separated and decoupled super server architecture, the distributed system provides the capabilities of resource pooling scheduling and adaptive communication library to solve the problem of how customers' applications can run seamlessly on the AI ​​super server, and provides dynamic scheduling and adaptive capabilities to improve overall efficiency.

[0062] Figure 5 FIG. 2 shows a flow chart of a method for resource allocation in a separation and decoupling manner in an embodiment of the present disclosure. Figure 5 As shown, the method includes:

[0063] S601: Obtain resource demand information of a target application for at least one resource.

[0064] In one embodiment of the present disclosure, it is first necessary to obtain resource requirement information of the target application, wherein the target application may require one or more resources such as CPU, GPU and memory. Therefore, according to actual conditions, it is necessary to obtain resource requirement information of the target application for at least one resource.

[0065] S602: According to the resource demand information, corresponding resources are scheduled from at least one pre-built resource pool and allocated to the target application, wherein at least one resource pool is obtained by separating and decoupling the resources to be scheduled.

[0066] In one embodiment of the present disclosure, one or more resources to be scheduled in the system are abstracted and pooled in advance to build a resource pool. According to the acquired resource demand information, corresponding resources are scheduled from at least one pre-built resource pool and allocated to the target application.

[0067] As can be seen from the above, the embodiment of the present disclosure schedules corresponding resources from at least one pre-built resource pool to allocate to the target application according to the resource demand information of the target application for at least one resource. The embodiment of the present disclosure can improve the resource utilization of the super server while ensuring the flexibility of resource allocation and service.

[0068] In one embodiment of the present disclosure, the resources to be scheduled are resources provided by the cluster service system; the cluster service system includes: multiple servers; before the above S602, the method also includes: obtaining resource information of at least one of the following resources provided by each server in the cluster service system: central processing resources, computing resources, storage resources and communication interface resources; according to the resource information provided by each server in the cluster service system, the resources provided by each server in the cluster service system are separated and decoupled to build at least one resource pool corresponding to the cluster service system. To solve the problems of insufficient scalability and low utilization caused by the tight coupling of traditional server resources, the resource combination can be flexibly customized according to the needs of AI application scenarios, compatible with products of multiple manufacturers, and provide the most cost-effective solution, and resources can be independently upgraded and optimized, which is conducive to reducing the overall total cost of ownership TCO of the AI ​​cluster system.

[0069] In one embodiment of the present disclosure, after separating and decoupling the resources provided by each server in the cluster service system based on the resource information provided by each server in the cluster service system, and constructing at least one resource pool corresponding to the cluster service system, the method also includes: monitoring the resource change information provided by each server in the cluster service system; and updating at least one resource pool corresponding to the cluster service system based on the resource change information provided by each server in the cluster service system.

[0070] In one embodiment of the present disclosure, at least one resource pool corresponding to the cluster service system is updated based on resource change information provided by each server in the cluster service system, including: when information about resources to be added in the cluster service system is monitored, the information about resources to be added is identified, and at least one resource pool related to the information about resources to be added in the cluster service system is updated; when information about resources to be removed in the cluster service system is monitored, running resources in the cluster service system are migrated, and at least one resource pool related to the information about resources to be removed in the cluster service system is updated.

[0071] In one embodiment of the present disclosure, resource change information provided by each server in the cluster service system is monitored. If idle resources are found, the resources are reallocated and adjusted without affecting the business. If a resource failure is found, the faulty resources are automatically removed from the resource pool, and the resources are reallocated to ensure business continuity and stability. If new resources are found, the new resources are automatically identified and resource integration and business data binding are completed to ensure efficient use of resources.

[0072] In one embodiment of the present disclosure, the target application is an AI application, the central processing resource is a CPU resource, the computing power resource is a GPU resource, the storage resource is a memory resource, and the communication interface resource is a switch resource; the resource pool corresponding to the cluster service system includes at least one of the following: a CPU pool, a GPU pool, a memory pool, and an I / O pool; the CPU pool includes: at least one CPU node; the GPU pool includes: at least one GPU node; the memory pool includes: at least one memory sharing node; the I / O pool includes: at least one switching aggregation node.

[0073] In one embodiment of the present disclosure, the method also includes: obtaining resource status information of each GPU node; building a network topology of the cluster service system based on the resource status information of each GPU node; and dynamically adjusting the communication strategy based on the resource status information of each GPU node and the network topology of the cluster service system to ensure the communication performance of the cluster service system.

[0074] In one embodiment of the present disclosure, when online resource scheduling occurs, the GPU networking topology data pre-stored in the super server will be updated in real time, and the AI ​​communication strategy will be automatically learned based on the GPU interconnection network conditions within each super server, and the optimal path will be calculated during AI training and inference.

[0075] In one embodiment of the present disclosure, a separated and decoupled AI super server can be built using the existing CPU, GPU, and CXL memory. When training or reasoning a large model, first, the resource pooling scheduling module abstracts and pools the CPU, GPU, memory, and I / O resources in the system to form an independent resource pool. According to the needs of large model training, the system dynamically allocates multiple GPU nodes and connects these nodes into an efficient GPU interconnection network through CXL / PCIE and ROCE interfaces. When initialized, the adaptive communication library automatically senses the status of all GPU nodes, builds the current network topology, and continuously monitors the status and network traffic of each node. As the training process proceeds, the adaptive communication library dynamically adjusts the communication strategy based on real-time monitoring data, optimizes the data transmission path, and ensures high bandwidth and low latency communication performance. During the training process, if a GPU node fails, the adaptive communication library will immediately reroute the data and switch to the backup path to ensure the continuity and stability of the training task. After the training is completed, the system reallocates resources through the resource pooling scheduling module and uses some GPU nodes for model reasoning tasks. The adaptive communication library optimizes the communication path again to ensure data transmission efficiency during the reasoning process. In this way, the AI ​​super server can efficiently complete the training and inference tasks of large models, make full use of system resources, and provide excellent computing performance and reliability.

[0076] Figure 6A flow chart of a resource allocation method based on a separation and decoupling of a resource pooling scheduling module in an embodiment of the present disclosure is shown. Figure 6 As shown, the method includes:

[0077] S701, abstract and pool various resources in the cluster service system.

[0078] In one embodiment of the present disclosure, resources in the cluster service system may include CPU, GPU, memory, I / O, etc.

[0079] It should be noted that the above resource types are only exemplary, and different types of resources can be set according to actual conditions. The embodiments of the present disclosure do not specifically limit the types of resources.

[0080] S702, obtaining resource demand information of the target application, and dynamically allocating and scheduling resources according to the obtained resource demand information.

[0081] S703: Monitor and adjust the resource usage in the cluster service system in real time.

[0082] S704: When it is detected that a resource in the cluster service system fails, the failed resource is automatically removed from the resource pool and the resource is reallocated.

[0083] In one embodiment of the present disclosure, when new resources are detected in the cluster service system, the resources are automatically identified and resource integration and business data binding are completed to ensure efficient utilization of resources; when idle resources are detected in the cluster service system, the resources are reallocated and adjusted without affecting the business; when faulty resources are detected in the cluster service system, the faulty resources are automatically removed from the resource pool, and the resources are reallocated through the resource pooling scheduling module to ensure business continuity and stability.

[0084] In one embodiment of the present disclosure, the resource pooling scheduling module continuously monitors resource usage, so that the cluster service system can dynamically adjust the resource allocation strategy according to actual needs and optimize the overall system performance.

[0085] In one embodiment of the present disclosure, the resource pooling scheduling module provides multi-layer resource abstraction capabilities. The first layer abstracts the CPU scheduling nodes, memory sharing nodes, GPU computing nodes, and exchange aggregation nodes into CPU pools, GPU pools, memory pools, and I / O pools, respectively, all of which provide fine-grained resource scheduling capabilities. The second layer aggregates the CPU pool, GPU pool, memory pool, and I / O pool into a unified server virtual object, on which a normal operating system and virtualization components can run, supporting multi-tenant isolation.

[0086] In one embodiment of the present disclosure, the resource pool scheduling module supports online resource allocation and real-time monitoring capabilities. When adding resources, it automatically identifies the resources and completes resource inspection and business data copying. The business can identify the newly added resources and use them in the next round of scheduling. When removing resources, the system automatically completes the migration of running data before allowing the customer to confirm the removal of resources. The system automatically monitors the resource situation in real time, adds resources and removes faulty resources according to needs, and all resource allocations do not interrupt business.

[0087] In one embodiment of the present disclosure, the resource pool scheduling module supports online resource allocation and real-time monitoring capabilities. When adding resources, it automatically identifies the resources and completes resource inspection and business data copying. The business can identify the newly added resources and use them in the next round of scheduling. When removing resources, the system automatically completes the migration of running data before allowing the customer to confirm the removal of resources. The system automatically monitors the resource situation in real time, adds resources and removes faulty resources according to needs, and all resource allocations do not interrupt business.

[0088] As can be seen from the above, the resource pool scheduling module in the disclosed embodiment first needs to abstract and pool various resources in the cluster service system to construct various resource scheduling pools, and then dynamically allocate and schedule resources in the resource pool according to the resource demand information of the target application obtained. At the same time, it is also necessary to monitor and adjust the resource usage in the cluster service system in real time. When a resource failure is detected in the cluster service system, the faulty resource is automatically removed from the resource pool and the resource is reallocated. The disclosed embodiment can improve the resource utilization of the super server while ensuring the flexibility of resource allocation and service.

[0089] Figure 7 FIG. 1 shows a flow chart of a resource allocation method based on separation and decoupling of an adaptive communication library module in an embodiment of the present disclosure. Figure 7 As shown, the method includes:

[0090] S801, perform initialization configuration.

[0091] In one embodiment of the present disclosure, necessary configuration files and parameters are loaded, and initial communication paths and strategies are established according to the hardware configuration and network topology of the system.

[0092] S802, perceive the computing resource status in the cluster service system and construct the current network topology map.

[0093] In one embodiment of the present disclosure, after the current network topology map is constructed, it is necessary to perform scanning and detection in real time to identify the connection status and available bandwidth of each computing node and update the topology map accordingly.

[0094] It should be noted that the above-mentioned computing resources and computing nodes may refer to GPU resources and GPU nodes, but are not limited to this. Depending on actual conditions, the cluster service system may also include other computing resources, which is not specifically limited in the embodiments of the present disclosure.

[0095] S803, continuously monitor the status and network traffic of each GPU node in the cluster service system, and collect and analyze data in real time.

[0096] In one embodiment of the present disclosure, through monitoring, the status and network traffic of each GPU node in the cluster service system are collected and analyzed in real time to identify resource usage and potential bottlenecks, thereby ensuring the efficiency of the communication path.

[0097] S804, dynamically adjust the communication strategy according to the collected data and analysis results.

[0098] In one embodiment of the present disclosure, the process of dynamically adjusting the communication strategy may include selecting the optimal communication path, adjusting bandwidth allocation, optimizing data transmission protocols, etc. to ensure high bandwidth and low latency communication performance.

[0099] In one embodiment of the present disclosure, when a communication failure or an abnormality in a GPU node is detected, fault processing is automatically performed.

[0100] In one embodiment of the present disclosure, the fault handling process may include: rerouting data, switching backup paths, notifying system administrators, etc., to ensure stable operation of the system.

[0101] In one embodiment of the present disclosure, the adaptive communication library module may also have a self-learning capability to optimize communication strategies and path selection by continuously accumulating and analyzing historical data.

[0102] In one embodiment of the present disclosure, the adaptive communication library has the ability to automatically sense the network topology of the global GPU. The adaptive communication library module of each AI super server will retain the networking topology of all GPUs in the entire cluster system, and will update the topology data in real time when online resource allocation occurs. According to the GPU interconnection network conditions within each AI super server, the AI ​​communication strategy is automatically learned to calculate the optimal path during AI training and reasoning.

[0103] As can be seen from the above, after initialization, the adaptive communication library module in the disclosed embodiment senses the computing resource status in the cluster service system and builds the current network topology map, while continuously monitoring the status and network traffic of each GPU node in the cluster service system, and dynamically adjusts the communication strategy based on the collected data and analysis results. The disclosed embodiment can improve the resource utilization of the super server while ensuring the flexibility of resource allocation and service.

[0104] Based on the same inventive concept, the embodiment of the present disclosure also provides a separated and decoupled resource allocation device, as described in the following embodiment. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0105] Figure 8 A schematic diagram of a separated and decoupled resource allocation device in an embodiment of the present disclosure is shown. Figure 8 As shown, the device includes: an information acquisition module 901 and a resource allocation module 902.

[0106] Among them, the information acquisition module 901 is used to obtain the resource demand information of the target application for at least one resource; the resource allocation module 902 is used to schedule corresponding resources from at least one pre-built resource pool according to the resource demand information and allocate them to the target application, wherein at least one resource pool is obtained after separating and decoupling the resources to be scheduled.

[0107] As can be seen from the above, the embodiment of the present disclosure schedules corresponding resources from at least one pre-built resource pool to allocate to the target application according to the resource demand information of the target application for at least one resource. The embodiment of the present disclosure can improve the resource utilization of the super server while ensuring the flexibility of resource allocation and service.

[0108] In one embodiment of the present disclosure, the resources to be scheduled are resources provided by a cluster service system; the cluster service system includes: a plurality of servers.

[0109] In one embodiment of the present disclosure, the device also includes: a resource pool construction module 903, which is used to obtain resource information of at least one of the following resources provided by each server in the cluster service system: central processing resources, computing power resources, storage resources and communication interface resources; based on the resource information provided by each server in the cluster service system, the resources provided by each server in the cluster service system are separated and decoupled to construct at least one resource pool corresponding to the cluster service system.

[0110] In one embodiment of the present disclosure, the device also includes: an information monitoring module 904, which is used to monitor resource change information provided by each server in the cluster service system; and update at least one resource pool corresponding to the cluster service system according to the resource change information provided by each server in the cluster service system.

[0111] In one embodiment of the present disclosure, the information monitoring module 904 is also used to identify the resource information to be added and update at least one resource pool related to the resource information to be added in the cluster service system when the resource information to be added in the cluster service system is monitored; when the resource information to be removed in the cluster service system is monitored, migrate the running resources in the cluster service system and update at least one resource pool related to the resource information to be removed in the cluster service system.

[0112] In one embodiment of the present disclosure, the target application is an AI application, the central processing resource is a CPU resource, the computing power resource is a GPU resource, the storage resource is a memory resource, and the communication interface resource is a switch resource; the resource pool corresponding to the cluster service system includes at least one of the following: a CPU pool, a GPU pool, a memory pool, and an I / O pool; the CPU pool includes: at least one CPU node; the GPU pool includes: at least one GPU node; the memory pool includes: at least one memory sharing node; the I / O pool includes: at least one switching aggregation node.

[0113] In one embodiment of the present disclosure, the device also includes: a communication strategy adjustment module 905, which is used to obtain resource status information of each GPU node; build a network topology of the cluster service system based on the resource status information of each GPU node; and dynamically adjust the communication strategy based on the resource status information of each GPU node and the network topology of the cluster service system to ensure the communication performance of the cluster service system.

[0114] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0115] Refer to the following Fig. 9 The electronic device 1000 according to this embodiment of the present disclosure is described. Fig. 9 The electronic device 1000 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0116] Fig. 9 A block diagram of an electronic device in an embodiment of the present disclosure is shown. Fig. 9 The electronic device 1000 according to this embodiment of the present disclosure is described. Fig. 9 The electronic device 1000 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0117] like Fig. 9As shown, the electronic device 1000 is in the form of a general computing device. The components of the electronic device 1000 may include but are not limited to: the at least one processing unit 1010, the at least one storage unit 1020, and a bus 1030 connecting different system components (including the storage unit 1020 and the processing unit 1010).

[0118] The storage unit stores a program code, and the program code can be executed by the processing unit 1010, so that the processing unit 1010 performs the steps described in the "Exemplary Method" section of this specification according to various exemplary embodiments of the present disclosure. For example, the processing unit 1010 can perform the following steps of the above method embodiment: obtaining resource demand information of the target application for at least one resource; according to the resource demand information, scheduling corresponding resources from at least one pre-built resource pool and allocating them to the target application, wherein at least one resource pool is obtained after separating and decoupling the resources to be scheduled.

[0119] The storage unit 1020 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 10201 and / or a cache storage unit 10202 , and may further include a read-only storage unit (ROM) 10203 .

[0120] The storage unit 1020 may also include a program / utility 10204 having a set (at least one) of program modules 10205, such program modules 10205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0121] Bus 1030 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0122] The electronic device 1000 may also communicate with one or more external devices 1040 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 1000, and / or communicate with any device that enables the electronic device 1000 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 1050. Furthermore, the electronic device 1000 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 1060. As shown, the network adapter 1060 communicates with other modules of the electronic device 1000 via a bus 1030. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 1000, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0123] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0124] Based on the same inventive concept, the embodiment of the present disclosure also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, any of the above-mentioned separation and decoupling resource allocation methods is implemented. Since the implementation of the computer-readable storage medium embodiment can refer to the implementation of the above-mentioned method embodiment, the repeated parts will not be repeated.

[0125] More specific examples of computer-readable storage media in the present disclosure may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0126] In the present disclosure, a computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0127] Alternatively, the program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0128] In a specific implementation, the program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0129] Based on the same inventive concept, a computer program product is also provided in the embodiment of the present disclosure, including: a computer program or an instruction, which implements the separation and decoupling resource allocation method of any one of the above method embodiments when the computer program or the instruction is executed by a processor. Since the principle of solving the problem in the computer program product embodiment is similar to that in the above method embodiment, the implementation of the computer program product embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0130] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0131] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0132] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0133] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Claims

1. A separation and decoupling resource allocation method, characterized in that: include: Obtaining resource demand information of a target application for at least one resource; According to the resource demand information, corresponding resources are scheduled from at least one pre-built resource pool and allocated to the target application, wherein the at least one resource pool is obtained by separating and decoupling the resources to be scheduled.

2. The resource allocation method according to claim 1, characterized in that: The resources to be scheduled are resources provided by a cluster service system; the cluster service system comprises: a plurality of servers; Wherein, before scheduling corresponding resources from at least one pre-built resource pool according to the resource demand information and allocating them to the target application, the method further includes: Obtain resource information of at least one of the following resources provided by each server in the cluster service system: central processing resources, computing resources, storage resources, and communication interface resources; According to the resource information provided by each server in the cluster service system, the resources provided by each server in the cluster service system are separated and decoupled to construct at least one resource pool corresponding to the cluster service system.

3. The resource allocation method according to claim 2, characterized in that: After separating and decoupling the resources provided by each server in the cluster service system according to the resource information provided by each server in the cluster service system and constructing at least one resource pool corresponding to the cluster service system, the method further includes: Monitoring resource change information provided by each server in the cluster service system; At least one resource pool corresponding to the cluster service system is updated according to the resource change information provided by each server in the cluster service system.

4. The resource allocation method according to claim 3, characterized in that: Updating at least one resource pool corresponding to the cluster service system according to resource change information provided by each server in the cluster service system includes: When the resource information to be added in the cluster service system is monitored, the resource information to be added is identified, and at least one resource pool related to the resource information to be added in the cluster service system is updated; When the to-be-removed resource information in the cluster service system is monitored, the running resources in the cluster service system are migrated, and at least one resource pool related to the to-be-removed resource information in the cluster service system is updated.

5. The resource allocation method according to any one of claims 1 to 4, characterized in that: The target application is an AI application, the central processing resource is a CPU resource, the computing power resource is a GPU resource, the storage resource is a memory resource, and the communication interface resource is a switch resource; the resource pool corresponding to the cluster service system includes at least one of the following: a CPU pool, a GPU pool, a memory pool, and an I / O pool; The CPU pool includes: at least one CPU node; the GPU pool includes: at least one GPU node; the memory pool includes: at least one memory sharing node; the I / O pool includes: at least one exchange aggregation node.

6. The resource allocation method according to claim 5, characterized in that: The method further comprises: Get the resource status information of each GPU node; Constructing a network topology of the cluster service system according to the resource status information of each GPU node; According to the resource status information of each GPU node and the network topology of the cluster service system, the communication strategy is dynamically adjusted to ensure the communication performance of the cluster service system.

7. A separated and decoupled resource allocation device, characterized in that: include: An information acquisition module, used to acquire resource demand information of a target application for at least one resource; The resource allocation module is used to schedule corresponding resources from at least one pre-built resource pool according to the resource demand information and allocate them to the target application, wherein the at least one resource pool is obtained after separating and decoupling the resources to be scheduled.

8. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to execute the separated and decoupled resource allocation method as described in any one of claims 1 to 6 by executing the executable instructions.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the separated and decoupled resource allocation method described in any one of claims 1 to 6 is implemented.

10. A computer program product comprising: A computer program or instruction, characterized in that when the computer program or instruction is executed by a processor, it implements the separated and decoupled resource allocation method described in any one of claims 1 to 6.

Images