Cluster and communication method

Abstract

The invention discloses a cluster and a communication method, which can present various resources in a pooling manner and decouple each resource pool so as to improve the utilization rate of the resources. The cluster comprises a data processor pool and a plurality of computing resource pools of different types, the data processor pool comprises a plurality of data processors, and any one of the computing resource pools comprises a plurality of computing resources; the data processor pool is connected with the plurality of computing resource pools through a high-speed interconnection bus; the data processor pool is to manage communications between the plurality of computing resource pools and an exterior of the cluster, communications between the plurality of computing resource pools, and communications within any one of the computing resource pools.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of cloud technology, and in particular to a cluster and communication method. Background Art

[0002] In a cloud service system, cloud vendors add an offload card to the physical server in the system, thereby offloading some tasks (functions) of the processor in the physical server to the card for processing, thereby reducing the burden on the processor in the physical server and improving the performance of the entire cloud service system.

[0003] The current cloud service system includes multiple physical servers. For any one of the multiple physical servers, the physical server not only includes a series of necessary hardware resources such as a central processing unit, memory, network card, and hard disk, but also has an additional offload card. The offload card has a dedicated processor and memory, so the offload card can replace the central processing unit to complete a series of tasks such as resource scheduling, network forwarding, storage front-end, etc., so the central processing unit of the physical server can handle more additional tasks.

[0004] Due to the local peak of tasks in the cloud service system, that is, some physical servers in the system need to process a large number of tasks, while other physical servers only need to process a small number of tasks. This will lead to high utilization of offload cards on some servers and low utilization of offload cards on other physical servers, resulting in a waste of resources. Summary of the invention

[0005] The embodiments of the present application provide a cluster and communication method, which can present various resources in a pooled manner and decouple various resource pools, thereby improving resource utilization.

[0006] A first aspect of an embodiment of the present application provides a cluster, which includes a data processor pool and multiple computing resource pools of different types, wherein the data processor pool includes multiple data processors, and any one of the multiple computing resource pools includes multiple computing resources.

[0007] The data processor pool is connected to multiple computing resource pools through a high-speed interconnect bus. That is to say, any computing resource pool can communicate with the outside of the cluster through the high-speed interconnect bus and the data processor pool. Multiple computing resource pools can also communicate with each other through the high-speed interconnect bus and the data processor pool. The inside of any computing resource pool can also communicate with each other through the high-speed interconnect bus and the data processor pool. It can be seen that the data processor pool can be used to manage the communication between multiple computing resource pools and the outside of the cluster, the communication between multiple computing resource pools, and the communication within any computing resource pool.

[0008] From the above cluster, we can see that: in this cluster, various types of computing resources are deployed in a pooled manner, and multiple computing resource pools are decoupled. They can be configured according to the type of cloud services required by the cluster, thereby improving the performance and resource utilization of the cluster. In this cluster, data processors are also deployed in a pooled manner to form a data processor pool, and serve as the communication dispatch center of the cluster to achieve centralized allocation and utilization of each data processor in the pool, thereby improving the utilization of each data processor.

[0009] In one possible implementation, the data processor pool and the plurality of computing resource pools are arranged in a plurality of cabinets, and the cabinet used to arrange the data processor pool and the cabinet used to arrange any one of the computing resource pools are different cabinets. In the aforementioned implementation, the data processor pool and the plurality of computing resource pools are arranged in a plurality of cabinets respectively. Among them, the cabinet used to arrange the data processor pool and the cabinet used to arrange any one of the computing resource pools are usually different cabinets. Similarly, the cabinet used to arrange a computing resource pool and the cabinet used to arrange another computing resource pool are usually also different cabinets.

[0010] In a possible implementation, multiple data processors include a general processing unit and multiple accelerated processing units, and the general processing unit is used to allocate at least one of the multiple accelerated processing units to a computing resource with communication requirements in multiple computing resource pools, and the at least one accelerated processing unit allocated to the computing resource is used to meet the communication requirements of the computing resource. In the above implementation, when any computing resource in the multiple computing resource pools has a communication requirement, the general processing unit can allocate at least one of the multiple accelerated processing units to the computing resource, that is, establish a connection between the computing resource and the at least one accelerated processing unit allocated to the computing resource, so the at least one accelerated processing unit allocated to the computing resource can meet the communication requirements of the computing resource. In a possible implementation, multiple data processors include a general processing unit and multiple accelerated processing units; the general processing unit is used to construct a first communication path between multiple computing resource pools and the outside of the cluster, a first communication path between multiple computing resource pools, and a first communication path inside any one computing resource pool. Multiple accelerated processing units are used to construct a second communication path between multiple computing resource pools and the outside of the cluster, a second communication path between multiple computing resource pools, and a second communication path inside any one computing resource pool. In the aforementioned implementation, the multiple data processors include one or more general processing units and multiple accelerated processing units, wherein the one or more general processing units are used to construct a first communication path between any one of the multiple computing resource pools and the outside of the cluster, a first communication path between computing resources in any two computing resource pools, and a first communication path between two computing resources in any one computing resource pool. The multiple accelerated processing units are used to construct a second communication path between any one of the multiple computing resource pools and the outside of the cluster, a second communication path between computing resources in any two computing resource pools, and a second communication path between two computing resources in any one computing resource pool.

[0011] In one possible implementation, a general processing unit and multiple accelerated processing units are deployed inside a data processor pool. In the aforementioned implementation, among the multiple data processors in the data processor pool, any data processor has a general processing unit and an accelerated processing unit inside, in which case the general processing unit and the accelerated processing unit are coupled to each other.

[0012] In one possible implementation, multiple acceleration processing units are deployed inside a data processor pool, and general processing units are deployed outside the data processor pool, and the outside of the data processor pool is the inside of at least one computing resource pool. In the aforementioned implementation, among the multiple data processors included in the data processor pool, any data processor has only one acceleration processing unit inside. The number of general processing units is usually one, and the general processing unit can be deployed remotely, that is, the general processing unit is not deployed in any data processor of the data processor pool, but is deployed in one or some computing resource pools. In this case, the general processing unit and the acceleration processing unit are decoupled.

[0013] In one possible implementation, the first communication path between multiple computing resource pools and the outside of the cluster has any computing resource in the multiple computing resource pools and the outside of the cluster as endpoints, passing through a general processing unit and at least one of the multiple acceleration processing units; the first communication path between multiple computing resource pools has computing resources in any two computing resource pools as endpoints, passing through a general processing unit and at least one of the multiple acceleration processing units; the first communication path inside any one computing resource pool has two computing resources in any one computing resource pool as endpoints, passing through a general processing unit and at least one of the multiple acceleration processing units. In the aforementioned implementation, for the first communication path, due to the different starting points and end points, it can be specifically divided into three categories: the first communication path between multiple computing resource pools and the outside of the cluster, the first communication path between multiple computing resource pools, and the first communication path inside any one computing resource pool, but these three types of first communication paths all pass through a general processing unit and at least one acceleration processing unit.

[0014] In one possible implementation, the second communication path between multiple computing resource pools and the outside of the cluster has any computing resource in the multiple computing resource pools and the outside of the cluster as endpoints, passing through at least one of the multiple acceleration processing units in the middle; the second communication path between multiple computing resource pools has computing resources in any two computing resource pools as endpoints, passing through at least one of the multiple acceleration processing units in the middle; the second communication path inside any one computing resource pool has two computing resources in any one computing resource pool as endpoints, passing through at least one of the multiple acceleration processing units in the middle. In the aforementioned implementation, for the second communication path, due to the different starting points and end points, it can be specifically divided into three categories: the second communication path between multiple computing resource pools and the outside of the cluster, the second communication path between multiple computing resource pools, and the second communication path inside any one computing resource pool, but these three types of second communication paths only pass through at least one acceleration processing unit in the middle.

[0015] In a possible implementation, the general processing unit establishes connections between multiple accelerated processing units and computing resources in multiple computing resource pools by issuing tokens. For any one of the multiple accelerated processing units, the number of tokens that the accelerated processing unit can receive is equal to the number of computing resources that the accelerated processing unit can connect to. In the above implementation, in a round of token issuance, the general processing unit sends a token issuance message to the first accelerated processing unit, and the accelerated processing unit applies for a token based on the message and establishes a connection with a computing resource. Then, the accelerated processing unit can determine whether its own token bucket is full. If so, it means that all tokens have been taken, and then notify the general processing unit so that the general processing unit removes the accelerated processing unit from the multiple accelerated processing units. Then, the accelerated processing unit can continue to send the token issuance message to the next accelerated processing unit until it returns to the general processing unit. At this point, the token issuance of this round is completed, and the token issuance of the next round can be started. Then, when the general processing unit completes the token issuance for all accelerated processing units, any one of the multiple accelerated processing units completes the connection with at least one computing resource. It can be seen that, for any one of the multiple acceleration processing units, the number of tokens that the acceleration processing unit can receive is equal to the number of computing resources that can be connected to the acceleration processing unit.

[0016] In one possible implementation, when the high-speed interconnect bus supported by the acceleration processing unit is different from the high-speed interconnect bus supported by the computing resource, the computing resource and the acceleration processing unit are connected through the high-speed interconnect bus supported by the computing resource, the conversion device and the high-speed interconnect bus supported by the acceleration processing unit. In the aforementioned implementation, when the high-speed interconnect bus supported by the acceleration processing unit is the same as the high-speed interconnect bus supported by the computing resource, the computing resource and the acceleration processing unit can be connected through the high-speed interconnect bus. When the high-speed interconnect bus supported by the acceleration processing unit is different from the high-speed interconnect bus supported by the computing resource, the computing resource and the acceleration processing unit are connected through the high-speed interconnect bus supported by the computing resource, the conversion device and the high-speed interconnect bus supported by the acceleration processing unit, wherein the conversion device is used to realize the conversion between the high-speed interconnect bus supported by the computing resource and the high-speed interconnect bus supported by the acceleration processing unit.

[0017] In one possible implementation, the multiple computing resource pools include at least two of the following: a central processing unit (CPU) pool, a neural network processor (NPU) pool, a graphics processing unit (GPU) pool, a memory pool, and a solid state disk (SSD) pool, the CPU pool includes multiple CPUs, the NPU pool includes multiple NPUs, the GPU pool includes multiple GPUs, the memory pool includes multiple memories, and the SSD pool includes multiple SSDs.

[0018] In one possible implementation, any computing resource pool includes multiple computing resources for deploying cloud instances that provide cloud services, and the cloud instances include any of the following: computing resources, virtual machines, containers, and micro virtual machines.

[0019] A second aspect of an embodiment of the present application provides a communication method, which is implemented through a cluster. The cluster includes a data processor pool and multiple computing resource pools of different types. The data processor pool includes multiple data processors. Any one of the multiple computing resource pools includes multiple computing resources. The data processor pool is connected to the multiple computing resource pools through a high-speed interconnection bus. The method includes: the data processor pool manages the communication between the multiple computing resource pools and the outside of the cluster, the communication between the multiple computing resource pools, and the internal communication of any one of the computing resource pools.

[0020] From the above method, it can be seen that in this cluster, various types of computing resources are deployed in a pooled manner, and multiple computing resource pools are decoupled. They can be configured according to the type of cloud services required by the cluster, thereby improving the performance and resource utilization of the cluster. In this cluster, data processors are also deployed in a pooled manner to form a data processor pool, and serve as the communication scheduling center of the cluster to achieve centralized allocation and utilization of each data processor in the pool, thereby improving the utilization of each data processor.

[0021] In a possible implementation, the data processor pool and the multiple computing resource pools are set in multiple cabinets, and the cabinet used to set the data processor pool is different from the cabinet used to set any computing resource pool.

[0022] In one possible implementation, multiple data processors include a general processing unit and multiple accelerated processing units, and the method also includes: the general processing unit allocates at least one of the multiple accelerated processing units to computing resources with communication requirements in multiple computing resource pools, so that the at least one accelerated processing unit allocated to the computing resources meets the communication requirements of the computing resources.

[0023] In one possible implementation, multiple data processors include general processing units and multiple accelerated processing units; the data processor pool manages the communication between multiple computing resource pools and the outside of the cluster, the communication between multiple computing resource pools, and the internal communication of any computing resource pool, including: the general processing unit builds a first communication path between multiple computing resource pools and the outside of the cluster, a first communication path between multiple computing resource pools, and a first communication path inside any computing resource pool. Multiple accelerated processing units build a second communication path between multiple computing resource pools and the outside of the cluster, a second communication path between multiple computing resource pools, and a second communication path inside any computing resource pool.

[0024] In one possible implementation, a general processing unit and a plurality of accelerated processing units are deployed inside a data processor pool.

[0025] In a possible implementation, multiple acceleration processing units are deployed inside a data processor pool, and general processing units are deployed outside the data processor pool, where the outside of the data processor pool is the inside of at least one computing resource pool.

[0026] In one possible implementation, a first communication path between multiple computing resource pools and the outside of the cluster has any one computing resource in the multiple computing resource pools and the outside of the cluster as endpoints, passing through a general processing unit and at least one of the multiple acceleration processing units; a first communication path between multiple computing resource pools has computing resources in any two computing resource pools as endpoints, passing through a general processing unit and at least one of the multiple acceleration processing units; a first communication path inside any one computing resource pool has two computing resources in any one computing resource pool as endpoints, passing through a general processing unit and at least one of the multiple acceleration processing units.

[0027] In one possible implementation, the second communication path between multiple computing resource pools and the outside of the cluster has any one computing resource in the multiple computing resource pools and the outside of the cluster as endpoints, passing through at least one acceleration processing unit among the multiple acceleration processing units; the second communication path between multiple computing resource pools has computing resources in any two computing resource pools as endpoints, passing through at least one acceleration processing unit among the multiple acceleration processing units; the second communication path inside any computing resource pool has two computing resources in any one computing resource pool as endpoints, passing through at least one acceleration processing unit among the multiple acceleration processing units.

[0028] In one possible implementation, the method also includes: the general processing unit establishes connections between multiple acceleration processing units and computing resources in multiple computing resource pools by issuing tokens, and for any one of the multiple acceleration processing units, the number of tokens that the acceleration processing unit can receive is equal to the number of computing resources that can be connected to the acceleration processing unit.

[0029] In one possible implementation, when the high-speed interconnect bus supported by the acceleration processing unit is different from the high-speed interconnect bus supported by the computing resources, the computing resources and the acceleration processing unit are connected via the high-speed interconnect bus supported by the computing resources, the conversion device, and the high-speed interconnect bus supported by the acceleration processing unit.

[0030] In one possible implementation, the multiple computing resource pools include at least two of the following: a CPU pool, an NPU pool, a GPU pool, a memory pool, and an SSD pool. The CPU pool includes multiple CPUs, the NPU pool includes multiple NPUs, the GPU pool includes multiple GPUs, the memory pool includes multiple memories, and the SSD pool includes multiple SSDs.

[0031] In one possible implementation, any computing resource pool includes multiple computing resources for deploying cloud instances that provide cloud services, and the cloud instances include any of the following: computing resources, virtual machines, containers, and micro virtual machines.

[0032] A third aspect of an embodiment of the present application provides a cloud service system, which includes multiple clusters as described in the first aspect or any possible implementation method of the first aspect.

[0033] A fourth aspect of an embodiment of the present application provides a computing device cluster, the computing device cluster includes at least one computing device, each computing device includes a processor and a memory: the memory is used to store instructions; the processor is used to enable the computing device cluster to execute the method described in the second aspect or any possible implementation method of the second aspect according to the instructions.

[0034] A fifth aspect of an embodiment of the present application provides a computer storage medium, which stores one or more instructions. When the instructions are executed by one or more computers, the one or more computers implement the method described in the second aspect or any possible implementation method of the second aspect.

[0035] A sixth aspect of the embodiments of the present application provides a computer program product, which stores instructions. When the instructions are executed by a computer, the computer implements the method described in the second aspect or any possible implementation method of the second aspect.

[0036] In the cluster provided by the embodiment of the present application, various types of computing resources are deployed in a pooled manner, and multiple computing resource pools are decoupled, and can be configured according to the type of cloud service required to be provided by the cluster, thereby improving the performance and resource utilization of the cluster. In the cluster, data processors are also deployed in a pooled manner, which can form a data processor pool and serve as the communication dispatch center of the cluster to achieve centralized allocation and utilization of each data processor in the pool, thereby improving the utilization of each data processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A schematic diagram of the structure of a cloud service system provided in an embodiment of the present application;

[0038] Figure 2 A schematic diagram of the structure of a cluster provided in an embodiment of the present application;

[0039] Figure 3 Another schematic diagram of the structure of the cluster provided in the embodiment of the present application;

[0040] Figure 4 Another schematic diagram of the structure of the cluster provided in the embodiment of the present application;

[0041] Figure 5 Another schematic diagram of the structure of the cluster provided in the embodiment of the present application;

[0042] Figure 6 A schematic diagram of a structure of token issuance provided in an embodiment of the present application;

[0043] Figure 7 A flowchart of token issuance provided in an embodiment of the present application;

[0044] Figure 8 Another schematic diagram of the structure of the cluster provided in the embodiment of the present application;

[0045] Fig. 9 A schematic diagram of a communication method provided in an embodiment of the present application;

[0046] Fig.10 A schematic diagram of a computing device provided in an embodiment of the present application;

[0047] Fig.11 A schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application;

[0048] Fig.12 A schematic diagram of computer devices in a computer cluster provided in an embodiment of the present application being connected via a network. DETAILED DESCRIPTION

[0049] The embodiments of the present application provide a cluster and communication method, which can present various resources in a pooled manner and decouple various resource pools, thereby improving resource utilization.

[0050] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0051] In a cloud service system, cloud vendors add an offload card to the physical server in the system, thereby offloading some tasks (functions) of the processor in the physical server to the card for processing, thereby reducing the burden on the processor in the physical server and improving the performance of the entire cloud service system.

[0052] The current cloud service system includes multiple physical servers. For any one of the multiple physical servers, the physical server not only includes a series of necessary hardware resources such as a central processing unit, memory, network card, and hard disk, but also has an additional offload card. The offload card has a dedicated processor and memory, so the offload card can replace the central processing unit to complete a series of tasks such as resource scheduling, network forwarding, storage front-end, etc., so the central processing unit of the physical server can handle more additional tasks.

[0053] Due to the local peak of tasks in the cloud service system, that is, some physical servers in the system need to process a large number of tasks, while other physical servers only need to process a small number of tasks. This will lead to high utilization of offload cards on some servers and low utilization of offload cards on other physical servers, resulting in a waste of resources.

[0054] Furthermore, when the offload card is required to take on additional tasks (functions), it is often difficult to expand the new functions of the offload card due to the limited resources of the offload card itself, thereby limiting the application of the entire cloud service system in various new scenarios.

[0055] Furthermore, if a physical server is used to provide cloud services in an artificial intelligence (AI) scenario, the neural network model in the AI ​​scenario often has an independent data communication plane during training, and the offload card on the physical server cannot be used directly. It is necessary to add a new offload card for separate model training, which leads to a sharp increase in costs.

[0056] Furthermore, due to process limitations, the specifications of the offload cards on the physical servers cannot be updated in real time as the industry advances. In other words, the offload cards on the physical servers cannot be replaced quickly, which will affect the performance of the entire cloud service system.

[0057] In order to solve the above problems, the present application embodiment provides a cloud service system, such as Figure 1 As shown ( Figure 1 A structural diagram of a cloud service system provided for an embodiment of the present application) includes multiple clusters (pods), and the multiple clusters can be connected through a certain network (for example, a data center network (DCN), the data center network can be Ethernet, etc.). Among these multiple clusters, a cluster can provide one or more types of cloud services to the outside. For example, a cluster can provide AI services on the cloud to the outside, and another example is that a cluster can provide traditional cloud computing services to the outside, and another example is that a cluster can provide traditional cloud storage services to the outside, and so on.

[0058] For any of the multiple clusters, if Figure 2 As shown ( Figure 2 A schematic diagram of a cluster structure provided in an embodiment of the present application) is shown in FIG. 1 , which may also be referred to as a hypernode. The cluster may include multiple hardware pools at the rack level, including a data processing unit (DPU) pool and multiple computing resource pools of different types (also referred to as computing power resource pools). The following introduces the data processor and computing resource pools respectively:

[0059] For any one of the multiple computing resource pools of different types, the computing resource pool may include multiple computing resources of the same type. For example, the computing resource pool may be a central processing unit (CPU) pool, and the CPU pool includes multiple CPUs. In another example, the computing resource pool may be a neural network processing unit (NPU) pool, and the NPU pool includes multiple NPUs. In another example, the computing resource pool may be a graphics processing unit (GPU), and the GPU pool includes multiple GPUs. In another example, the computing resource pool may be a tensor processing unit (TPU), and the TPU pool includes multiple TPUs. In another example, the computing resource pool may be a memory pool, and the memory pool includes multiple memories. In another example, the computing resource pool may be a solid state disk (SSD) pool, and the SSD pool includes multiple SSDs, and so on. It can be seen that the multiple computing resource pools contained in the cluster may be any combination of computing resource pools of at least one type mentioned above. It should be noted that the main types of computing resource pools contained in the cluster are related to the types of cloud services that the cluster can provide. For example, if the cluster is used to provide AI services, the multiple computing resource pools contained in the cluster are mainly CPU pools, NPU pools, and GPU pools. For another example, if the cluster is used to provide cloud computing services for general loads, the multiple computing resource pools contained in the cluster are mainly CPU pools. For another example, if the cluster is used to provide cloud storage services, the multiple computing resource pools contained in the cluster are mainly SSD pools, and so on.

[0060] The data processor pool includes multiple data processors, which can be presented in various forms such as software defined infrastructure (SDI) chips, application specific integrated circuits (ASIC) chips, field programmable gate arrays (FPGA) chips, or programmable logic devices (PLD) chips. It should be noted that in the cluster, the data processor pool can be used as an unloading card pool for multiple computing resource pools, which can be used to coordinate the communication between multiple computing resource pools and the outside of the cluster (for example, another cluster, or the traditional data center, available zone, etc. in the cloud service system), the communication between any two computing resource pools, and the internal communication of any resource pool. Therefore, the data processor pool can be regarded as the exit of the cluster, for accessing the outside of the cluster, and the data processor pool can also be regarded as a switching interface between multiple computing resource pools, for connecting multiple computing resource pools, and the data processor pool can also be regarded as a switching interface between multiple computing resources contained in any one computing resource pool, for connecting multiple computing resources contained in the computing resource pool.

[0061] It is worth noting that the data processor pool can be connected to multiple computing resource pools through a high-speed interconnect bus (which may include a high-speed interconnect switch and a high-speed interconnect communication connection line, etc.), that is, any computing resource pool can communicate with the outside of the cluster through a high-speed interconnect bus and a data processor pool, and multiple computing resource pools can also communicate with each other through a high-speed interconnect bus and a data processor pool, and the inside of any computing resource pool can also communicate with each other through a high-speed interconnect bus and a data processor pool. It can be seen that the data processor pool can manage the communication between multiple computing resource pools and the outside of the cluster, the communication between any two computing resource pools, and the communication inside any computing resource pool based on the high-speed interconnect bus. Among them, the high-speed interconnect bus can be any of the following: a peripheral component interconnect express (PCIe) bus, an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a computer express link (CXL), a cache coherent interconnect for accelerators (CCIX), and the like.

[0062] It is also worth noting that any computing resource contained in any computing resource pool can be used to deploy one or more cloud instances that provide cloud services. These cloud instances include any of the following: computing resources (that is, bare metal), virtual machines (VM), containers (docker), and micro virtual machines (microVM). It can be seen that any computing resource pool can be deployed with a cloud instance cluster, which can be used to provide a certain type of cloud service to the outside world.

[0063] It is also worth noting that, from the perspective of physical hardware, the data processor pool and multiple computing resource pools are respectively set in multiple cabinets (racks). Generally, the cabinet used to set up the data processor pool (that is, the cabinet where the multiple data processors contained in the data processor pool are located) and the cabinet used to set up any computing resource pool (that is, the cabinet where the multiple computing resources contained in the computing resource pool are located) are usually different cabinets. Similarly, the cabinet used to set up a computing resource pool (that is, the cabinet where the multiple computing resources contained in the computing resource pool are located) and the cabinet used to set up another computing resource pool (that is, the cabinet where the multiple computing resources contained in the computing resource pool are located) are usually different cabinets. It can be seen that there is physical isolation between the data processor pool and the multiple computing resource pools, and the isolation is at the cabinet level.

[0064] For example, Figure 3 As shown ( Figure 3 Another structural diagram of the cluster provided in the embodiment of the present application), assuming that the cloud service system includes p clusters such as cluster 1, cluster 2, cluster 3, ..., cluster p. Cluster 1 includes SDI card pool 1, CPU pool 1 and NPU pool 1, SDI card pool 1 includes m SDI cards connected to each other, CPU pool 1 includes k CPUs connected to each other, NPU pool 1 includes h NPUs connected to each other, and m SDI cards, k CPUs and h NPUs are all connected to the UB switch, that is, SDI card pool 1 is connected to CPU pool 1 and NPU pool 1 through the UB switch, CPU pool 1 and NPU pool 1 are also connected through the UB switch, the CPUs in CPU pool 1 are also connected through the UB switch, and the CPUs in NPU pool 1 are also connected through the UB switch. Among them, p, m, k, h are all greater than 1, and the sum of k and h is equal to n. It can be seen that the ratio between SDI card and xPU (CPU+NPU) is m:n. Of course, cluster 2 can also include SDI card pool 2, CPU pool 2, NPU pool 2, GPU pool 2, etc. The same is true for clusters 3 to cluster p, which will not be repeated here.

[0065] based on Figure 3It can be seen that SDI card pool 1, SDI card pool 2, ..., SDI card pool p can be connected to DCN, so in cluster 1, the CPU in CPU pool 1 and the NPU in NPU pool 1 can communicate with cluster 2, cluster 3, ..., cluster p and the virtual private cloud in the system, that is, the outside of cluster 1, through the SDI card in SDI card pool 1. Further, in cluster 1, the CPU in CPU pool 1 can communicate with the NPU in NPU pool 1 through the SDI card in SDI card pool 1. Further, in cluster 1, the CPUs in CPU pool 1 can communicate with each other through the SDI card in SDI card pool 1. Similarly, the NPUs in NPU pool 1 can also communicate with each other through the SDI card in SDI card pool 1. Of course, the same is true for cluster 2 to cluster p, which will not be repeated here.

[0066] Specifically, in the cluster, the multiple data processors included in the data processor pool can be presented in the following ways:

[0067] The multiple data processors include one or more general processing units (service procesors unit, SPU) and multiple accelerated processing units (network processor unit, NPU). It should be noted that the number of data processors is usually equal to the number of accelerated processing units, and the number of data processors is usually greater than or equal to the number of general processing units. That is to say, when the number of general processing units is one, the general processing unit is used to uniformly manage multiple accelerated processing units. When the number of general processing units is multiple, one general processing unit is used to manage one accelerated processing unit among multiple accelerated processing units.

[0068] Among them, one or more general processing units are used to build a first communication path (also called a slow communication path) between any one of the computing resources in the multiple computing resource pools and the outside of the cluster, a first communication path between computing resources in any two computing resource pools, and a first communication path between two computing resources in any one computing resource pool. Multiple acceleration processing units are used to build a second communication path (also called a fast communication path) between any one of the computing resources in the multiple computing resource pools and the outside of the cluster, a second communication path between computing resources in any two computing resource pools, and a second communication path between two computing resources in any one computing resource pool.

[0069] More specifically, in the cluster, the general processing unit and the accelerated processing unit can be configured in the following ways:

[0070] (1) The general processing unit and the accelerated processing unit are coupled to each other. In this case, among the multiple data processors included in the data processor pool, any data processor has a general processing unit and an accelerated processing unit inside. It should be noted that for any data processor, the general processing unit and the accelerated processing unit in the data processor can be integrated into the same chip or deployed in two different chips respectively, which is not limited here.

[0071] Still as in the above example, Figure 4 As shown ( Figure 4 Another schematic diagram of the structure of the cluster provided in the embodiment of the present application is shown in FIG. Figure 4 is Figure 3 ), in cluster 1, for the m SDI cards included in SDI card pool 1, each SDI card has 1 SPU and 1 NPU, and 1 SPU is used to manage the corresponding 1 NPU.

[0072] (2) Decoupling between general processing units and accelerated processing units. In this case, among the multiple data processors included in the data processor pool, any data processor has only one accelerated processing unit inside. The number of general processing units is usually one, and the general processing unit can be deployed remotely, that is, the general processing unit is not deployed in any data processor in the data processor pool, but is deployed in one or more computing resource pools. For example, a computing resource in a computing resource pool can be directly used as a general processing unit, etc.

[0073] Still as in the above example, Figure 5 As shown ( Figure 5 Another schematic diagram of the structure of the cluster provided in the embodiment of the present application is shown in FIG. Figure 5 is Figure 3 In cluster 1, for the m SDI cards contained in SDI card pool 1, each SDI card has one NPU. It is worth noting that the SPU is not deployed in SDI card pool 1, but one of the CPUs in CPU pool 1 is directly used as the SPU, which is used to uniformly manage the m NPUs.

[0074] More specifically, in the cluster, the general processing unit and the accelerated processing unit may construct the first communication path and the second communication path in the following manner:

[0075] (1) Assume that there is a message to be sent to a computing resource in a computing resource pool. When an acceleration processing unit receives the message, it can first determine whether there is a flow table entry matching the message from its own flow table. If not, the acceleration processing unit sends the message to the general processing unit that manages the acceleration processing unit. Then, the general processing unit can determine the flow table entry matching the message from its own flow table, and send the flow table entry to the acceleration processing unit, so that the acceleration processing unit sends the message to the computing resource based on the flow table entry, and updates the flow table entry to its own flow table. Therefore, if there is a new message to be sent to the computing resource later, the acceleration processing unit can directly send the new message to the computing resource based on the flow table entry. Based on the above process, it can be known that the communication path used to transmit the message in this process passes through a general processing unit and at least one acceleration processing unit between the starting point and the end point of the communication path (in the above process, the acceleration processing unit may also pass through other acceleration processing units when sending the message to the computing resource), so the communication path can be regarded as the first communication path. Due to the difference in the starting point and the end point, the first communication path can be divided into the following three types:

[0076] (1.1) If the starting point of a first communication path is outside the cluster, the end point is any computing resource in multiple computing resource pools, and passes through a general processing unit and at least one accelerated processing unit, then the first communication path can be regarded as the first communication path between the computing resource and the outside of the cluster. Figure 5 In the example shown, if VM2 on a CPU in CPU pool 2 (located in cluster 2) needs to send a message to VM1 on a CPU in CPU pool 1 (located in cluster 1), after the message is received by an NPU in SDI card pool 1, since there is no flow table entry matching the message in the flow table of the NPU itself, the NPU sends the message to the SPU. After the SPU sends the flow table entry matching the message to the NPU, the NPU can update its own flow table and send the message to VM1 based on the flow table entry. It can be seen that the communication path used to transmit the message passes through VM2 (the CPU where it is located), the NPU, the SPU, and VM1 (the CPU where it is located) in sequence.

[0077] (1.2) If the starting point of a first communication path is a computing resource in one of the multiple computing resource pools, and the ending point is a computing resource in another of the multiple computing resource pools, and passes through a general processing unit and at least one accelerated processing unit, then the first communication path can be regarded as the first communication path between the computing resources in the two computing resource pools. Figure 5In the example shown, if VM3 on a CPU in CPU pool 1 needs to send a message to VM4 on an NPU in NPU pool 1, after the message is received by an NPU in SDI card pool 1, since the flow table of the NPU itself does not have a flow table entry that matches the message, the NPU sends the message to the SPU. After the SPU sends the flow table entry that matches the message to the NPU, the NPU can update its own flow table and send the message to VM4 based on the flow table entry. It can be seen that the communication path used to transmit the message passes through VM3 (the CPU where it is located), the NPU, the SPU, and VM4 (the CPU where it is located) in sequence.

[0078] (1.3) If the starting point of a first communication path is a computing resource in a computing resource pool among multiple computing resource pools, and the ending point is another computing resource in the computing resource pool, and passes through a general processing unit and at least one accelerated processing unit in the middle, then the first communication path can be regarded as the first communication path between two computing resources in the computing resource pool. Figure 5 In the example shown, if VM5 on a CPU in CPU pool 1 needs to send a message to VM6 on another CPU in CPU pool 1, after the message is received by an NPU in SDI card pool 1, since the flow table of the NPU itself does not have a flow table entry that matches the message, the NPU sends the message to the SPU. After the SPU sends the flow table entry that matches the message to the NPU, the NPU can update its own flow table and send the message to VM6 based on the flow table entry. It can be seen that the communication path used to transmit the message passes through VM4 (the CPU where it is located), the NPU, the SPU, and VM6 (the CPU where it is located) in sequence.

[0079] (2) Assume that there is a message to be sent to a computing resource in a computing resource pool. When an acceleration processing unit receives the message, it can first determine from its own flow table whether there is a flow table entry that matches the message. If so, the acceleration processing unit directly sends the message to the computing resource based on the flow table entry. Based on the above process, it can be known that the communication path used to transmit the message does not pass through a general processing unit between the starting point and the end point of the communication path, but only passes through at least one acceleration processing unit (in the above process, the acceleration processing unit may also pass through other acceleration processing units in the process of sending the message to the computing resource). Therefore, the communication path can be regarded as a second communication path. Due to the difference in the starting point and the end point, the second communication path can be divided into the following three types:

[0080] (2.1) If the starting point of a second communication path is outside the cluster, the end point is any computing resource in multiple computing resource pools, and passes through at least one accelerated processing unit, then the second communication path can be regarded as the second communication path between the computing resource and the outside of the cluster. Figure 5 In the example shown, if VM2 on a CPU in CPU pool 2 (located in cluster 2) needs to send a message to VM1 on a CPU in CPU pool 1 (located in cluster 1), after the message is received by an NPU in SDI card pool 1, since the flow table of the NPU itself has a flow table entry that matches the message, the NPU sends the message to VM1 based on the flow table entry. It can be seen that the communication path for transmitting the message passes through VM2 (the CPU where it is located), the NPU, and VM1 (the CPU where it is located) in sequence.

[0081] (2.2) If the starting point of a second communication path is a computing resource in one of the multiple computing resource pools, and the ending point is a computing resource in another of the multiple computing resource pools, and passes through at least one accelerated processing unit, then the second communication path can be regarded as the second communication path between the computing resources in the two computing resource pools. Figure 5 In the example shown, if VM3 on a CPU in CPU pool 1 needs to send a message to VM4 on an NPU in NPU pool 1, after the message is received by an NPU in SDI card pool 1, since the flow table of the NPU itself has a flow table entry that matches the message, the NPU sends the message to VM4 based on the flow table entry. It can be seen that the communication path for transmitting the message passes through VM3 (the CPU where it is located), the NPU, and VM4 (the CPU where it is located) in sequence.

[0082] (2.3) If the starting point of a second communication path is a computing resource in a computing resource pool among multiple computing resource pools, the ending point is another computing resource in the computing resource pool, and passes through at least one accelerated processing unit in the middle, then the second communication path can be regarded as the second communication path between the two computing resources in the computing resource pool. Figure 5 In the example shown, if VM5 on a CPU in CPU pool 1 needs to send a message to VM6 on another CPU in CPU pool 1, after the message is received by an NPU in SDI card pool 1, since the flow table of the NPU itself has a flow table entry that matches the message, the NPU sends the message to VM6 based on the flow table entry. It can be seen that the communication path for transmitting the message passes through VM4 (the CPU where it is located), the NPU, and VM6 (the CPU where it is located) in sequence.

[0083] It should be understood that in the embodiments of the present application, only a computing resource in the cluster is used as the end point of the communication path, and another computing resource outside the cluster or in the cluster is used as the starting point of the communication path for schematic introduction. In actual applications, the two can also be switched, that is, a computing resource in the cluster can also be the starting point of the communication path, and another computing resource outside the cluster or in the cluster is the end point of the communication path, which will not be repeated here.

[0084] More specifically, in the cluster, the general processing unit can also manage multiple accelerated processing units in the following ways:

[0085] When any computing resource in a plurality of computing resource pools has a communication demand (for example, the general processing unit determines that the computing resource has been started, which can be regarded as the computing resource having a communication demand, and the general processing unit can actively allocate an acceleration processing unit to the computing resource. For another example, the computing resource can notify the general processing unit that it has a communication demand, and the computing resource can apply to the general processing unit for the required acceleration processing unit, etc.), the general processing unit can allocate at least one of the plurality of acceleration processing units to the computing resource, that is, establish a connection between the computing resource and the at least one acceleration processing unit allocated to the computing resource, so that the at least one acceleration processing unit allocated to the computing resource can meet the communication demand of the computing resource.

[0086] For example, the general processing unit can establish connections between multiple acceleration processing units and computing resources in multiple computing resource pools by issuing tokens. The token issuance process is as follows: Figure 6 and Figure 7 As shown ( Figure 6 A schematic diagram of a structure of token issuance provided in an embodiment of the present application, Figure 7 A flow chart of token issuance provided for an embodiment of the present application) is provided, assuming that the cluster has a general processing unit, and the general processing unit serves as a token issuance module, and each acceleration processing unit serves as a token receiving module, wherein each acceleration processing unit has a token bucket, and the capacity of the token bucket is limited, that is, the tokens that each acceleration processing unit can receive are limited.

[0087] Since the token issuance process is multi-round, and the process of each round is similar, one of the rounds is introduced below. In a round of token issuance, the general processing unit sends a token issuance message to the first acceleration processing unit, and the acceleration processing unit applies for a token based on the message and establishes a connection with a computing resource. Then, the acceleration processing unit can determine whether its own token bucket is full. If so, it means that all tokens have been taken, and then notify the general processing unit so that the general processing unit removes the acceleration processing unit from multiple acceleration processing units. Then, the acceleration processing unit can continue to send the token issuance message to the next acceleration processing unit until it returns to the general processing unit. At this point, the token issuance of this round is completed, and the token issuance of the next round can be started. Then, when the general processing unit completes the token issuance for all acceleration processing units, any one of the multiple acceleration processing units completes the connection with at least one computing resource. It can be seen that for any one of the multiple acceleration processing units, the number of tokens that the acceleration processing unit can receive is equal to the number of computing resources that can be connected to the acceleration processing unit.

[0088] More specifically, in the cluster, the accelerated processing units and computing resources can be connected in the following ways:

[0089] (1) When the high-speed interconnect bus supported by the acceleration processing unit is the same as the high-speed interconnect bus supported by the computing resource, the computing resource and the acceleration processing unit can be connected via the high-speed interconnect bus. Figure 4 and Figure 5 Example shown.

[0090] (2) When the high-speed interconnect bus supported by the acceleration processing unit is different from the high-speed interconnect bus supported by the computing resource, the computing resource and the acceleration processing unit are connected through the high-speed interconnect bus supported by the computing resource, the conversion device and the high-speed interconnect bus supported by the acceleration processing unit, wherein the conversion device is used to realize the conversion between the high-speed interconnect bus supported by the computing resource and the high-speed interconnect bus supported by the acceleration processing unit. For example, Figure 8 As shown ( Figure 8 Another schematic diagram of the structure of the cluster provided in the embodiment of the present application is shown in FIG. Figure 8 is Figure 3 In cluster 1, for the m SDI cards contained in SDI card pool 1, each SDI card has 1 NPU. It is worth noting that the high-speed interconnection bus supported by the m NPUs in SDI card pool 1 is UB, while the high-speed interconnection bus supported by the k CPUs in CPU pool 1 is PCIe, so these k CPUs are connected to the UB switch through adapter cards to communicate with the m NPUs.

[0091] The present application embodiment provides a cluster in a cloud service system, which has the following advantages:

[0092] (1) In this cluster, various types of computing resources are deployed in a pooled manner, and multiple computing resource pools are decoupled. They can be configured according to the type of cloud service required by the cluster, thereby improving the performance and resource utilization of the cluster. In this cluster, data processors are also deployed in a pooled manner to form a data processor pool, which serves as the communication scheduling center of the cluster, realizing the centralized allocation and utilization of each data processor in the pool, thereby improving the utilization of each data processor.

[0093] (2) In the cluster, the data processor pool and multiple computing resource pools are connected through a high-speed interconnection bus. That is to say, multiple computing resource pools and the outside of the cluster, multiple computing resource pools, and the interior of any computing resource pool can be connected through a high-speed interconnection bus. When the high-speed interconnection bus can provide sufficient bandwidth, the external and internal communication speed of the cluster can be improved and the cost required for the cluster can be reduced.

[0094] (3) In this cluster, computing resources can be combined in a Lego-like manner. For example, CPU, NPU, and GPU can be flexibly matched and uniformly managed at the software level.

[0095] (4) If the cluster is used to provide external AI services on the cloud, since the cluster contains a data processor pool with multiple data processors, which provides sufficiently powerful data processing capabilities, and the existence of a high-speed interconnection bus also provides sufficient bandwidth, the data processor pool of the cluster can support the transmission of a large number of model parameters required for AI model training, thereby improving the reliability of model training and reducing the occurrence of model training interruptions caused by the network.

[0096] (5) In this cluster, the general processing units and accelerated processing units contained in the data processor can be deployed in a decoupled manner, that is, deployed separately, further releasing the chip area occupied by the general processing units and the accelerated processing units, realizing the pooled deployment of accelerated processing units and the centralized deployment of general processing units, and having the ability to flexibly scale out.

[0097] (6) In the cluster, a memory pool containing multiple memories can be set up. The computing resources in other types of computing resource pools, such as CPU, NPU, GPU, etc., can use more memory in the memory pool. They are no longer restricted to the CPU, NPU, GPU, etc. in a physical server and can only use the memory in the physical server, thus forming a memory sharing (borrowing) scenario with extremely low latency.

[0098] (7) The cluster can provide a vertical integration platform for software and hardware. That is, the various pools in the cluster form a cabinet-level resource collection, which uniformly supports the management of cabinet-level resources such as computing, heterogeneous, and storage, as well as traffic export at the abstract layer.

[0099] (8) In the cluster, a data processor pool containing multiple data processors can be set up as a cabinet-level traffic outlet to implement a resource management and scheduling solution for distributed traffic. Among them, data processors and computing resources can implement pooled management of m:n mapping, so that the bandwidth available to all computing resources in the cluster is increased many times. In addition, such a deployment can also simplify the form of physical servers, and multiple data processors can achieve super-division multiplexing between different types of computing resources, thereby reducing costs.

[0100] (9) In this cluster, the data processor pool is deployed remotely, and key issues such as head-of-line blocking (HOL blocking) and congestion are effectively resolved.

[0101] (10) In this cluster, the configuration between the data processor and the computing resources can be m:n. When the computing resources and the data processor are processing the user's business, whether the computing resources or the data processor fails, the primary and secondary swap can be quickly started to realize business transfer. The user's perception of this is at the second level, which can improve the user experience.

[0102] (11) In this cluster, general-purpose processing units can be integrated on one chip. Compared with the previous situation where one general-purpose processing unit and one acceleration processing unit are integrated in one card, this can not only reduce the chip area occupied by the general-purpose processing unit, but also improve the utilization rate of the general-purpose processing unit.

[0103] (12) In this cluster, switches can collaborate to implement in-network computing (all reduce, scatter-gather), database acceleration (DB filter, Table Scan\regular expression) and hash acceleration, etc.

[0104] (13) In this cluster, the cores statically allocated to the current data plane can be uniformly managed according to the resource pool, thereby improving the utilization rate of computing resources and reducing the occupancy rate of computing resources.

[0105] (14) In this cluster, various types of computing resources are compatible with traditional PCIe-based virtio, nvme and other types, and also support a new ecosystem based on a new high-speed interconnect bus, which is conducive to the implementation of the system in various scenarios.

[0106] (15) In this cluster, if the computing resources and data processors support different high-speed interconnection buses, the two can be normalized to the same high-speed interconnection bus solution through a switching device, which is conducive to the application of the system in more scenarios.

[0107] (16) In this cluster, memory borrowing and unified addressing are supported between networked XPUs. With the support of high-speed interconnection buses, ultra-large bandwidth memory hot migration can be achieved.

[0108] (17) The cluster supports memory pin-free solution based on vfio / vdpa device pass-through, memory super-division multiplexing, etc.

[0109] The above is a detailed description of the cluster provided in the embodiment of the present application. The following is an introduction to the communication method provided in the embodiment of the present application. Fig. 9 A schematic diagram of a communication method provided in an embodiment of the present application, such as Fig. 9 As shown, this method is Figure 2 The cluster implementation shown in the figure includes a data processor pool and multiple computing resource pools of different types, the data processor pool includes multiple data processors, any one of the multiple computing resource pools includes multiple computing resources, and the data processor pool is connected to the multiple computing resource pools via a high-speed interconnect bus. The method includes:

[0110] 901. The data processor pool manages the communication between the plurality of computing resource pools and the outside of the cluster, the communication between the plurality of computing resource pools, and the internal communication of any computing resource pool.

[0111] In one possible implementation, multiple data processors include general processing units and multiple accelerated processing units; the data processor pool manages the communication between multiple computing resource pools and the outside of the cluster, the communication between multiple computing resource pools, and the internal communication of any computing resource pool, including: the general processing unit builds a first communication path between multiple computing resource pools and the outside of the cluster, a first communication path between multiple computing resource pools, and a first communication path inside any computing resource pool. Multiple accelerated processing units build a second communication path between multiple computing resource pools and the outside of the cluster, a second communication path between multiple computing resource pools, and a second communication path inside any computing resource pool.

[0112] In a possible implementation, the data processor pool and the multiple computing resource pools are set in multiple cabinets, and the cabinet used to set the data processor pool is different from the cabinet used to set any computing resource pool.

[0113] In one possible implementation, multiple data processors include a general processing unit and multiple accelerated processing units, and the method also includes: the general processing unit allocates at least one of the multiple accelerated processing units to computing resources with communication requirements in multiple computing resource pools, so that the at least one accelerated processing unit allocated to the computing resources meets the communication requirements of the computing resources.

[0114] In one possible implementation, a general processing unit and a plurality of accelerated processing units are deployed inside a data processor pool.

[0115] In a possible implementation, multiple acceleration processing units are deployed inside a data processor pool, and general processing units are deployed outside the data processor pool, where the outside of the data processor pool is the inside of at least one computing resource pool.

[0116] In one possible implementation, a first communication path between multiple computing resource pools and the outside of the cluster has any one computing resource in the multiple computing resource pools and the outside of the cluster as endpoints, passing through a general processing unit and at least one of the multiple acceleration processing units; a first communication path between multiple computing resource pools has computing resources in any two computing resource pools as endpoints, passing through a general processing unit and at least one of the multiple acceleration processing units; a first communication path inside any one computing resource pool has two computing resources in any one computing resource pool as endpoints, passing through a general processing unit and at least one of the multiple acceleration processing units.

[0117] In one possible implementation, the second communication path between multiple computing resource pools and the outside of the cluster has any one computing resource in the multiple computing resource pools and the outside of the cluster as endpoints, passing through at least one acceleration processing unit among the multiple acceleration processing units; the second communication path between multiple computing resource pools has computing resources in any two computing resource pools as endpoints, passing through at least one acceleration processing unit among the multiple acceleration processing units; the second communication path inside any computing resource pool has two computing resources in any one computing resource pool as endpoints, passing through at least one acceleration processing unit among the multiple acceleration processing units.

[0118] In one possible implementation, the method also includes: the general processing unit establishes connections between multiple acceleration processing units and computing resources in multiple computing resource pools by issuing tokens, and for any one of the multiple acceleration processing units, the number of tokens that the acceleration processing unit can receive is equal to the number of computing resources that can be connected to the acceleration processing unit.

[0119] In one possible implementation, when the high-speed interconnect bus supported by the acceleration processing unit is different from the high-speed interconnect bus supported by the computing resources, the computing resources and the acceleration processing unit are connected via the high-speed interconnect bus supported by the computing resources, the conversion device, and the high-speed interconnect bus supported by the acceleration processing unit.

[0120] In one possible implementation, the multiple computing resource pools include at least two of the following: a CPU pool, an NPU pool, a GPU pool, a memory pool, and an SSD pool. The CPU pool includes multiple CPUs, the NPU pool includes multiple NPUs, the GPU pool includes multiple GPUs, the memory pool includes multiple memories, and the SSD pool includes multiple SSDs.

[0121] In one possible implementation, any computing resource pool includes multiple computing resources for deploying cloud instances that provide cloud services, and the cloud instances include any of the following: computing resources, virtual machines, containers, and micro virtual machines.

[0122] It should be noted that the implementation process of each step in the above method and other contents are based on the same concept as the device embodiment of the present application, and the technical effects they bring are the same as those of the method embodiment of the present application. For specific contents, please refer to the description in the device embodiment shown above in the embodiment of the present application, and will not be repeated here.

[0123] See also Fig.10 , Fig.10 A schematic diagram of a computing device provided in an embodiment of the present application. Fig.10 As shown, the computing device 1000 includes: a processor 1001, a memory 1002, a communication interface 1003 and a bus 1004. The processor 1001, the memory 1002 and the communication interface 1003 are coupled via a bus (not marked in the figure). The memory 1002 stores instructions. When the execution instructions in the memory 1002 are executed, the computing device 1000 executes the communication method in the above method embodiment.

[0124] The computing device 1000 may be one or more integrated circuits configured to implement the above method, such as one or more application specific integrated circuits (ASIC), or one or more digital signal processors (DSP), or one or more field programmable gate arrays (FPGA), or a combination of at least two of these integrated circuit forms. For another example, when a unit in the device can be implemented in the form of a processing element scheduler, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call a program. For another example, these units may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0125] The processor 1001 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. A general-purpose processor may be a microprocessor or any conventional processor.

[0126] The memory 1002 may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. Among them, the nonvolatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0127] The memory 1002 stores executable program codes, and the processor 1001 executes the executable program codes to respectively implement the functions of the general processing unit and the accelerated processing unit, thereby implementing the above communication method. That is, the memory 1002 stores instructions for executing the above communication method.

[0128] The communication interface 1003 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1000 and other devices or a communication network.

[0129] In addition to the data bus, the bus 1004 may also include a power bus, a control bus, a status signal bus, etc. The bus may be a peripheral component interconnect express (PCIe) bus, or an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX), etc. The bus may be divided into an address bus, a data bus, a control bus, etc.

[0130] See also Fig.11 , Fig.11 A schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application. Fig.11 As shown, the computing device cluster 1100 includes at least one computing device 1000 .

[0131] like Fig.11 As shown, the computing device cluster 1100 includes at least one computing device 1000. The memory 1002 in one or more computing devices 1000 in the computing device cluster 1100 may store the same instructions for executing the above communication method.

[0132] In some possible implementations, the memory 1002 of one or more computing devices 1000 in the computing device cluster 1100 may also store partial instructions for executing the above communication method. In other words, the combination of one or more computing devices 1000 may jointly execute the above communication method.

[0133] It should be noted that the memory 1002 in different computing devices 1000 in the computing device cluster 1100 may store different instructions, which are respectively used to execute part of the functions of the above-mentioned data processor. That is, the instructions stored in the memory 1002 in different computing devices 1000 may implement the functions of one or more modules in the general processing unit and the accelerated processing unit.

[0134] In some possible implementations, one or more computing devices 1000 in the computing device cluster 1100 may be connected via a network, which may be a wide area network or a local area network.

[0135] See also Fig.12 , Fig.12A schematic diagram of computer devices in a computer cluster provided in an embodiment of the present application being connected via a network. Fig.12 As shown, two computing devices 1000A and 1000B are connected via a network. Specifically, they are connected to the network via a communication interface in each computing device.

[0136] In a possible implementation, the memory in the computing device 1000A stores instructions for executing the functions of a general processing unit and the like. Meanwhile, the memory in the computing device 1000B stores instructions for executing the functions of an accelerated processing unit and the like.

[0137] It should be understood that Fig.12 The functions of the computing device 1000A shown in FIG. 1000A may also be completed by multiple computing devices. Similarly, the functions of the computing device 1000B may also be completed by multiple computing devices.

[0138] The embodiment of the present application also relates to a computer storage medium, in which a program for signal processing is stored. When the program is run on a computer, the computer executes the following steps: Fig. 9 The communication method in the illustrated embodiment.

[0139] The present application also relates to a computer program product, which stores instructions, which, when executed by a computer, enable the computer to execute the following Fig. 9 The communication method in the illustrated embodiment.

[0140] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0141] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0142] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0143] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0144] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

Claims

1. A cluster, characterized in that: The cluster includes a data processor pool and multiple computing resource pools of different types, the data processor pool includes multiple data processors, and any one of the multiple computing resource pools includes multiple computing resources; The data processor pool is connected to the plurality of computing resource pools via a high-speed interconnect bus; The data processor pool is used to manage the communication between the multiple computing resource pools and the outside of the cluster, the communication between the multiple computing resource pools, and the internal communication of any one of the computing resource pools.

2. The cluster according to claim 1, characterized in that: The data processor pool and the plurality of computing resource pools are arranged in a plurality of cabinets, and the cabinet used for setting the data processor pool and the cabinet used for setting any one of the computing resource pools are different cabinets.

3. The cluster according to claim 2, characterized in that: The multiple data processors include a general processing unit and a plurality of accelerated processing units. The general processing unit is used to allocate at least one of the multiple accelerated processing units to computing resources with communication requirements in the multiple computing resource pools. The at least one accelerated processing unit allocated to the computing resource is used to meet the communication requirements of the computing resource.

4. The cluster according to claim 3, characterized in that: The general processing unit is used to construct a first communication path between the multiple computing resource pools and the outside of the cluster, a first communication path between the multiple computing resource pools, and a first communication path inside any one of the computing resource pools; The multiple acceleration processing units are used to construct second communication paths between the multiple computing resource pools and the outside of the cluster, second communication paths between the multiple computing resource pools, and a second communication path inside any one of the computing resource pools.

5. The cluster according to claim 3 or 4, characterized in that: The general processing unit and the plurality of accelerated processing units are deployed inside the data processor pool.

6. The cluster according to claim 3 or 4, characterized in that: The multiple acceleration processing units are deployed inside the data processor pool, and the general processing unit is deployed outside the data processor pool. The outside of the data processor pool is the inside of at least one computing resource pool.

7. The cluster according to any one of claims 3 to 6, characterized in that: a first communication path between the plurality of computing resource pools and the outside of the cluster, with any one computing resource in the plurality of computing resource pools and the outside of the cluster as endpoints, passing through the general processing unit and at least one accelerated processing unit among the plurality of accelerated processing units; The first communication path between the plurality of computing resource pools has computing resources in any two computing resource pools as endpoints and passes through the general processing unit and at least one of the plurality of accelerated processing units; The first communication path inside any one of the computing resource pools has two computing resources in the any one of the computing resource pools as endpoints, and passes through the general processing unit and at least one of the multiple acceleration processing units.

8. The cluster according to any one of claims 3 to 6, characterized in that: A second communication path between the plurality of computing resource pools and the outside of the cluster has any one computing resource in the plurality of computing resource pools and the outside of the cluster as endpoints, and passes through at least one acceleration processing unit in the plurality of acceleration processing units; The second communication path between the plurality of computing resource pools has computing resources in any two computing resource pools as endpoints and passes through at least one acceleration processing unit among the plurality of acceleration processing units; The second communication path inside any one of the computing resource pools has two computing resources in the any one of the computing resource pools as endpoints and passes through at least one of the multiple acceleration processing units.

9. The cluster according to any one of claims 2 to 8, characterized in that: When the high-speed interconnect bus supported by the acceleration processing unit is different from the high-speed interconnect bus supported by the computing resource, the computing resource and the acceleration processing unit are connected through the high-speed interconnect bus supported by the computing resource, the conversion device and the high-speed interconnect bus supported by the acceleration processing unit.

10. The cluster according to any one of claims 1 to 9, characterized in that: The multiple computing resource pools include at least two of the following: a central processing unit CPU pool, a neural network processor NPU pool, a graphics processing unit GPU pool, a memory pool and a hard disk SSD pool. The CPU pool includes multiple CPUs, the NPU pool includes multiple NPUs, the GPU pool includes multiple GPUs, the memory pool includes multiple memories, and the SSD pool includes multiple SSDs.

11. The cluster according to any one of claims 1 to 10, characterized in that: Any one of the computing resource pools includes multiple computing resources for deploying a cloud instance that provides cloud services, and the cloud instance includes any one of the following: computing resources, virtual machines, containers, and micro virtual machines.

12. A communication method, characterized in that: The method is implemented by a cluster, the cluster includes a data processor pool and multiple computing resource pools of different types, the data processor pool includes multiple data processors, any one of the multiple computing resource pools includes multiple computing resources, and the data processor pool is connected to the multiple computing resource pools via a high-speed interconnect bus; The method comprises: The data processor pool manages the communication between the plurality of computing resource pools and the outside of the cluster, the communication between the plurality of computing resource pools, and the internal communication of any one of the computing resource pools.

13. The method according to claim 12, characterized in that The data processor pool and the plurality of computing resource pools are arranged in a plurality of cabinets, and the cabinet used for setting the data processor pool and the cabinet used for setting any one of the computing resource pools are different cabinets.

14. The method according to claim 13, characterized in that The plurality of data processors include a general processing unit and a plurality of accelerated processing units, and the method further includes: The general processing unit allocates at least one of the multiple accelerated processing units to a computing resource with communication requirements in the multiple computing resource pools, so that the at least one accelerated processing unit allocated to the computing resource meets the communication requirements of the computing resource.

15. The method according to claim 14, characterized in that The multiple data processors include a general processing unit and a plurality of accelerated processing units; the data processor pool manages the communication between the multiple computing resource pools and the outside of the cluster, and the communication between the multiple computing resource pools and the internal communication of any one of the computing resource pools include: The general processing unit constructs a first communication path between the plurality of computing resource pools and the outside of the cluster, a first communication path between the plurality of computing resource pools, and a first communication path inside any one of the computing resource pools; The multiple accelerated processing units construct second communication paths between the multiple computing resource pools and the outside of the cluster, second communication paths between the multiple computing resource pools, and a second communication path inside any one of the computing resource pools.

16. The method according to claim 14 or 15, characterized in that The general processing unit and the plurality of accelerated processing units are deployed inside the data processor pool.

17. The method according to claim 14 or 15, characterized in that The multiple acceleration processing units are deployed inside the data processor pool, and the general processing unit is deployed outside the data processor pool. The outside of the data processor pool is the inside of at least one computing resource pool.

18. The method according to any one of claims 14 to 17, characterized in that: a first communication path between the plurality of computing resource pools and the outside of the cluster, with any one computing resource in the plurality of computing resource pools and the outside of the cluster as endpoints, passing through the general processing unit and at least one accelerated processing unit among the plurality of accelerated processing units; The first communication path between the plurality of computing resource pools has computing resources in any two computing resource pools as endpoints and passes through the general processing unit and at least one of the plurality of accelerated processing units; The first communication path inside any one of the computing resource pools has two computing resources in the any one of the computing resource pools as endpoints, and passes through the general processing unit and at least one of the multiple acceleration processing units.

19. The method according to any one of claims 14 to 18, characterized in that: A second communication path between the plurality of computing resource pools and the outside of the cluster has any one computing resource in the plurality of computing resource pools and the outside of the cluster as endpoints, and passes through at least one acceleration processing unit in the plurality of acceleration processing units; The second communication path between the plurality of computing resource pools has computing resources in any two computing resource pools as endpoints and passes through at least one acceleration processing unit among the plurality of acceleration processing units; The second communication path inside any one of the computing resource pools has two computing resources in the any one of the computing resource pools as endpoints and passes through at least one of the multiple acceleration processing units.

20. The method according to any one of claims 13 to 19, characterized in that: When the high-speed interconnect bus supported by the acceleration processing unit is different from the high-speed interconnect bus supported by the computing resource, the computing resource and the acceleration processing unit are connected through the high-speed interconnect bus supported by the computing resource, the conversion device and the high-speed interconnect bus supported by the acceleration processing unit.

21. The method according to any one of claims 12 to 20, characterized in that The multiple computing resource pools include at least two of the following: a CPU pool, an NPU pool, a GPU pool, a memory pool and an SSD pool, the CPU pool includes multiple CPUs, the NPU pool includes multiple NPUs, the GPU pool includes multiple GPUs, the memory pool includes multiple memories, and the SSD pool includes multiple SSDs.

22. The method according to any one of claims 12 to 21, characterized in that Any one of the computing resource pools includes multiple computing resources for deploying a cloud instance that provides cloud services, and the cloud instance includes any one of the following: computing resources, virtual machines, containers, and micro virtual machines.

23. A cloud service system, characterized in that: The cloud service system comprises a plurality of clusters as described in any one of claims 1 to 11.

24. A computing device cluster, characterized in that: The computing device cluster includes at least one computing device, each computing device including a processor and a memory: The memory is used to store instructions; The processor is configured to cause the computing device cluster to execute the method according to any one of claims 12 to 22 according to the instructions.

25. A computer storage medium, characterized in that The computer storage medium stores one or more instructions, which, when executed by one or more computers, cause the one or more computers to implement the method of any one of claims 12 to 22.

26. A computer program product, characterized in that The computer program product stores instructions, which, when executed by a computer, cause the computer to implement the method according to any one of claims 12 to 22.

Images