Heterogeneous acceleration pooling server system, pooling resource allocation method and medium

Abstract

The invention provides a heterogeneous acceleration pooling server system, a pooling resource allocation method and a medium, and relates to the technical field of computers, the heterogeneous acceleration pooling server system comprises a general computing resource pool, an internal bus switching module, an Ethernet switching module and a plurality of heterogeneous computing acceleration resource pools; the internal bus exchange module comprises a plurality of mutually connected high-speed data interconnection modules, each module is provided with a plurality of IO devices, the IO devices of the plurality of high-speed data interconnection modules overall allocate global unique codes, the general computing resource pool configures the required IO devices according to resource requirements after obtaining the resource requirements, and the required IO devices are distributed to the general computing resource pool according to the resource requirements. And establishing a heterogeneous computing resource allocation path to enable the general computing resource pool to be matched with one or more heterogeneous computing acceleration resource pools corresponding to the resource requirements, so that each general computing power unit can access any heterogeneous acceleration device, the system expansion capability is improved, and the resource utilization rate is improved when large-scale and diversified applications are dealt with. And the operation efficiency of the heterogeneous acceleration server is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a heterogeneous acceleration pooled server system, a pooled resource allocation method and a medium. Background Art

[0002] With the development of computer applications such as artificial intelligence and big data, the demand for computing power has grown rapidly. At the same time, with the gradual diversification of computing application scenarios, the emergence of different application fields such as machine learning, artificial intelligence, unmanned driving, and industrial Internet, traditional general-purpose processors encounter more and more performance bottlenecks when processing massive calculations and massive data / images, such as low computing parallelism, low bandwidth, and high latency. The traditional server architecture with single-core processors as the computing core can no longer meet today's needs. In order to efficiently process various computing tasks, different types of heterogeneous acceleration processors have emerged. Traditional heterogeneous acceleration services are simply integrated with existing processors of different specifications and models. The traditional heterogeneous acceleration service architecture adopts an I / O tree structure. During the startup process, the host unit enumerates the I / O bridges or I / O devices in the I / O tree structure based on a depth-first enumeration method, and assigns a unique ID number to them in each I / O tree. The routing path can only be allocated within each I / O tree, which limits the diversified allocation and use of I / O device resources. The system has limited expansion capabilities, a closed ecosystem, wastes energy, and has high costs for upgrading and expanding capacity. When dealing with large-scale and diversified applications, traditional heterogeneous acceleration servers are not efficient. Summary of the invention

[0003] The present invention provides a heterogeneous acceleration pooled server system, a pooled resource allocation method and a medium, which are used to solve the defects of low operating efficiency and limited performance of traditional heterogeneous acceleration pooled server systems, which restrict the commercial application of high-performance computing technology.

[0004] The present invention provides a heterogeneous acceleration pooled server system, comprising: General computing resource pool, internal bus switching module, Ethernet switching module and multiple heterogeneous computing acceleration resource pools; The general computing resource pool is communicatively connected to the internal bus switching module via a first intra-system interconnect bus; Each heterogeneous computing acceleration resource pool is communicatively connected to the internal bus switching module via a second intra-system interconnection bus; The internal bus switching module includes a plurality of interconnected high-speed data interconnect modules, each of which is provided with a plurality of IO devices. The IO devices of the plurality of high-speed data interconnect modules are allocated a globally unique code in a coordinated manner so as to access any IO device in any high-speed data interconnect module through the globally unique code. After the general computing resource pool obtains the resource requirements, the required IO devices are configured according to the resource requirements through the internal bus switching module, and a heterogeneous computing resource allocation path is established to enable the general computing resource pool to match one or more heterogeneous computing acceleration resource pools corresponding to the resource requirements.

[0005] According to the heterogeneous acceleration pooled server system provided by the present invention, the heterogeneous computing acceleration resource pool includes: At least one accelerator card, wherein the accelerator card has an integrated accelerator processor and an Ethernet controller; The acceleration processor is directly connected to the Ethernet controller; The Ethernet controller is connected to the Ethernet switching module.

[0006] According to the heterogeneous acceleration pooled server system provided by the present invention, the system further includes: A high-speed serial computer expansion bus standard link is provided, and the high-speed serial computer expansion bus standard link is used for interconnecting accelerated processors between multiple servers.

[0007] According to the heterogeneous acceleration pooled server system provided by the present invention, the system further comprises a remote direct data access network card, the remote direct data access network card communicates with the kernel driver of the acceleration processor through a kernel driver standard interface,

[0008] The remote direct data access network card is connected to the Ethernet switching module, so that the multiple acceleration processors connected to the multiple remote direct data access network cards can complete the direct transmission of data and the remote direct access to the video memory.

[0009] According to the heterogeneous acceleration pooled server system provided by the present invention, the internal bus switching module includes an I / O switching chip controller, and the I / O switching chip controller is used to enumerate all I / O devices, and define network topology division according to the resource requirements, dynamically adjust the matching between the acceleration processor and the I / O device, and establish a resource allocation path.

[0010] According to the heterogeneous acceleration pooled server system provided by the present invention, the globally unique code is obtained by encoding the high-speed data interconnect module number and the IO device number inside it using a segmented encoding method.

[0011] According to the heterogeneous acceleration pooled server system provided by the present invention, the general computing resource pool includes: Multiple general computing units, each of which includes a general processor and memory. Any number of general processors and memories can be combined through virtualization software to form a computing resource container of any granularity.

[0012] According to the heterogeneous acceleration pooled server system provided by the present invention, the system further includes a pooled management controller; A pooled node management controller is respectively provided in the general computing power unit in the general computing resource pool and the acceleration card of the heterogeneous computing acceleration resource pool. The pooled node management controller manages multiple pooled node management controllers in a unified manner to perform at least one of node topology identification, centralized display of asset information, and coordinated power on and off for the general computing resource pool and the heterogeneous computing acceleration resource pool.

[0013] According to the heterogeneous accelerated pooled server system provided by the present invention, the pooled management controller is also used to perform at least one of integrated monitoring, fault warning and visual management of the system through a standardized service interface.

[0014] According to the heterogeneous acceleration pooled server system provided by the present invention, the system further includes a pooling management engine; The pooling management engine is used to manage the internal bus switching module, discover the current network topology by calling the physical channels and software interfaces in the internal bus switching module to obtain the current asset allocation situation, and coordinate the overall computing power resources to switch the network topology according to the physical channels and software interfaces to meet the resource requirements.

[0015] According to the heterogeneous acceleration pooled server system provided by the present invention, the pooled management engine is further used for: Hot removal, hot insertion and hot reset are performed on any acceleration card in the heterogeneous computing acceleration resource pool to control whether the acceleration card is put into use.

[0016] According to the heterogeneous acceleration pooled server system provided by the present invention, the pooled management engine is further used for: Based on the load balancing optimization scheduling algorithm, the acceleration cards in the heterogeneous computing acceleration resource pool are dynamically allocated and adjusted.

[0017] According to the heterogeneous acceleration pooled server system provided by the present invention, the pooled management engine is further used for: Establish expert templates for different application scenarios; When the application scenario is switched, the API is called according to the current application scenario to switch different expert templates; the expert templates that meet the requirements of the application scenario are visually displayed through a visual WEB interface.

[0018] According to the heterogeneous acceleration pooled server system provided by the present invention, the heterogeneous acceleration pooled server system is a single-machine heterogeneous acceleration pooled server system, and a single host in the single-machine heterogeneous acceleration pooled server system supports 8 acceleration card expansion, 16 acceleration card expansion and 32 acceleration card expansion.

[0019] According to the heterogeneous acceleration pooled server system provided by the present invention, the heterogeneous acceleration pooled server system is a multi-host heterogeneous acceleration pooled server system, in which each host is connected to a high-speed data interconnect module, and 32 acceleration processor cards are shared through multiple high-speed data interconnect modules.

[0020] According to the heterogeneous acceleration pooled server system provided by the present invention, the system further includes a computing power virtualization subsystem, which aggregates the heterogeneous computing acceleration resource pools into a unified virtual computing power resource pool. The computing power virtualization subsystem includes: Pooled servers, which are computing nodes in the system cluster and are used for task execution; A control server, wherein the control server is used to run the control component in the system cluster; An application server, wherein the application server is used to run an application program in the system cluster, and the application server is a container, a virtual machine, or a physical machine.

[0021] The present invention also provides a pooled resource allocation method, which is applicable to any of the above-mentioned heterogeneous acceleration pooled server systems, comprising: A general computing unit in the general computing resource pool initiates a dynamic adjustment request for heterogeneous computing resources to the pooling management controller based on resource requirements; Determine the IO device to be accessed in the internal bus switching module according to the resource demand through the pooling management engine, access the IO devices in one or more high-speed data interconnect modules through the global unique code, reconstruct the heterogeneous computing resource allocation path, and match the heterogeneous computing resources corresponding to the resource demand for the general computing unit; The heterogeneous computing resources include resources in a single acceleration resource pool, resources in cross-acceleration resource pools, and resources in cross-pooled servers.

[0022] According to the pooled resource allocation method provided by the present invention, when reconstructing the heterogeneous computing resource allocation path, the method includes releasing the accelerator card in a general computing unit and allocating it to other general computing units, specifically including: after the pooled management engine performs hot removal on the accelerator card, it sends a request to the internal bus switching module to obtain the physical location of the accelerator card; Find the accelerator card according to the physical location and reset the accelerator card.

[0023] Reassign the reset accelerator card to the general computing unit to be assigned.

[0024] According to the pooled resource allocation method provided by the present invention, the accelerator card in a general computing unit is released and allocated to other general computing units, and further includes: Training the reset accelerator card according to the application scenario of the general computing unit to be allocated; Re-allocate the trained accelerator card to the general computing unit to be allocated.

[0025] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program implements any of the above-mentioned pooled resource allocation methods when executed by a processor.

[0026] The heterogeneous acceleration pooled server system, pooled resource allocation method and medium provided by the present invention, the heterogeneous acceleration pooled server system comprises a general computing resource pool, an internal bus switching module, an Ethernet switching module and a plurality of heterogeneous computing acceleration resource pools; the general computing resource pool is communicatively connected to the internal bus switching module via a first system-interconnecting bus; each heterogeneous computing acceleration resource pool is communicatively connected to the internal bus switching module via a second system-interconnecting bus; the internal bus switching module comprises a plurality of interconnected high-speed data interconnection modules, each high-speed data interconnection module is provided with a plurality of IO devices, and the IO devices of the plurality of high-speed data interconnection modules are allocated in a coordinated manner A globally unique code is used to access any IO device in any high-speed data interconnect module through the globally unique code; after the universal computing resource pool obtains the resource demand, the required IO device is configured according to the resource demand through the internal bus switching module, and a heterogeneous computing resource allocation path is established to match the universal computing resource pool with one or more heterogeneous computing acceleration resource pools corresponding to the resource demand. By assigning a globally unique code to multiple IO devices on each high-speed data interconnect module, each universal computing unit can access any heterogeneous acceleration device, thereby improving the system expansion capability and improving the operating efficiency of the heterogeneous acceleration server when dealing with large-scale and diversified applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the present invention or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0028] Figure 1 It is a functional structure diagram of a heterogeneous acceleration pooled server system provided by an embodiment of the present invention; Figure 2 This is a schematic diagram of the data flow of a heterogeneous acceleration card provided by an embodiment of the present invention; Figure 3 This is a schematic diagram of RDMA communication data flow of a heterogeneous accelerator card provided by an embodiment of the present invention; Figure 4 This is a schematic diagram of end-to-end communication of a distributed heterogeneous acceleration server provided by an embodiment of the present invention; Figure 5 It is a schematic diagram of distributed heterogeneous acceleration server cluster expansion provided by an embodiment of the present invention; Figure 6 It is a schematic diagram of a heterogeneous computing resource pool management architecture provided by an embodiment of the present invention; Figure 7 It is a schematic diagram of the architecture of a heterogeneous computing resource pooling management engine provided by an embodiment of the present invention; Figure 8 Schematic diagram of the expert template mechanism of heterogeneous computing resource pool provided by the embodiment of the present invention

[0029] Fig. 9 It is a schematic diagram of the structure of an existing heterogeneous acceleration server system provided by an embodiment of the present invention; Fig.10 It is a schematic diagram of the structure of a single-machine 32-card heterogeneous acceleration server system provided by an embodiment of the present invention; Fig.11 This is a schematic diagram of the structure of a single-cabinet 64-card heterogeneous acceleration server provided by an embodiment of the present invention; Fig.12 This is a schematic diagram of the structure of a heterogeneous acceleration server with 32 cards shared by multiple hosts provided by an embodiment of the present invention; Fig.13 This is a schematic diagram of the structure of a heterogeneous acceleration server with 32 cards shared by multiple hosts provided by an embodiment of the present invention; Fig.14 It is a relationship diagram of heterogeneous computing power software-defined virtualization and pooled servers provided by an embodiment of the present invention; Fig.15 It is a schematic diagram of the dynamic allocation process of heterogeneous computing resource devices provided by an embodiment of the present invention; Fig.16 It is a schematic diagram of the heterogeneous computing power software-defined virtualization deployment process provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] Figure 1 The functional structure diagram of the heterogeneous acceleration pooled server system provided by the embodiment of the present invention is as follows: Figure 1 As shown, the heterogeneous acceleration pooled server system provided by the embodiment of the present invention includes: General computing resource pool, internal bus switching module, Ethernet switching module and multiple heterogeneous computing acceleration resource pools; The general computing resource pool is communicatively connected to the internal bus switching module via a first intra-system interconnect bus; Each heterogeneous computing acceleration resource pool is communicatively connected to the internal bus switching module via a second intra-system interconnection bus; Each heterogeneous computing acceleration resource pool is communicatively connected to the Ethernet switching module; The internal bus switching module includes a plurality of interconnected high-speed data interconnect modules, each of which is provided with a plurality of IO devices. The IO devices of the plurality of high-speed data interconnect modules are allocated a globally unique code in a coordinated manner so as to access any IO device in any high-speed data interconnect module through the globally unique code. After the general computing resource pool obtains the resource requirements, the required IO devices are configured according to the resource requirements through the internal bus switching module, and a heterogeneous computing resource allocation path is established to enable the general computing resource pool to match one or more heterogeneous computing acceleration resource pools corresponding to the resource requirements.

[0032] In the embodiment of the present invention, the globally unique code is obtained by encoding the high-speed data interconnect module number and the IO device number inside it using a segmented encoding method.

[0033] In an embodiment of the present invention, the general computing resource pool is a resource pool composed of general processors such as CPUs, the heterogeneous computing acceleration resource pool is a resource pool composed of heterogeneous processors such as GPUs, the multiple high-speed data interconnect modules in the internal bus switching module are, for example, four high-performance switching boards, each high-performance switching board externally expands 10 high-speed interfaces (IO devices), the general computing resource pool is composed of general computing power units, which mainly provide the computing function of the system, and the heterogeneous computing acceleration resource pool is composed of heterogeneous computing power units, which mainly provide accelerated computing functions for the entire system to achieve higher computing performance, parallelism, and energy efficiency; the internal bus switching module is a high-performance switching unit, including a high-speed data interconnection module, which is closely integrated with the basic software to achieve physical decoupling of general computing and heterogeneous accelerated computing, and at the same time achieve efficient configuration management of heterogeneous acceleration cards and cross-node direct sharing of resources, improve the access efficiency of heterogeneous resources, and provide a highly reliable and high-performance data interconnection network for the entire system.

[0034] In the embodiment of the present invention, the heterogeneous acceleration pooled server system adopts a distributed architecture, facing the diversified application needs, changing the traditional CPU-centric design, and through key technologies such as hardware decoupling and consistent high-speed interconnection, with the internal bus switching module as the core, decoupling and reconstructing the traditional server architecture, realizing the collaborative computing of multiple general-purpose processor platforms and multiple heterogeneous computing acceleration units, and realizing the hardware decoupling, pooling and reconstruction of large-scale computing resources, memory resources, heterogeneous acceleration resources, and storage resources; through software-defined system design, resource collaborative dynamic scheduling is realized. The system resource pool is integrated through unified management, heat dissipation, and power supply to form a high-performance host with high heterogeneous computing power and flexible resource scheduling and allocation. The entire system can flexibly provide multiple computing powers for a variety of application scenarios, and realize application demand-oriented integration of multiple computing powers through system-level architecture key technology innovation to improve computing performance under the condition of limited key components and devices.

[0035] In an embodiment of the present invention, the acceleration processor is a graphics processing unit (GPU), a field programmable gate array (FPGA), etc. Through the interconnection of heterogeneous computing acceleration resource pools and software-defined system design, the system topology can be dynamically adjusted to achieve on-demand allocation of heterogeneous resources.

[0036] The distributed heterogeneous acceleration pooled server system realizes the device resource pooling through the high-performance switching of the system internal bus and the software-defined system design, which releases the binding relationship between the device and the CPU at the physical link level, and can flexibly adjust the port configuration and resource allocation path of the switching network, and divide the shared resource pool in a fine-grained manner to realize the on-demand elastic allocation of device resources and multi-host sharing. In view of the current problems that the performance expansion of the host system in the data center is limited by the system interconnection bandwidth, the performance mismatch between storage at all levels, and the low utilization of I / O resources, through the pooled system design, the dynamic allocation of resources and load balancing can be flexibly realized for the needs of diversified scenarios, and the fusion of multi-platform processor computing power and the coordinated scheduling of heterogeneous acceleration resources can be realized to alleviate the bottleneck problem of data center performance expansion.

[0037] Traditional heterogeneous acceleration services are simply integrated by integrating existing processors of different specifications and models. The traditional heterogeneous acceleration service architecture adopts an I / O tree structure. During the startup process, the host unit enumerates the I / O bridges or I / O devices in the I / O tree structure based on a depth-first enumeration method, and assigns a unique ID number to each I / O tree. The routing path can only be allocated within each I / O tree, which limits the diversified allocation and use of I / O device resources. The system has limited expansion capabilities, a closed ecosystem, wastes energy, and has high upgrade and expansion costs. When dealing with large-scale and diversified applications, the traditional heterogeneous acceleration server has low operating efficiency.

[0038] The heterogeneous acceleration pool server system provided by the embodiment of the present invention comprises a general computing resource pool, an internal bus switching module, an Ethernet switching module and a plurality of heterogeneous computing acceleration resource pools; the general computing resource pool is communicatively connected to the internal bus switching module via a first system interconnection bus; each heterogeneous computing acceleration resource pool is communicatively connected to the internal bus switching module via a second system interconnection bus; the internal bus switching module comprises a plurality of interconnected high-speed data interconnection modules, each high-speed data interconnection module is provided with a plurality of IO devices, and the IO devices of the plurality of high-speed data interconnection modules are allocated a global unique code in a coordinated manner, so as to access any IO device in any high-speed data interconnection module through the global unique code; after the general computing resource pool obtains the resource demand, the required IO device is configured according to the resource demand through the internal bus switching module, and a heterogeneous computing resource allocation path is established so that the general computing resource pool matches one or more heterogeneous computing acceleration resource pools corresponding to the resource demand, and by allocating a global unique code to the plurality of IO devices on each high-speed data interconnection module, each general computing power unit can access any heterogeneous acceleration device, thereby improving the system expansion capability, and improving the operation efficiency of the heterogeneous acceleration server when dealing with large-scale and diversified applications.

[0039] Based on any of the above embodiments, the heterogeneous computing acceleration resource pool includes: At least one accelerator card, wherein the accelerator card has an integrated accelerator processor and an Ethernet controller; The acceleration processor is directly connected to the Ethernet controller; The Ethernet controller is connected to the Ethernet switching module.

[0040] In the embodiment of the present invention, the heterogeneous acceleration card is innovatively designed to directly connect the acceleration processor to the Ethernet controller to meet the computing power challenge of large models. The acceleration processor is directly connected to the Ethernet controller through the switching chip. Different from the traditional acceleration processor connected to the switching chip through the network interface card (NIC), then connected to the CPU through the switching chip, and then connected to the network card through the CPU, the heterogeneous computing acceleration card integrates GPU chips and Ethernet controllers to realize direct reading of data streams, such as Figure 2 As shown in the figure, the RDMA communication library needs to call the interface to initiate a network transmission request. The driver in the software stack supports registering the RDMA network card related interface. The RDMA network card will communicate with the GPU kernel driver through the kernel driver standard interface to complete the direct data transmission and realize the remote direct access to the video memory. Only the GPU and the RDMA network card participate in the entire data path, avoiding redundant jumps of data to the system memory and effectively reducing communication delays.

[0041] In an embodiment of the present invention, the RDMA network card communicates with the GPU kernel driver through the kernel driver standard interface to complete direct data transmission, and the number of data hops is reduced by 50% compared with the traditional distributed typical path; the acceleration processor integrates the remote direct access function of memory, and the acceleration card supports dual-port 100G Ethernet and remote direct data access (Remote Direct Memory Access, RDMA) function, and supports GPU chips and Ethernet through software stacks and drivers.

[0042] In an embodiment of the present invention, the accelerator card further includes a remote direct data access network card, which communicates with the kernel driver of the acceleration processor through a kernel driver standard interface to complete direct data transmission and remote direct access to the video memory. The communication software stack for remote direct data access of the system is as follows: Figure 3 As shown in the figure, it can reduce the data transmission delay of the acceleration processor and improve the computing performance.

[0043] In the embodiment of the present invention, the distributed acceleration server supports two end-to-end communication modes compared to the traditional AI server, such as Figure 4 As shown. The first end-to-end communication mode is the high-speed serial computer expansion bus standard (Peripheral Component Interconnect Express, PCIe) link communication, and the second end-to-end communication mode is the direct connection communication technology between the switch chip and the Ethernet controller. Through the two end-to-end communication modes, the entire system can be horizontally expanded to achieve interconnection between systems, thereby realizing larger-scale heterogeneous computing acceleration pooling.

[0044] Based on this heterogeneous acceleration card, a distributed heterogeneous acceleration server is developed. The whole machine can be equipped with a switch chip and a PCIe-compatible heterogeneous acceleration processor module. Servers are interconnected through heterogeneous acceleration cards and switches, and the hardware interconnection of multiple training server machines is formed to achieve cluster computing power.

[0045] Based on any of the above embodiments, the heterogeneous acceleration pooled server system further includes: A high-speed serial computer expansion bus standard link, which is used to interconnect the acceleration processors between multiple servers. The high-speed serial computer expansion bus standard link is also provided with a parameter plane switch, a service plane switch, a storage plane switch, a storage node and a service node; The parameter plane switch is used to perform cluster expansion on the acceleration processors in the heterogeneous computing acceleration resource pool; The storage plane switch is used to perform cluster expansion on the storage nodes; The service plane switch is used to perform cluster expansion on the service nodes.

[0046] Heterogeneous acceleration server cluster expansion method based on high-speed serial computer expansion bus standard link Figure 5 As shown in the figure: the distributed acceleration server GPU nodes achieve cluster expansion through the parameter plane switch, 1:1 non-blocking; the business plane and storage plane cluster networks are connected to separate switches respectively, which can support 1:1 non-blocking or 2:1 convergence; the cluster out-of-band management network is connected to the TOR (Top of Rack) out-of-band management switch. Through independent physical ports or networks, the management data of the cluster is transmitted to the TOR switch to achieve cluster management and control. Out-of-band management is a management method independent of user business data transmission. It performs centralized and integrated management of network devices through dedicated management channels. In a cluster environment, the out-of-band management network is connected to the TOR out-of-band management switch, and the management data of the cluster is transmitted to the TOR switch through a special connection method, thereby achieving management and control of the entire cluster. This management method is usually used to ensure the high availability, manageability and security of the cluster. Specifically, the way the out-of-band management network is connected to the TOR out-of-band management switch can include: Use independent physical ports or networks to ensure that management data and user business data are transmitted on different links to avoid interference between management data and business data. Through the management interface provided by the TOR switch, management and monitoring of each node in the cluster can be achieved, including configuration, status monitoring, fault diagnosis, etc. Without interfering with normal business operations, network equipment can be remotely managed and maintained. At the same time, centralized management through the TOR switch can greatly improve management efficiency and accuracy, and reduce the risk of service interruption caused by manual intervention.

[0047] It should be noted that the cluster expansion topology solution can be adjusted according to the actual solution, and the present invention does not limit it.

[0048] Based on any of the above embodiments, the general computing resource pool includes: Multiple general computing units, each of which includes a general processor and memory. Any number of general processors and memories can be combined through virtualization software to form a computing resource container of any granularity.

[0049] In the embodiment of the present invention, the general computing resource pool is based on I / O centralization, and further decouples the CPU and memory to form a centralized computing resource pool. Assisted by software-defined capabilities, computers with massive CPUs and memories can be formed into computing resource containers of arbitrary granularity through virtualization software to form computing nodes of various scales and configurations.

[0050] Based on any of the above embodiments, the internal bus switching module includes an I / O switching chip controller, which is used to enumerate all I / O devices, define network topology division according to the resource requirements, dynamically adjust the matching between the acceleration processor and the I / O devices, and establish a resource allocation path.

[0051] The embodiments of the present invention fully adapt to the complexity of distributed heterogeneous acceleration pooled servers, provide reliable support for whole-machine management control and pooled resource management; develop a whole-machine management system for heterogeneous computing resource pools to improve the availability and manageability of server systems; develop a heterogeneous computing resource pool management engine to achieve reconstruction management and dynamic allocation of heterogeneous computing resources, and improve the usability and manageability of pooled resources. Meet the needs of unified monitoring and management of ultra-large-scale distributed heterogeneous acceleration pooled servers. Breakthroughs in key technologies such as complex topology automatic discovery, centralized asset management, and automatic switching of heterogeneous resources in heterogeneous computing pool systems, to achieve rapid on-demand deployment, automatic switching, and centralized management of physical resources in heterogeneous computing systems. The overall architecture of heterogeneous computing resource pool management is as follows: Figure 6 shown.

[0052] In an embodiment of the present invention, the system includes: A pooled management controller, wherein the general computing power unit in the general computing resource pool and the acceleration card of the heterogeneous computing acceleration resource pool are respectively provided with a pooled node management controller, and the pooled management controller uniformly manages multiple pooled node management controllers to perform at least one of node topology identification, centralized display of asset information, and coordinated power on and off for the general computing resource pool and the heterogeneous computing acceleration resource pool; The pooling management controller is used for at least one of node topology identification, centralized display of asset information, and coordinated power on and off.

[0053] In the embodiment of the present invention, the pooling management controller is further used to perform at least one of integrated monitoring, fault warning and visual management of the system through a standardized service interface.

[0054] Traditional servers usually use a general computing unit as the center and match a fixed number of accelerators. At the management level, the baseboard controller of the general computing unit manages the accelerator modules attached to it. For distributed heterogeneous acceleration pooled servers, the general computing unit and the heterogeneous computing acceleration unit perform resource decoupling, pooling and reconstruction through a high-performance switching unit, requiring the matching relationship between general computing and heterogeneous computing resources to be allocated on demand according to computing power requirements. The traditional management architecture centered on the general computing unit cannot perceive and coordinate the overall computing power resources, and cannot meet the requirements of automatic discovery of complex topologies, centralized asset management, and automatic switching of heterogeneous resources. Therefore, the embodiment of the present invention designs a hierarchical heterogeneous computing power resource pool management architecture, with the Pooled System Management Controller (PSMC) in the high-performance switching unit as the center, and the heterogeneous computing acceleration unit and the general computing unit are respectively matched with a pooled node management controller. The pooled management controller serves as the central management node, and the distributed nodes are uniformly managed by the central management node to achieve full life cycle management of the heterogeneous computing power resource pool.

[0055] The pooled management controller is crucial in the management of distributed heterogeneous acceleration pooled servers. It is a bridge for information communication and is responsible for unified management. Its main functions include node topology identification, centralized display of asset information, and coordinated power on and off. Through standardized service interfaces, it provides external operation and maintenance capabilities to achieve integrated monitoring, fault warning, and visual management. The heterogeneous computing resource pool management system interconnects the pooled node management controllers at all levels through the network to build an independent engine management platform.

[0056] In an embodiment of the present invention, the pooling management engine is used to manage the internal bus switching module, discover the current network topology by calling the physical channels and software interfaces in the internal bus switching module to obtain the current asset allocation situation, and coordinate the overall computing power resources to switch the network topology according to the physical channels and software interfaces to meet the resource requirements.

[0057] In the heterogeneous computing resource pool system, the discovery, management, and elastic adjustment of heterogeneous computing resources are key. This project designs a pooled management engine as the core management unit for dynamic resource adjustment. The pooled management engine will control high-performance switching units to achieve automatic discovery of resource topology, flexible and automatic resource switching, and provide a standard interface for the data center monitoring and management platform to achieve centralized management of massive resources at the data center level.

[0058] The pooled management controller serves as a centralized management layer, and the pooled management engine manages and controls high-performance I / O switching chips to achieve on-demand allocation and elastic expansion of general computing resources and heterogeneous computing resources.

[0059] Based on any of the above embodiments, for heterogeneous computing resource systems, the internal bus switching module as a high-performance switching unit is the infrastructure for connecting general computing units and heterogeneous computing units, building general resource pools and heterogeneous acceleration resource pools, and the pooled management engine is the glue that realizes resource management and switching scheduling between the data center monitoring and management platform and the infrastructure. The pooled management engine will control the high-performance switching unit to realize automatic discovery of resource topology, flexible and automatic switching of resources, and provide a standard interface for the data center monitoring and management platform to realize centralized resource scheduling and management. The overall software and hardware technical solution of the pooled management engine is as follows: Figure 7 As shown, the pooling management engine includes: Hardware layer, driver layer, function layer and interface layer; The hardware layer is used to connect the general computing resource pool and the multiple heterogeneous computing acceleration resource pools, and allocate resources to the general computing resource pool and the multiple heterogeneous computing acceleration resource pools; The driver layer is used for data transmission between the hardware layer and the function layer; The functional layer is used to realize the functional requirements required by users; The interface layer is used to obtain user demand information and provide visual information to the user.

[0060] In the embodiment of the present invention, the functional layer is used for but not limited to: (1) Information is exchanged with the pooled management controller through a universal asynchronous receiver / transmitter, and a network interface is provided to visualize a resource list and topology information, wherein the resource list and topology information include at least one of heterogeneous device resource pool information, universal computing unit information, and I / O port information.

[0061] (2) Establish expert templates with different performance indicators, and switch different expert templates according to the performance indicator requirements corresponding to the application scenario when the application scenario switches; And, a callable API is provided to display resource logical topology for the visualization application of the expert template according to the callable API.

[0062] In an embodiment of the present invention, the binding relationship between heterogeneous computing devices and the CPU at the physical link level is released to achieve on-demand elastic allocation of heterogeneous computing resources by general computing units. Different device allocation relationship expert templates can be established in the pooling management engine for different application scenarios. When the application scenario switches, the topology is automatically switched according to different templates, simplifying the application scenario switching process. Based on the network API interface provided by the pooling management engine, the system interface standard is studied, and the system framework standard is established to implement the expert template description and dynamic switching interface for heterogeneous computing resources, so that applications can apply for resources based on expert templates; at the same time, a callable API is provided to realize the visual application of expert templates, and a logical topology display of allocated resources is provided, such as Figure 8 shown.

[0063] In an embodiment of the present invention, based on parameters such as business requirements, resource status and performance indicators, expert template description and dynamic switching interface of heterogeneous computing resources are implemented, so that applications can apply for resources based on expert templates. At the same time, network interface and visual WEB interface are provided to realize the visual application of expert templates.

[0064] (3) Dynamically allocate and adjust heterogeneous computing units through hot removal, hot insertion, and hot reset of devices. In an embodiment of the present invention, the pooling management engine uses key technologies such as hot removal, hot insertion, and hot reset of devices to achieve dynamic allocation and adjustment of heterogeneous computing units in seconds.

[0065] (4) Dynamically and evenly allocate physical resources according to user needs, and release the computing power of heterogeneous computing units. In the embodiment of the present invention, the pooling management engine dynamically and evenly allocates physical resources based on the load balancing optimization scheduling algorithm, realizes fine-grained storage resource allocation on demand, and maximizes the release of heterogeneous computing power.

[0066] In the distributed heterogeneous acceleration pooled server, the general host resource pool and the heterogeneous acceleration resource pool are interconnected through high-performance switching units. The high-performance switching unit is composed of eight high-performance I / O switching chips. The high-performance I / O switching chip has a built-in embedded controller and provides dedicated physical channels and dedicated software interfaces for the pooled management engine to call. The pooled management engine uses these programmable interactive interfaces to achieve unified and centralized management and control of all high-performance I / O switching chips. The pooled management engine provides expert templates for different scenarios (such as deep learning, numerical calculation, etc.), flexibly implements topology switching, and implements a visual UI layer, providing a standard network interface.

[0067] In the heterogeneous computing resource pool, users can use the interface provided by the pool management engine to implement operations such as on-demand allocation, dynamic scaling, and resource release of general and heterogeneous computing units. Taking AI applications as an example, users can call different numbers of heterogeneous computing accelerator cards to match general computing resources according to load requirements. When the AI ​​application stops, users can release heterogeneous computing resources back to the entire heterogeneous computing resource pool to facilitate efficient resource circulation and full utilization.

[0068] Based on any of the above embodiments, in the related art, the prior art solution heterogeneous acceleration pool server system topology is as follows: Fig. 9 As shown in the figure, most of them use heterogeneous chips to support 8 NVIDIA A100 Tensor Core GPUs and 2 AMD Milan CPUs interconnected by NVIDIA NVLink in 4U space, and support both liquid cooling and air cooling technology. NF5688M6 is a GPU server with extreme scalability optimized for large-scale data centers. It supports 8 third-generation NVLink-interconnected NVIDIA A100 Tensor Core GPUs and two Intel Ice Lake CPUs, and supports up to 13 PCIeGen4 I / O cards.

[0069] In an embodiment of the present invention, the heterogeneous acceleration pooling server system is a single-machine 32-card heterogeneous acceleration pooling server system, and the single-machine 32-card heterogeneous acceleration pooling server system includes: Two groups of fully interconnected topologies containing four switching chips, supporting expansion of 32 general-purpose processor chip cards; The internal interconnection bandwidth of the single-machine 32-card heterogeneous acceleration pooled server system reaches 384GB / s, and the inter-card interconnection bandwidth reaches 128GB / s.

[0070] A single 32-card heterogeneous acceleration server system solution Fig.10 , 11 As shown: The entire cabinet can accommodate two sets of 32-card systems and support expansion of up to 64 GPU cards; the internal interconnection bandwidth of 16 cards reaches 384GB / s, and the cross-CPU interconnection bandwidth between 16 cards is 128GB / s.

[0071] Distributed heterogeneous acceleration resource pool servers are different from traditional servers. They are centered on multiple data exchange networks and I / O interconnection networks. The system architecture can be divided into general computing resource pools, high-performance exchange units, heterogeneous acceleration computing resource pools, and network exchange units according to different logical functions. Among them, the general computing resource pool is based on I / O centralization, further decoupling CPU and memory to form a centralized computing resource pool, and then supplemented by software-defined capabilities, it can form computing resource containers of any granularity through virtualization software, forming computing nodes of various scales and configurations; the high-performance exchange unit mainly realizes high-performance data exchange functions. By adopting a distributed data exchange architecture, it can realize flexible network topology division through software definition, quickly and dynamically adjust the matching between computing and I / O modules, realize dynamic combination between the two, effectively improve the scalability and flexibility of the entire system, and ensure the realization of hardware reconstruction; the heterogeneous acceleration resource pool mainly provides heterogeneous acceleration services, dynamically reconstructs CPU and GPU resources to form a server cluster of heterogeneous computing, and meets the computing power requirements of high-performance applications.

[0072] Based on any of the above embodiments, the heterogeneous acceleration pool server system is an 8-host shared 32-card heterogeneous acceleration pool server system, and the 8-host shared 32-card heterogeneous acceleration pool server system includes: Two groups of fully interconnected topologies with four switch chips, and eight hosts share 32 acceleration processor cards.

[0073] 8 hosts share 32 cards heterogeneous acceleration server system solution Fig.12 , 13 As shown, it supports dynamic allocation and rapid deployment of expert templates, and enables multiple hosts to share heterogeneous computing acceleration resource pools.

[0074] In the embodiment of the present invention, a single system supports expansion of 8 / 16 / 32 heterogeneous computing acceleration cards, supports no less than 8 domestically produced hosts to share a heterogeneous computing acceleration resource pool, and supports dynamic resource allocation in seconds.

[0075] Based on any of the above embodiments, the system further includes a computing power virtualization subsystem, which aggregates the heterogeneous computing acceleration resource pool into a unified virtual computing power resource pool, and the computing power virtualization subsystem includes: Pooled servers, which are computing nodes in the system cluster and are used for task execution; A control server, wherein the control server is used to run the control component in the system cluster; An application server, wherein the application server is used to run an application program in the system cluster, and the application server is a container, a virtual machine, or a physical machine.

[0076] In an embodiment of the present invention, the distributed heterogeneous accelerated pooled server is a new form of high-performance computing infrastructure, which is very different from traditional servers in terms of system architecture, hardware logic, infrastructure design, etc., and requires a more efficient and flexible management architecture to achieve life cycle management of pooled resources.

[0077] In the embodiment of the present invention, the logical relationship between the heterogeneous computing power software-defined virtualization management system and its subsystems and the pooled server system is as follows: Fig.14 As shown in the figure: By aggregating computing resources in a unified virtual computing resource pool, computing tasks can occupy resources when they are executed and not occupy resources when they are idle; for users, the cost of resource usage is reduced. For clusters, the turnover frequency of cluster resources can be increased, and limited resources can be provided to more people. It supports remote invocation of computing resources to break the limitation of space; it allows users to call computing resources on any node in the cluster. It supports virtualization of computing resources and divides one computing card into multiple computing cards. It matches computing resources with computing tasks to improve the utilization rate of resources. It supports the integration of various types of computing resources for one computing task across nodes. In this way, computing resources that are far greater than physical machine resources can be provided to computing tasks. It supports dynamic adjustment of virtual resources. During the operation of computing tasks, the virtual resources used by computing tasks can be dynamically adjusted, so as to support functions such as colocation and time-sharing. Colocation is to schedule different types of tasks to the same physical resources. Through control measures such as scheduling and resource isolation, it fully utilizes resource capabilities and reduces operating costs on the basis of guaranteeing service level agreements. This technology dynamically adjusts resource allocation according to the size of online pressure by deploying online services and computing tasks in a mixed manner. For example, when the online pressure is high, computing tasks will exit and occupy resources to ensure the smooth operation of online business; when the online pressure is low, computing tasks will occupy idle resources to improve resource utilization. Time-sharing is a temporal resource allocation strategy that allocates resources to different applications in different time periods to adapt to changes in demand. Through the time-sharing scheduling of operation and maintenance, resources are allocated to different applications according to different time periods to reduce resource procurement during the promotion period and further save costs. Through functions such as mixed deployment and time-sharing, the resource utilization of data centers can be significantly improved, operating costs can be reduced, and the stable operation of businesses can be guaranteed.

[0078] The heterogeneous acceleration pooled server system provided by the embodiment of the present invention designs a new type of heterogeneous computing acceleration card, a distributed heterogeneous acceleration pooled server system and a management and scheduling method, supports remote direct access to memory, and reduces the number of data hops by 50% compared with the traditional distributed typical path; based on the heterogeneous computing acceleration card hardware and software prototype verification platform, it carries out adaptation and optimization of multiple components such as processors, acceleration cards, networks, and operating systems to ensure the compatibility of acceleration card hardware and software. Breakthroughs are made in the development of distributed heterogeneous acceleration pooled server systems and the coordinated optimization of hardware and software, and research on the full life cycle management and scheduling technology of heterogeneous computing resource pools is carried out to achieve business-adaptive dynamic resource scheduling, support dynamic adjustment of processor / accelerator ratios, topology discovery and on-demand switching, and meet the needs of diversified application loads; realize remote calls of heterogeneous computing resources based on Ethernet, support TCP, RDMA data transmission protocols; and realize virtualization and dynamic scaling management of heterogeneous computing resources.

[0079] The embodiment of the present invention is application-oriented and takes system architecture design as the core. Through the multi-level collaborative design of "system-component-chip-software", it studies and develops a high-performance and high-efficiency distributed heterogeneous acceleration pool server system, realizes unified management and scheduling of heterogeneous computing resources, gives full play to the system innovation advantages of diversified computing power, and makes computing power easy to use, which is crucial to promoting the popularization and application of artificial intelligence technology and the development of my country's computing power industry. Master the core technology of distributed heterogeneous acceleration pool server and form an independent technology system. Heterogeneous computing is the main computing architecture adopted by the computing power infrastructure of data centers in the future. The lag in the current research on key technologies of heterogeneous computing has led to external dependence of important computing power infrastructure such as intelligent computing data centers and supercomputing data centers, which are restricted in computing power expansion and computing power management. Through the embodiment of the present invention, academia and enterprises are driven to carry out core technology research on distributed heterogeneous acceleration pool servers to form an independent technology system. Breakthrough the key technologies of heterogeneous computing acceleration cards, distributed heterogeneous acceleration pool servers, heterogeneous computing power resource pool life cycle management and scheduling technologies, and heterogeneous computing power resource virtualization and remote scheduling technologies, to build an independent and controllable industrial chain, value chain and ecosystem. Developing independent and controllable heterogeneous acceleration chips, heterogeneous acceleration cards, distributed heterogeneous acceleration pool servers and related supporting software, and carrying out demonstration applications in large-scale data centers will help promote the development of the computing power industry. It will drive the coordinated development of the upstream and downstream industrial chains and create huge economic benefits. It will drive more companies and research institutions to increase investment in heterogeneous computing core components and equipment, and provide strong support for the development of a new generation of information technology.

[0080] The pooled resource allocation method provided by the present invention is described below. The pooled resource allocation method described below and the heterogeneous acceleration pooled server system described above can be referenced to each other.

[0081] The pooled resource allocation method provided by the present invention is applicable to any of the above-mentioned heterogeneous acceleration pooled server systems, comprising: Step 101: A general computing unit in a general computing resource pool initiates a request for dynamic adjustment of heterogeneous computing resources to a pooling management controller according to resource requirements; Step 102: Determine the IO device to be accessed in the internal bus switching module according to the resource demand through the pooling management engine, access the IO devices in one or more high-speed data interconnect modules through the global unique code, reconstruct the heterogeneous computing resource allocation path, and match the heterogeneous computing resources corresponding to the resource demand for the general computing unit; The heterogeneous computing resources include resources in a single acceleration resource pool, resources in cross-acceleration resource pools, and resources in cross-pooled servers.

[0082] In the embodiment of the present invention, for the heterogeneous computing resource system, the internal bus switching module as a high-performance switching unit is the infrastructure for connecting the general computing unit and the heterogeneous computing unit, building the general resource pool and the heterogeneous acceleration resource pool, and the pooled management engine is the glue for realizing resource management and switching scheduling between the data center monitoring and management platform and the infrastructure. The pooled management engine will control the high-performance switching unit to realize automatic discovery of resource topology, flexible and automatic switching of resources, and provide a standard interface for the data center monitoring and management platform to realize centralized resource scheduling and management.

[0083] In the embodiment of the present invention, the distribution of resources is divided into three situations from the dimension of resource distribution, so that resources can be effectively utilized. On this basis, a scheduling system based on heterogeneous distance is developed to support heterogeneous resource scheduling capabilities of a single acceleration resource pool, across acceleration resource pools, and across pooled servers according to the scheduling strategy, thereby meeting the computing power requirements of different applications for pooled servers.

[0084] The embodiments of the present invention fully adapt to the complexity of distributed heterogeneous acceleration pooled servers, provide reliable support for whole-machine management control and pooled resource management; develop a whole-machine management system for heterogeneous computing resource pools to improve the availability and manageability of server systems; develop a heterogeneous computing resource pooling management engine to achieve reconstruction management and dynamic allocation of heterogeneous computing resources, and improve the usability and manageability of pooled resources. Meet the needs of unified monitoring and management of ultra-large-scale distributed heterogeneous acceleration pooled servers. Breakthroughs in key technologies such as complex topology automatic discovery, centralized asset management, and automatic switching of heterogeneous resources in heterogeneous computing power pooling systems, and achieve rapid on-demand deployment, automatic switching, and centralized management of physical resources in heterogeneous computing power systems.

[0085] Fig.15 A timing diagram of the pooled resource allocation method provided by an embodiment of the present invention, such as Fig.15As shown, the pooled resource allocation method provided by the embodiment of the present invention includes: (0) Taking the release of a heterogeneous accelerator card device and its allocation to a general computing unit as an example, before the user initiates a request for dynamic adjustment of heterogeneous computing resources through the pooling management engine, it is necessary to ensure that the application layer process related to the heterogeneous computing device has ended to avoid abnormal access to the device by the application layer, which may cause program abnormalities; (1) The pooling management engine completes the hot removal of the device; (2) The pooling management engine sends a request to the high-performance switching unit to obtain the physical location of the device; (3) The pooling management engine confirms the physical location of the device; (4) The pooling management engine resets the heterogeneous computing device resources and retrains them to restore the operating status to the default value; (5) The pooling management engine confirms that the device has been reset; (6) The pooling management engine reallocates the device to another general computing unit, which will see the newly added device without the business being aware of it, thus completing the dynamic switching of heterogeneous computing resources.

[0086] The embodiments of the present invention can realize cross-node and multi-host sharing, on-demand allocation and elastic application of resources, maximize the release of heterogeneous computing power, and achieve performance optimization in different application scenarios.

[0087] In an embodiment of the present invention, the pooled resource allocation method further includes: The control service receives a resource application initiated by the heterogeneous resource management communication component in the application server, and sends a configuration to the switching network according to the resource application; The configuration is sent to the heterogeneous virtual computing power management node component in the pooled server through the search exchange network, so that the pooled server loads the configuration to complete the virtualization deployment of computing power resources.

[0088] In the embodiment of the present invention, the heterogeneous intelligent computing power virtualization subsystem aggregates the accelerated computing resource pools on the pooled servers into a unified virtual computing power resource pool. Based on the unified virtual computing power resource pool, the high-speed network heterogeneous computing power resource remote call subsystem, heterogeneous virtual computing power management subsystem, heterogeneous computing cluster resource allocation and system performance optimization subsystem are responsible for managing and using the unified virtual computing power. The deployment of the heterogeneous computing power software-defined virtualization management system requires pooled servers, control servers and application servers. The specific subsystem module deployment locations and call links are as follows: Fig.16 shown.

[0089] In the embodiments of the present invention, for a cluster, the turnover frequency of cluster resources can be improved, and limited resources can be provided to more people. It supports remote invocation of computing resources, breaking the limitation of space; it allows users to call computing resources on any node in the cluster. It supports virtualization of computing resources, and divides a computing card into multiple computing cards for use. It matches computing resources with computing tasks, and improves the utilization rate of profit sources. It supports the integration of various types of computing resources for one computing task across nodes. Thus, computing resources that are far greater than the resources of physical machines can be provided to computing tasks.

[0090] The GPU nodes of the distributed acceleration server achieve cluster expansion through the parameter plane switch, 1:1 non-blocking; the business plane and storage plane cluster networks are connected to separate switches respectively, which can support 1:1 non-blocking or 2:1 convergence; the cluster out-of-band management network is connected to the TOR (Top of Rack) out-of-band management switch. Through independent physical ports or networks, the management data of the cluster is transmitted to the TOR switch to achieve cluster management and control. Out-of-band management is a management method independent of user business data transmission. It centrally integrates and manages network devices through dedicated management channels. In a cluster environment, the out-of-band management network is connected to the TOR out-of-band management switch, and the management data of the cluster is transmitted to the TOR switch through a special connection method, thereby achieving management and control of the entire cluster. This management method is usually used to ensure the high availability, manageability and security of the cluster.

[0091] During the operation of computing tasks, the virtual resources used by computing tasks can be dynamically adjusted to support functions such as colocation and time-sharing. Colocation is to schedule different types of tasks to the same physical resources. Through control measures such as scheduling and resource isolation, on the basis of guaranteeing the service level agreement, the resource capacity is fully utilized to reduce operating costs. This technology dynamically adjusts the allocation of resources according to the size of online pressure by deploying online services and computing tasks in a mixed manner. For example, when the online pressure is high, the computing tasks will exit and occupy resources to ensure the smooth operation of online business; when the online pressure is low, the computing tasks will occupy idle resources to improve resource utilization. Time-sharing is a resource allocation strategy in time, which allocates resources to different applications in different time periods to adapt to changes in demand. Through the time-sharing scheduling of operation and maintenance, resources are allocated to different applications according to different time periods to reduce resource procurement during the promotion period and further save costs. Through colocation, time-sharing and other functions, the resource utilization of data centers can be significantly improved, operating costs can be reduced, and the stable operation of businesses can be ensured.

[0092] The pooled resource allocation method provided in the embodiment of the present invention realizes the interconnection of servers between systems through an Ethernet switching module; adjusts the network port configuration and the heterogeneous computing resource allocation path according to the resource demand so that the general computing resource pool matches one or more heterogeneous computing acceleration resource pools corresponding to the resource demand, supports dynamic adjustment of the processor / accelerator ratio, topology discovery and on-demand switching, can realize business-adaptive dynamic resource scheduling, and meet the diverse application load requirements; improves the operating efficiency and performance of the heterogeneous acceleration pooled server system.

[0093] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the pooled resource allocation method provided by the above-mentioned methods, the method comprising: a general computing power unit in a general computing resource pool initiates a dynamic adjustment request for heterogeneous computing power resources to a pooled management controller according to resource requirements; adjusts the network port configuration and the heterogeneous computing resource allocation path through a pooled management engine to match the heterogeneous computing power resources corresponding to the resource requirements for the general computing power unit; resets and retrains the matched heterogeneous computing power resources, and allocates the trained acceleration processor to the general computing power unit to complete the dynamic switching of heterogeneous computing power resources; wherein the heterogeneous computing power resources include resources in a single acceleration resource pool, resources across acceleration resource pools, and resources across pooled servers.

[0094] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0095] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0096] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A heterogeneous acceleration pooled server system, characterized in that: include: General computing resource pool, internal bus switching module, Ethernet switching module and multiple heterogeneous computing acceleration resource pools; The general computing resource pool is connected to the internal bus switching module through a first intra-system interconnection bus; each heterogeneous computing acceleration resource pool is connected to the internal bus switching module through a second intra-system interconnection bus; each heterogeneous computing acceleration resource pool is connected to the Ethernet switching module, and the Ethernet switching module is used to interconnect servers between systems; The internal bus switching module includes a plurality of interconnected high-speed data interconnect modules, each of which is provided with a plurality of IO devices. The IO devices of the plurality of high-speed data interconnect modules are allocated a globally unique code in a coordinated manner so as to access any IO device in any high-speed data interconnect module through the globally unique code. After the general computing resource pool obtains the resource requirements, the required IO devices are configured according to the resource requirements through the internal bus switching module, and a heterogeneous computing resource allocation path is established to enable the general computing resource pool to match one or more heterogeneous computing acceleration resource pools corresponding to the resource requirements.

2. The heterogeneous acceleration pooled server system according to claim 1, characterized in that: The heterogeneous computing acceleration resource pool includes: At least one accelerator card, wherein the accelerator card has an integrated accelerator processor and an Ethernet controller; The acceleration processor is directly connected to the Ethernet controller; The Ethernet controller is connected to the Ethernet switching module.

3. The heterogeneous acceleration pooled server system according to claim 2, characterized in that: The system further comprises: A high-speed serial computer expansion bus standard link is provided, and the high-speed serial computer expansion bus standard link is used for interconnecting accelerated processors between multiple servers.

4. The heterogeneous acceleration pooled server system according to claim 2, characterized in that: The system further comprises a remote direct data access network card, wherein the remote direct data access network card communicates with the kernel driver of the acceleration processor via a kernel driver standard interface. The remote direct data access network card is connected to the Ethernet switching module, so that the multiple acceleration processors connected to the multiple remote direct data access network cards can complete the direct transmission of data and the remote direct access to the video memory.

5. The heterogeneous acceleration pooled server system according to claim 2, characterized in that: The internal bus switching module includes an I / O switching chip controller, which is used to enumerate all I / O devices, define network topology division according to the resource requirements, dynamically adjust the matching between the acceleration processor and the I / O devices, and establish a resource allocation path.

6. The heterogeneous acceleration pooled server system according to claim 1, characterized in that: The globally unique code is obtained by encoding the high-speed data interconnect module number and the IO device number inside it using a segmented encoding method.

7. The heterogeneous acceleration pooled server system according to claim 1, characterized in that: The general computing resource pool includes: Multiple general computing units, each of which includes a general processor and memory. Any number of general processors and memories can be combined through virtualization software to form a computing resource container of any granularity.

8. The heterogeneous acceleration pooled server system according to claim 1, characterized in that: The system also includes a pooling management controller; A pooled node management controller is respectively provided in the general computing power unit in the general computing resource pool and the acceleration card of the heterogeneous computing acceleration resource pool. The pooled node management controller manages multiple pooled node management controllers in a unified manner to perform at least one of node topology identification, centralized display of asset information, and coordinated power on and off for the general computing resource pool and the heterogeneous computing acceleration resource pool.

9. The heterogeneous acceleration pooled server system according to claim 8, characterized in that: The pooling management controller is also used to perform at least one of integrated monitoring, fault warning and visual management of the system through a standardized service interface.

10. The heterogeneous acceleration pooled server system according to claim 1, characterized in that: The system also includes a pooling management engine; The pooling management engine is used to manage the internal bus switching module, discover the current network topology by calling the physical channels and software interfaces in the internal bus switching module to obtain the current asset allocation status, and coordinate the overall computing power resources to switch the network topology according to the physical channels and software interfaces to meet the resource requirements.

11. The heterogeneous acceleration pooled server system according to claim 10, characterized in that: The pooling management engine is also used for: Hot removal, hot insertion and hot reset are performed on any acceleration card in the heterogeneous computing acceleration resource pool to control whether the acceleration card is put into use.

12. The heterogeneous acceleration pooled server system according to claim 10, characterized in that: The pooling management engine is also used for: Based on the load balancing optimization scheduling algorithm, the acceleration cards in the heterogeneous computing acceleration resource pool are dynamically allocated and adjusted.

13. The heterogeneous acceleration pooled server system according to claim 10, characterized in that: The pooling management engine is also used for: Establish expert templates for different application scenarios; When the application scenario is switched, the API is called according to the current application scenario to switch different expert templates; the expert templates that meet the requirements of the application scenario are visually displayed through a visual WEB interface.

14. The heterogeneous acceleration pooled server system according to claim 1, characterized in that: The heterogeneous acceleration pooled server system is a single-machine heterogeneous acceleration pooled server system, in which a single host supports 8 acceleration card expansions, 16 acceleration card expansions, and 32 acceleration card expansions.

15. The heterogeneous acceleration pooled server system according to claim 1, characterized in that: The heterogeneous acceleration pooled server system is a multi-host heterogeneous acceleration pooled server system, in which each host is connected to a high-speed data interconnection module, and 32 acceleration processor cards are shared through multiple high-speed data interconnection modules.

16. The heterogeneous acceleration pooled server system according to claim 1, characterized in that: The system further includes a computing power virtualization subsystem, which aggregates the heterogeneous computing acceleration resource pools into a unified virtual computing power resource pool. The computing power virtualization subsystem includes: Pooled servers, which are computing nodes in the system cluster and are used for task execution; A control server, wherein the control server is used to run the control component in the system cluster; An application server, wherein the application server is used to run an application program in the system cluster, and the application server is a container, a virtual machine, or a physical machine.

17. A pooled resource allocation method, applicable to the heterogeneous acceleration pooled server system according to any one of claims 1 to 16, characterized in that: include: A general computing unit in the general computing resource pool initiates a dynamic adjustment request for heterogeneous computing resources to the pooling management controller based on resource requirements; Determine the IO device to be accessed in the internal bus switching module according to the resource demand through the pooling management engine, access the IO devices in one or more high-speed data interconnect modules through the global unique code, reconstruct the heterogeneous computing resource allocation path, and match the heterogeneous computing resources corresponding to the resource demand for the general computing unit; The heterogeneous computing resources include resources in a single acceleration resource pool, resources in cross-acceleration resource pools, and resources in cross-pooled servers.

18. The method for allocating pooled resources according to claim 17, characterized in that: The process of reconstructing the heterogeneous computing resource allocation path includes releasing an accelerator card in a general computing unit and allocating it to other general computing units, specifically including: after the pooling management engine performs hot removal on the accelerator card, sending a request to the internal bus switching module to obtain the physical location of the accelerator card; Find the accelerator card according to the physical location and reset the accelerator card. Reassign the reset accelerator card to the general computing unit to be assigned.

19. The method for allocating pooled resources according to claim 18, characterized in that: The releasing of the accelerator card in a general computing unit and allocating it to other general computing units also includes: Training the reset accelerator card according to the application scenario of the general computing unit to be allocated; Re-allocate the trained accelerator card to the general computing unit to be allocated.

20. A non-transitory readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the pooled resource allocation method as described in any one of claims 17 to 19 is implemented.

Images