Resource scheduling method, electronic device, medium and product

Abstract

The application discloses a resource scheduling method, an electronic device, a medium and a product, relates to the technical field of AI intelligent computing, and comprises the following steps: a host obtains the bandwidth utilization rate of a main switching device in a plurality of switching devices; if the bandwidth utilization rate of the main switching device is greater than a first preset utilization rate, a first cyclic query strategy is used to query whether a standby switching device that meets a preset condition exists in the remaining switching devices except the main switching device in the plurality of switching devices; if not, a target switching device that meets a preset transmission condition is determined from the plurality of first switching machines; and the target switching device is used to store to-be-computed data to a shared memory pool. By introducing a multi-level heat perception, dynamic threshold adjustment, standby path switching and cross-network data unloading mechanism in the hybrid interconnection architecture of the shared memory pool, low congestion, high throughput and high resource utilization rate under large-scale AI training and reasoning tasks are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of AI (Artificial Intelligence) intelligent computing technology, in particular to a resource scheduling method, an electronic device, a medium and a product. BACKGROUND

[0002] With the wide application of deep learning models in natural language processing, computer vision and other fields, the demand for computing power, memory capacity and data exchange bandwidth for AI training and inference has increased dramatically. In a multi-host and multi-GPU hybrid architecture, Scale up network and Scale out network coexist, and the data flow path is complex. When congestion occurs in a certain interconnection subnet or resources are not fully utilized, there is a lack of effective cross-network data scheduling mechanism, which may result in idle GPU computing resources, lack of memory resources, and switch bandwidth congestion.

[0003] In related technologies, multiple focuses are on single congestion control, memory or performance improvement, which may cause cross-network congestion control fragmentation, uneven memory and computing power allocation, thereby limiting the computing power of large-scale AI systems and resource utilization efficiency, which needs to be solved urgently. SUMMARY

[0004] The present application provides a resource scheduling method, an electronic device, a medium and a product to at least solve the problems of communication bottleneck, global resource unable to be optimized cooperatively and low resource utilization rate caused by cross-network congestion control fragmentation in AI systems in related technologies.

[0005] The present application provides a resource scheduling method, which is applied to a heterogeneous computing system including a host, a first switch and a shared memory pool, the host and the shared memory pool are connected with the first switch, and the host includes multiple switching devices. The method includes the following steps:

[0006] acquiring bandwidth utilization of a main switching device in the multiple switching devices through the host;

[0007] if the bandwidth utilization of the main switching device is greater than a first preset utilization rate, querying whether there is a standby switching device meeting a preset device condition in the remaining switching devices except the main switching device in the multiple switching devices based on a first cyclic query strategy;

[0008] if there is no standby switching device meeting the preset device condition, determining a target switch meeting a preset transmission condition from the multiple first switches, and storing to-be-computed data to the shared memory pool through the target switch.

[0009] The application further provides a resource scheduling device, which is applied to a heterogeneous computing system, the heterogeneous computing system comprising a host, a first switch and a shared memory pool, the host and the shared memory pool being connected with the first switch, the host comprising a plurality of switching devices, wherein the device comprises:

[0010] an acquisition module, configured to acquire a bandwidth utilization rate of a master switching device in the plurality of switching devices through the host;

[0011] a query module, configured to, if the bandwidth utilization rate of the master switching device is greater than a first preset utilization rate, query whether a standby switching device meeting a preset device condition exists in the remaining switching devices except the master switching device in the plurality of switching devices based on a first cyclic query strategy;

[0012] a storage module, configured to, if the standby switching device meeting the preset device condition does not exist, determine a target switch meeting a preset transmission condition from the plurality of first switches, and store to-be-computed data to the shared memory pool through the target switch.

[0013] The application further provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and capable of running on the processor, the processor executing the program to implement the steps of the resource scheduling method according to any one of the above.

[0014] The application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the resource scheduling method according to any one of the above.

[0015] The application further provides a computer program product, wherein the computer program is executed by a processor to implement the steps of the resource scheduling method according to any one of the above.

[0016] Through the application, the bandwidth utilization of the master switching device in the plurality of switching devices is acquired by the host computer, if the bandwidth utilization of the master switching device is greater than a first preset utilization rate, whether there is a standby switching device that meets the preset device condition in the remaining switching devices except the master switching device in the plurality of switching devices is inquired based on a first cyclic inquiry strategy, if not, a target switching device that meets the preset transmission condition is determined from the plurality of first switching devices, and the to-be-calculated data is stored to the shared memory pool through the target switching device, which solves the technical problems of communication bottleneck caused by cross-network congestion control fragmentation in the AI system, global resources cannot be cooperatively optimized, and low resource utilization, through introducing multi-level heat perception, dynamic threshold adjustment, standby path switching and cross-network data offloading mechanism in the mixed interconnection architecture of Scale-up+Scale-out+CXL (Compute Express Link, Compute Express Link), low congestion, high throughput and high resource utilization under large-scale AI training and inference tasks are realized. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0018] Figure 1 A flowchart of a resource scheduling method according to an embodiment of the present application is provided.

[0019] Figure 2 A system implementation scheme block diagram according to an embodiment of the present application is provided.

[0020] Figure 3 A multi-path standby switching device enabling scheme diagram according to an embodiment of the present application is provided.

[0021] Figure 4 A dynamic collaboration scheme diagram of video memory and computing power according to an embodiment of the present application is provided.

[0022] Figure 5 A block diagram of a resource scheduling device according to an embodiment of the present application is provided.

[0023] Figure 6 A structural diagram of an electronic device according to an embodiment of the present application is provided. DETAILED DESCRIPTION

[0024] With reference to the drawings and the specific embodiments described below, the technical solutions in the embodiments of the present application will be better understood by those skilled in the art.

[0025] It should be noted that in the description of the present application, the terms "comprising", "containing" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. The terms "first", "second" and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0026] In order for those skilled in the art to better understand the technical solutions of the present application, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0027] The embodiments of the present application provide a heterogeneous computing system, and the system is described in detail in combination with the heterogeneous computing system.

[0028] Specifically, before introducing the embodiments of the present application, first introduce the hardware architecture of the related technology adopted in the artificial intelligence server, mainly including: multi-GPU (Graphics Processing Unit, graphics processing unit) Scale up architecture based on PCIe (Peripheral Component Interconnect Express, high-speed serial computer expansion bus standard) or NVLink, direct data exchange between GPUs is realized through high-speed interconnection bus and switching chip, part of the system supports P2P (Peer-to-Peer, point-to-point) communication between GPUs to reduce cross-device access delay, however, in order to support larger scale AI model training / inference or more cumulative context and token amount, only the number of GPUs in the network can be increased or the data can be unloaded to the host DRAM, this way has limited video memory improvement and will greatly increase the cost, and will increase the probability of forming congestion on the hot link, resulting in increased data transmission delay; in addition, the Scale out architecture based on RDMA (Remote Direct Memory Access, remote direct memory access) protocol, using RDMA protocol to interconnect multiple host nodes, allowing a host node to directly read and write the memory of another host node without the intervention of the CPU and without passing through the operating system kernel, thereby reducing the cross-node communication delay to microseconds.

[0029] Generally, when congestion occurs, the congestion control mechanism mainly includes the following categories:

[0030] (1) Congestion control based on explicit feedback: When the switch detects that the queue length exceeds the threshold, it marks the ECN (Explicit Congestion Notification) bit in the data packet. The receiving end feeds back to the sending end after receiving the mark, and the sending end reduces the sending rate according to the feedback information, thereby preventing the queue from continuously growing;

[0031] (2) Flow control based on rate limiting: The sending end dynamically adjusts the data sending rate according to the link bandwidth utilization and historical packet loss to avoid link overload;

[0032] (3) Priority-based queue scheduling: RDMA flows are classified by priority, and different priority flows occupy different queue resources in the switch, ensuring that critical task flows can still obtain low delay and stable bandwidth in the case of congestion;

[0033] (4) Priority-based queue scheduling: RDMA flows are classified by priority, and different priority flows occupy different queue resources in the switch, ensuring that critical task flows can still obtain low delay and stable bandwidth in the case of congestion;

[0034] Although the above methods can effectively alleviate link congestion in a single network environment, in a multi-GPU + multi-host hybrid architecture coexisting with PCIe networks, RDMA congestion control and PCIe congestion control are independent of each other, lacking a global traffic scheduling mechanism across networks, making it difficult to achieve optimal resource utilization in the overall system. Therefore, the main problems are as follows:

[0035] (1) Lack of comprehensive optimization mechanism for large-scale models, most existing solutions are mainly for local improvement of single problems (such as memory expansion, single network congestion control or node performance optimization), and the overall solution lacks global cross-network optimization capability for large-scale AI model training / inference, making it difficult to meet the demand for low delay and high throughput at the same time;

[0036] (2) Cross-network congestion control is fragmented, and the congestion detection and scheduling mechanisms of PCIe / NVLink networks and RDMA networks are independent of each other, lacking global collaborative optimization. When one side of the network appears bottleneck, the link or computing resources on the other side may be in idle state, resulting in a decrease in overall computing power utilization;

[0037] (3) GPU memory and computing power allocation is uneven, in the existing architecture, the allocation strategy of GPU memory and computing power does not realize real-time dynamic cooperation, which is easy to cause GPU memory overflow, GPU memory idling or GPU waiting for data transmission and idling, affecting the training and inference efficiency;

[0038] (4) Uneven utilization of multi-exchange chip link, in large-scale training tasks, data flow through multiple exchange chips is easy to form a hot link, and the existing mechanism lacks a dynamic flow rerouting function based on real-time link utilization, which leads to local congestion limiting the overall throughput, and the transmission rate of low-priority transactions also makes the transmission performance of the hot link not optimal.

[0039] Therefore, based on the above problems, the embodiments of the present application mainly solve: (1) mixed architecture optimization adapting to large-scale AI tasks, constructing a multi-GPU+multi-host+CXL shared memory pool architecture adapting to large-scale AI tasks, replacing RDMA protocol with CXL shared memory pool to complete data interaction between hosts, while considering high throughput, low latency and high resource utilization in large-scale model training, inference and multi-modal processing scenarios; (2) cross-network congestion control mechanism, establishing a global congestion detection and scheduling method covering scaleup network and scaleout network, realizing dynamic coordination of traffic between different network domains, and avoiding single-sided bottleneck affecting global performance; (3) dynamic cooperative scheduling of memory and computing power, based on real-time memory utilization rate of GPU, computing power utilization rate and available CXL shared memory pool, dynamically optimizing memory and computing resource allocation to reduce memory overflow and computing power idling; (4) global optimization of multi-exchange chip link bandwidth, a new multi-exchange chip cross-network flow rerouting strategy based on link bandwidth utilization is proposed to alleviate hot link congestion and improve overall network throughput.

[0040] Specifically, Figure 1 A flowchart of a resource scheduling method provided by the embodiments of the present application is shown in the figure. The method is applied to a heterogeneous computing system, which includes a host, a first switch and a shared memory pool. The host and the shared memory pool are connected with the first switch, and the host includes multiple exchange devices.

[0041] First, the heterogeneous computing system and its architecture involved in the present application are introduced.

[0042] Specifically, to address the aforementioned problems, the resource scheduling method of this application is applied to a heterogeneous computing system, such as an efficient hybrid network congestion control and resource scheduling system for large-scale AI training and inference. In the heterogeneous computing system, it mainly includes a host, a first switch, and a shared memory pool (e.g., a CXL shared memory pool). The host and the shared memory pool are both connected to the first switch. The host includes multiple switching devices. At the same time, this heterogeneous computing system is suitable for training and inference of large-scale AI models and adopts a hybrid scale-up and scale-out network architecture with multiple hosts and multiple GPUs.

[0043] Among them, such as Figure 2 As shown, the first switch is a high-speed serial computer expansion bus standard switch (e.g., a CXL switch), and the switching devices are compute fast connection switches (e.g., PCIe switches). The host is configured to sequentially query the bandwidth utilization of the switching devices based on a first round-robin query strategy, starting from the main PCIe switch and querying the bandwidth utilization of the PCIe switches in a fixed order, ensuring the fairness and predictability of the query. The host includes multiple switching devices, and among these devices are multiple GPUs that communicate with at least some of the switching devices. The GPUs within each host are interconnected via a scale-up network. Each host is equipped with 8 PCIe switches, and each PCIe switch connects all 8 GPUs, thereby achieving high-speed data exchange between GPUs. Simultaneously, the multiple hosts are interconnected via a scale-out network. The shared memory pool includes multiple (e.g., 8) compute fast connection controllers (e.g., CXL controllers), meaning they share a CXL shared memory pool composed of 8 CXL controllers. Each CXL controller supports multiple remote direct memory access channels; for example, each CXL controller supports 5 CXL DRAM channels. Each host is connected to all CXL switches. In this embodiment, all CXL switches... All switches are connected to all CXL controllers, thus forming a high-bandwidth, low-latency cross-node memory access channel.

[0044] Furthermore, in this heterogeneous computing system, each GPU monitors its own memory utilization and computing power utilization in real time, while the PCIe switch and CXL switch monitor the bandwidth utilization of each port in real time, thereby forming fine-grained link load monitoring data to provide a basis for subsequent congestion detection and scheduling.

[0045] Therefore, this application constructs a multi-GPU + multi-host + CXL shared memory pool architecture adapted to large-scale AI tasks. The CXL shared memory pool replaces the RDMA protocol to complete data interaction between multiple hosts, while ensuring high throughput, low latency and high resource utilization in scenarios such as large-scale model training, inference and multimodal processing.

[0046] The resource scheduling method of the embodiment of the application comprises the following steps:

[0047] In step S101, the host obtains the bandwidth utilization of the primary switching device in the plurality of switching devices.

[0048] According to one embodiment of the application, before the host obtains the bandwidth utilization of the primary switching device in the plurality of switching devices, the method further comprises: determining whether there is a data transmission requirement; if there is a data transmission requirement, determining a target acceleration unit based on the data transmission requirement, and calculating a first transmission path between the target acceleration unit and at least part of the switching devices; and based on the first transmission path between the target acceleration unit and at least part of the switching devices, taking the switching device with the shortest first transmission path between the target acceleration unit and at least part of the switching devices as the primary switching device.

[0049] Specifically, as shown in Figure 2 , in the Scale-up network, after a transaction is transmitted by a GPU to a PCIe switch, the transaction is routed by the PCIe switch to a receiving GPU, and the data transmission between GPUs first selects the PCIe switch closest to the receiving GPU as the primary PCIe switch for data transmission, and when the bandwidth utilization of the primary PCIe switch exceeds a preset threshold, the system allows the transaction transmitted to the receiving GPU to switch to another PCIe switch to relieve the hotspot link pressure.

[0050] Specifically, as shown in Figure 3 , first, it is determined whether there is a data transmission requirement, and if there is a data transmission requirement, a target acceleration unit of the embodiment of the application is determined based on the data transmission requirement, which is the GPU for transaction reception, and then a first transmission path between the target acceleration unit and at least part of the switching devices is calculated, a plurality of first transmission paths can be obtained, and in the plurality of first transmission paths, the switching device with the shortest first transmission path is determined as the primary switching device, that is, the PCIe switch closest to the GPU for transaction reception is taken as the primary PCIe switch for data transmission.

[0051] Therefore, by selecting the PCIe switch closest to the GPU for transaction reception as the primary PCIe switch for data transmission, the physical delay of data transmission can be directly and effectively reduced, load balancing can be achieved, routing decisions can be simplified, the system can quickly respond to each new transaction request, and the overall scheduling efficiency and real-time performance are improved.

[0052] In step S102, if the bandwidth utilization of the main switching device is greater than the first preset utilization, whether there is a standby switching device meeting the pre-device condition in the remaining switching devices except the main switching device in the plurality of switching devices is queried based on the first cyclic query strategy.

[0053] According to one embodiment of the present application, whether there is a standby switching device meeting the pre-device condition in the remaining switching devices except the main switching device in the plurality of switching devices is queried based on the first cyclic query strategy, comprising: determining a target switching device according to the main switching device and obtaining the bandwidth utilization of the target switching device and the heat value of the target switching device based on the first cyclic query strategy; if the heat value of the target switching device is less than the first preset heat value, whether the bandwidth utilization of the target switching device is less than the second preset utilization is judged; if the bandwidth utilization of the target switching device is less than the second preset utilization, it is determined that there is a standby switching device meeting the pre-device condition.

[0054] According to one embodiment of the present application, after it is determined that there is a standby switching device meeting the pre-device condition, it further comprises: taking the target switching device as the standby switching device and determining the data transmission path between the standby switching device and the target acceleration unit; transmitting the to-be-computed data to the target acceleration unit through the standby switching device based on the data transmission path.

[0055] Wherein, the first preset utilization, the second preset utilization, the first preset heat value and the pre-device condition can be set by the person skilled in the art according to the actual use requirement, or can be obtained by simulation for a limited number of times, which is not limited here.

[0056] Specifically, as shown in Figure 3 As one implementation, first, PCIe switch heat query control is performed by the main PCIe switch, the main PCIe switch maintains a PCIe switch heat table, all PCIe switches are initially with a heat value of 0, the host starts from the main PCIe switch and queries the bandwidth utilization of each PCIe switch in turn to the right, i.e. the bandwidth utilization of the main PCIe switch is queried first in the first time, if the bandwidth utilization of the main PCIe switch is greater than or equal to the first preset utilization (for example, the bandwidth utilization exceeds 70%), whether there is a standby switching device meeting the pre-device condition in the remaining switching devices except the main switching device in the plurality of switching devices is queried based on the first cyclic query strategy, if there is, a target switching device is determined according to the main switching device, and the bandwidth utilization of the target switching device and the heat value of the target switching device are obtained.

[0057] Specifically, when the bandwidth utilization of the main PCIe switch is greater than the first preset utilization rate, a backup path query mechanism is triggered, and based on the first cyclic query strategy, the next PCIe switch to the right is queried in turn, and the next PCIe switch is taken as a target PCIe switch, and the bandwidth utilization and the heat value of the target PCIe switch are obtained. Then, it is judged whether the heat value of the target PCIe switch is less than the first preset heat value (for example, 5). That is, after the bandwidth utilization and the heat value of the target switch device are obtained, if the heat value of the target PCIe switch is less than 5, the heat value of the target PCIe switch is increased by 1, that is, in the query process, after passing through one PCIe switch, if the heat value of the PCIe switch is less than 5, the heat value of the PCIe switch is increased by 1, and then it is further judged whether the bandwidth utilization of the next switch device is less than the second preset utilization rate. If the bandwidth utilization of the next switch device is less than the second preset utilization rate, it is indicated that there is a backup switch device meeting the preset device utilization condition at this time, the next switch device is taken as a target switch device, the target switch device is the backup switch device, and then the data transmission path between the backup switch device and the target acceleration unit is determined, the target acceleration unit is a GPU, so that the to-be-calculated data can be transmitted to the target acceleration unit through the backup switch device based on the data transmission path, that is, the system instructs the GPU receiving the transaction to transmit data, and enables the target PCIe switch as a backup PCIe switch to transmit data.

[0058] It should be noted that if a PCIe switch does not respond in the query process, the bandwidth utilization of the PCIe switch is considered to be 100%, and after the rightmost PCIe switch is queried, the host computer continues to cyclically query the leftmost PCIe switch, forming a ring query mechanism.

[0059] Therefore, by using the heat value as a core index, the path selection, congestion prevention, load balancing and other functions are organically integrated together, the adaptive scheduling of data is realized, the fluency and efficiency of the ring query are maintained, the discovery time of the backup path is quickly shortened, and the overall communication delay caused by the path selection delay is reduced.

[0060] According to one embodiment of the application, before obtaining the bandwidth utilization of the target switch device and the heat value of the target switch device, it further comprises: based on the first cyclic query strategy, determining the first side of the main switch device and the second side of the main switch device, and obtaining the heat value of the first side and the heat value of the second side; and updating the first preset heat value according to the heat value of the first side and the heat value of the second side.

[0061] According to one embodiment of the present application, determining the first side of the main switching device and the second side of the main switching device based on the first cyclic query strategy comprises: determining a query direction according to the first cyclic query strategy; and based on the main switching device, taking the side opposite to the query direction as the first side and taking the side same as the query direction as the second side.

[0062] According to one embodiment of the present application, updating the first preset heat value according to the heat value of the first side and the heat value of the second side comprises: judging whether the heat value of the first side is greater than the heat value of the second side; if the heat value of the first side is greater than the heat value of the second side, updating the first preset bandwidth utilization rate to a third preset bandwidth utilization rate, otherwise, updating the first preset bandwidth utilization rate to a fourth preset bandwidth utilization rate; wherein the third preset bandwidth utilization rate is less than the first preset bandwidth utilization rate, and the fourth preset bandwidth utilization rate is greater than the first preset bandwidth utilization rate.

[0063] Specifically, before obtaining the bandwidth utilization rate and the heat value of the target switching device, the threshold value needs to be adjusted based on the measured heat value of the main switching device to prevent the congestion of the hardware link caused by the too high heat value of one side.

[0064] Specifically, in the embodiment of the present application, first, based on the first cyclic query strategy, that is, based on the query direction of the main switching device querying the next PCIe switch to the right in turn, the side opposite to the query direction is taken as the first side, that is, the first side of the main switching device is the left side, and the side same as the query direction is taken as the second side, that is, the second side of the main switching device is the right side, and the heat value of the first side and the heat value of the second side are obtained; then, it is judged whether the heat value of the first side is greater than the heat value of the second side, if the heat value of the first side is greater than the heat value of the second side, it indicates that the first side link is more congested, and the bandwidth utilization rate of the first side needs to be reduced, so that the first side of the main switching device can continue to accommodate more load, therefore, when the heat value of the first side is greater than the heat value of the second side, the first preset bandwidth utilization rate (for example, 70%) needs to be updated to the third preset bandwidth utilization rate (for example, 50%); on the contrary, if the heat value of the first side is less than the heat value of the second side, it indicates that the second side link is more congested, and the bandwidth utilization rate of the first side can continue to accommodate more load, at this time, the bandwidth utilization rate of the first side can be improved, and the first preset bandwidth utilization rate is updated to the fourth preset bandwidth utilization rate (for example, 90%) to improve the link utilization rate.

[0065] That is, in the threshold dynamic adjustment aspect, the first preset utilization rate can be set to 70% of the PCIe switch bandwidth utilization rate, and if the PCIe switch heat value on the left side of the main PCIe switch is higher than that on the right side, it indicates that the left side link is more congested, at this time, the first preset utilization rate can be reduced to 50% to accommodate more load; otherwise, if the left side heat value is lower than the right side, the first preset utilization rate is increased to 90%, and the link utilization rate is increased.

[0066] Therefore, by introducing the "threshold dynamic adjustment based on adjacent link heat comparison" mechanism, the system can perceive the "heat value" difference of the links on both sides of the main switch, thereby dynamically adjusting the congestion judgment threshold of the main path, so that it has an intelligent control system with environmental perception and self-regulation ability, significantly improving the intelligent level of the algorithm, and realizing active avoidance of local hot spots, avoiding the flow from continuing to flow into the already overloaded left side link, and enhancing the robustness of the system in dealing with unbalanced load.

[0067] According to one embodiment of the application, after obtaining the bandwidth utilization rate of the target switching device and the heat value of the target switching device, if the heat value of the target switching device is greater than or equal to the first preset heat value, the heat value of the target switching device is reduced according to the preset heat reduction strategy, and the next switching device is determined based on the first cycle query strategy according to the main switching device; the next switching device is taken as the target switching device, and the steps of obtaining the bandwidth utilization rate of the target switching device and the heat value of the target switching device are re-executed until it is determined that the heat value of at least part of the switching devices is greater than or equal to the first preset heat value or the bandwidth utilization rate of at least part of the switching devices is greater than or equal to the second preset utilization rate, and it is determined that there is no standby switching device that meets the preset condition.

[0068] Specifically, as Figure 3As another implementation manner shown, when the bandwidth utilization of the main PCIe switch is greater than the first preset utilization, a backup path query mechanism is triggered, the next PCIe switch is sequentially queried to the right based on the first cyclic query strategy, the next PCIe switch is taken as a target PCIe switch, and the bandwidth utilization and the hotness value of the target PCIe switch are obtained. Then, it is judged whether the hotness value of the target PCIe switch is greater than or equal to the first preset hotness value, for example, whether the hotness value of the target PCIe switch is greater than or equal to 5. If the hotness value of the target PCIe switch is greater than or equal to 5, in order to prevent system instability and realize intelligent scheduling, the preset hotness reduction strategy needs to be used to reduce the hotness value of the target switching device, that is, when the hotness value of the target PCIe switch is greater than or equal to 5, the query of the target PCIe switch is skipped, which means that even if the current bandwidth utilization of the target PCIe switch is lower than the first preset utilization, the system will temporarily ignore it and reduce the hotness value of the target PCIe switch by 1. During this period, the system will continue to determine the next switching device based on the first cyclic query strategy according to the main switching device, that is, continue to sequentially query the next PCIe switch to the right as a backup path, and re-execute the steps of obtaining the bandwidth utilization of the target switching device and the hotness value of the target switching device, until it is determined that the hotness values of at least part of the switching devices are greater than or equal to the first preset hotness value or the bandwidth utilizations of at least part of the switching devices are greater than or equal to the second preset utilization, that is, it is determined that there is no backup switching device meeting the preset device utilization condition, that is, the backup switching device cannot be enabled at this time, so that the traffic needs to be dispersed from the hot path.

[0069] Therefore, through the intervention measure of actively reducing the hotness value of the target switching device, the traffic can be guided to bypass in advance before the hot path really becomes a bottleneck, so that the probability of actual congestion is significantly reduced, and the stability of the network and the smoothness of data transmission are improved.

[0070] According to an embodiment of the application, after judging whether the bandwidth utilization of the target switching device is less than the second preset utilization, the method further comprises: if the bandwidth utilization of the target switching device is greater than or equal to the second preset utilization, determining the next switching device based on the first cyclic query strategy according to the main switching device; taking the next switching device as the target switching device, and re-executing the steps of obtaining the bandwidth utilization of the target switching device and the hotness value of the target switching device, until it is determined that the hotness values of at least part of the switching devices are greater than or equal to the first preset hotness value or the bandwidth utilizations of at least part of the switching devices are greater than or equal to the second preset utilization, and it is determined that there is no backup switching device meeting the preset device utilization condition.

[0071] Specifically, if the bandwidth utilization of the target switch device is greater than or equal to the second preset utilization, it indicates that the data transmission has exceeded the upper limit and cannot be forwarded in time, thereby increasing the speed of data processing, and therefore it is necessary to further query the next PCIe switch device to the right in sequence, and then take the next PCIe switch device as the target switch device, and re-execute the steps of obtaining the bandwidth utilization and the heat value of the target switch device, until it is determined that the heat value of at least part of the switch devices is greater than or equal to the first preset heat value or the bandwidth utilization of at least part of the switch devices is greater than or equal to the second preset utilization, and it is determined that there is no standby switch device that meets the preset device utilization condition. The step has been described above, and to avoid redundancy, it will not be discussed in detail here.

[0072] Therefore, when the bandwidth utilization of the target switch device is greater than or equal to the second preset utilization, querying the next PCIe switch device to the right ensures that the query process will not stop or jump randomly, and at the same time the system can sequentially and completely check each possible PCIe switch device, avoiding the unfair phenomenon that some switch devices are ignored for a long time and some switch devices are excessively queried due to random selection, thereby ensuring that all potential available paths have the opportunity to be discovered and utilized.

[0073] According to an embodiment of the present application, after transmitting the to-be-computed data to the target acceleration unit through the standby switch device based on the data transmission path, the method further comprises: obtaining a current transmission progress of the standby switch device in transmitting the to-be-computed data to the target acceleration unit; and if the current transmission progress is complete transmission, reducing the heat value of the target switch device according to a preset heat reduction strategy.

[0074] Specifically, after the bandwidth utilization of the target switch device is greater than or equal to the second preset utilization and the next PCIe switch device is queried to the right in sequence, the current transmission progress of the standby switch device in transmitting the to-be-computed data to the target acceleration unit is obtained, if all switch devices are queried, i.e. the current transmission progress is complete transmission, and the heat value of all switch devices is greater than 5, it is considered that the Scale-up network is in a full load state, and after the data transmission is completed, the PCIe switch heat value of the related link is reduced by 1 according to the preset heat reduction strategy, thereby ensuring that the heat table dynamically reflects the link load condition.

[0075] Thus, when the Scale-up network is in a full load state, at this time, it indicates that the effort to find a backup path between all available switches has failed, all paths are "overheated" due to frequent use, indicating that the elastic scheduling capability of the Scale-up network itself has been exhausted, and the overall load has reached the processing limit of its architecture, so the PCIe switch heat value is reduced, and the system is no longer limited to querying between PCIe switches, but opens a new high-bandwidth escape channel, that is, a CXL shared memory pool, which provides elastic scheduling capability across network domains for data flow, and transfers the pressure from the saturated Scale-up network to the relatively idle Scale-out network (CXL part below).

[0076] Specifically, after determining that the Scale-up network is in a full load state, the bandwidth utilization and heat value of the host's nearest high-speed serial computer expansion bus standard switch are obtained. When all PCIe switches in the Scale-up network have a heat value of 5, the system determines that the network has reached full load, and automatically opens a GPU data transaction offloading channel to store the transaction data to be transmitted by the GPU in the CXL controller of the shared memory pool (CXL shared memory pool) through the host using the Memory-scale-up network.

[0077] Further, the data transmission of the Memory-scale-up network uses a heat query mechanism similar to the Scale-up network to monitor the bandwidth utilization of the CXL switch. The host maintains a CXL switch heat table, and the query process is similar to the PCIe switch described above. By dynamically enabling a backup CXL switch to transmit transactions, single-path congestion can be effectively avoided. The CXL switch heat value is also limited to a maximum of 5, and when the limit is reached, the query is skipped and the CXL switch heat value is reduced by 1.

[0078] Further, after placing the transaction data to be transmitted in the CXL controller, the sending GPU continues the subsequent computing task, and the host repeatedly sends the data location to the host PCIe switch of the GPU until it is received. When the data transmission occupying the bandwidth is completed, the PCIe switch first determines whether there is transaction data to be transmitted. If there is transaction data to be transmitted, the PCIe switch will preferentially receive the transaction data to be transmitted stored in the CXL controller.

[0079] Thus, after the network is in a full load state, the transaction data to be transmitted is stored in the shared memory pool, actively avoiding potential congestion points, significantly reducing data transmission delay and packet loss risk, improving communication reliability and predictability, and avoiding performance degradation due to congestion.

[0080] In step S103, if there is no standby switching device satisfying the preset condition, a target switching device satisfying the preset transmission condition is determined from the plurality of first switching devices, and the to-be-calculated data is stored to the shared memory pool through the target switching device.

[0081] According to an embodiment of the present application, the target switching device satisfying the preset transmission condition is determined from the plurality of first switching devices, comprising: determining a second transmission path between the host and at least part of the first switching devices; determining, based on the second transmission path between the host and at least part of the first switching devices, a first switching device with the shortest second transmission path between the host and the first switching device as a standby switching device; obtaining a bandwidth utilization of the standby switching device and a hotness value of the initial switching device, and determining whether the hotness value of the standby switching device is less than a second preset hotness value; if the hotness value of the standby switching device is less than the second preset hotness value, determining whether the bandwidth utilization of the standby switching device is less than a third preset utilization; if the bandwidth utilization of the standby switching device is less than the third preset utilization, determining the standby switching device as the target switching device.

[0082] According to an embodiment of the present application, after determining whether the hotness value of the standby switching device is less than the second preset hotness value, further comprising: if the hotness value of the standby switching device is greater than or equal to the second preset hotness value, reducing the hotness value of the standby switching device according to a preset hotness reduction strategy, and determining a next switching device based on a second loop query strategy according to the host; determining the next switching device as the standby switching device, and re-executing the steps of obtaining the bandwidth utilization of the standby switching device and the hotness value of the standby switching device until determining that the hotness value of the standby switching device is less than the second preset hotness value.

[0083] According to an embodiment of the present application, after storing the to-be-calculated data to the shared memory pool through the target switching device, further comprising: determining whether the bandwidth utilization and the hotness value of the host switching device satisfy corresponding conditions; if the bandwidth utilization and the hotness value of the host switching device satisfy corresponding conditions, receiving the to-be-transmitted transaction data through the host switching device, and transmitting the to-be-transmitted transaction data to an acceleration unit for transaction receiving.

[0084] The preset transmission condition can be set by those skilled in the art based on actual network use requirements, and is not specifically limited herein.

[0085] Specifically, in the bandwidth utilization rate and the heat value of the target switching device calculated above, if there is no standby switching device in the remaining switching devices in the plurality of switching devices that meets the preset device condition, at this time, based on the first loop query strategy, the target switching device that meets the preset transmission condition is continuously queried in the plurality of first switches, that is, the second transmission path of the host computer and at least part of the first switches is determined, and then in the plurality of second transmission paths, the first switch with the shortest second transmission path is selected as the pending switching device, and the bandwidth utilization rate of the pending switching device and the heat value of the initial switching device are continuously obtained, and it is judged whether the heat value of the pending switching device is less than the second preset heat value (for example, 5). If the heat value of the pending switching device is less than the second preset heat value, the heat value of the pending switching device is increased by 1, and it is judged whether the bandwidth utilization rate of the pending switching device is less than the third preset utilization rate. If the bandwidth utilization rate of the pending switching device is less than the third preset utilization rate, the pending switching device is taken as the target switching device, and the determination process of the target switching device is the same as the determination process of the above-mentioned switching device. In order to avoid redundancy, it is not described in detail here.

[0086] Further, if the heat value of the pending switching device is greater than or equal to the second preset heat value, in order to prevent system instability and realize intelligent scheduling, it is necessary to continue to reduce the heat value of the pending switching device according to the preset heat reduction strategy, and based on the second loop query strategy, the next switching device is determined according to the host computer, that is, the next PCIe switching device is continuously queried to the right as a standby path, and the next switching device is taken as the pending switching device, and the steps of obtaining the bandwidth utilization rate of the pending switching device and the heat value of the pending switching device are re-executed until the heat value of the pending switching device is less than the second preset heat value. Then it is further judged whether the bandwidth utilization rate of the pending switching device is less than the third preset utilization rate. If the bandwidth utilization rate of the pending switching device is less than the third preset utilization rate, it means that there is a standby switching device that meets the preset device condition at this time, and the next switching device is taken as the target switching device, which is the standby switching device. Then the data transmission path between the standby switching device and the target acceleration unit is determined, so that the calculated data can be transmitted to the target acceleration unit through the standby switching device based on the data transmission path.

[0087] Therefore, by using the heat value as a core index, the functions of path selection, congestion prevention, and load balancing are organically integrated together, the adaptive scheduling of data is realized, the fluency and efficiency of the ring query are maintained, the discovery time of the standby path is quickly shortened, and the overall communication delay caused by path selection delay is reduced.

[0088] According to one embodiment of the present application, after judging whether the bandwidth utilization rate of the pending switch is less than the third preset utilization rate, further comprising: if the bandwidth utilization rate of the pending switch is greater than or equal to the third preset utilization rate, determining the next switch based on the second cyclic query strategy according to the host; taking the next switch as the pending switch, and re-executing the steps of obtaining the bandwidth utilization rate of the pending switch and the heat value of the pending switch until it is determined that the bandwidth utilization rate of the pending switch is less than the third preset utilization rate, and taking the pending switch as the target switch.

[0089] Specifically, if the bandwidth utilization rate of the pending switch is greater than or equal to the third preset utilization rate, it also indicates that the data transmission has exceeded the upper limit and cannot be forwarded in time, thereby increasing the speed of data processing, so it is necessary to query the next PCIe switch to the right in turn based on the second cyclic query strategy, and then take the next PCIe switch as the pending switch, and re-execute the steps of obtaining the bandwidth utilization rate of the pending switch and the heat value, until it is determined that the bandwidth utilization rate of the pending switch is less than the third preset utilization rate, and the pending switch is taken as the target switch, which is the standby switching device, and then based on the data transmission path between the standby switching device and the target acceleration unit, the to-be-calculated data is transmitted to the target acceleration unit through the standby switching device.

[0090] Therefore, when the bandwidth utilization rate of the target switching device is greater than or equal to the third preset utilization rate, querying the next PCIe switch to the right ensures that the query process will not stagnate or jump randomly, and at the same time the system can sequentially and completely check each possible PCIe switch, avoiding the unfair phenomenon that some switches are ignored for a long time and some switches are excessively queried due to random selection, thereby ensuring that all potential available paths have the opportunity to be discovered and utilized.

[0091] According to one embodiment of the present application, further comprising: obtaining the current GPU utilization rate and the current computing power utilization rate of the acceleration unit; judging whether the current GPU utilization rate is greater than the third preset utilization rate; if the current GPU utilization rate is greater than the third preset utilization rate, judging whether the current computing power utilization rate is less than the fourth preset utilization rate; if the current computing power utilization rate is less than the fourth preset utilization rate, opening the data offloading channel, and storing the data of the acceleration unit in the shared memory pool in the form of memory interleaving.

[0092] Among them, the third preset utilization rate and the fourth preset utilization rate can be set by those skilled in the art according to the actual use requirement, or can be obtained through a limited number of simulations, and are not limited here.

[0093] Specifically, as Figure 4As shown, since the GPU for transaction receiving can dynamically adjust the data offloading strategy according to the current memory utilization rate and the current computing power utilization rate calculated in real time by itself, the current memory utilization rate and the current computing power utilization rate of the GPU can be obtained, if the current memory utilization rate is greater than the third preset utilization rate (for example, the current memory utilization rate is greater than 90%), it is further judged whether the current computing power utilization rate is less than the fourth preset utilization rate, if the current computing power utilization rate is less than the fourth preset utilization rate (for example, the current computing power utilization rate is less than 70%), at this time, it is indicated that the system is in a state of great memory pressure, and the memory overflow error is easily triggered, thereby causing task interruption or performance sharp decline (due to frequent memory swap in and out), but since the current computing power utilization rate is less than 70%, at this time, it means that the GPU is waiting for data or waiting for a synchronization signal, at this time, the computing unit of the GPU is idle, but the I / O channel can have idle bandwidth, so the I / O bandwidth in the waiting period can be used to perform data offloading, and the influence on the core computing task is minimized, therefore, at this time, the data offloading channel is started, and the data of the GPU is stored in the CXL shared memory pool in the form of memory interleaving, the GPU memory pressure is released, and the overall system efficiency is improved.

[0094] According to one embodiment of the application, after judging whether the current memory utilization rate is greater than the third preset utilization rate, further comprising: if the current memory utilization rate is less than or equal to the third preset utilization rate, re-executing the step of obtaining the current memory utilization rate and the current computing power utilization rate of the acceleration unit.

[0095] According to one embodiment of the application, after storing the data of the acceleration unit in the shared memory pool in the form of memory interleaving, further comprising: re-obtaining the new computing power utilization rate of the acceleration unit; if the re-obtained computing power utilization rate is greater than the fifth preset utilization rate, closing the data offloading channel.

[0096] The fifth preset utilization rate can be set by a person skilled in the art according to actual use requirements, or can be obtained through simulation for a limited number of times, which is not limited here.

[0097] Specifically, as shown, Figure 4 if the current memory utilization rate is less than or equal to 90%, at this time, it is indicated that the system memory has space and has not reached the saturation state, therefore, the step of obtaining the current memory utilization rate and the current computing power utilization rate of the GPU can be re-executed, at this time, the new computing power utilization rate of the GPU can be re-obtained, if the new computing power utilization rate is greater than the fifth preset utilization rate, that is, when the current computing power utilization rate is greater than 90%, since the GPU computing power is close to saturation, the offloading operation at this time can make the situation worse and further slow down the core computing task, therefore, the data offloading can be paused first to ensure the computing performance.

[0098] Thus, when the current GPU memory utilization is less than or equal to 90%, by continuously performing the steps of obtaining the current GPU memory utilization and the current computing power utilization, a never-ending, dynamic resource state monitoring closed loop can be achieved, so as to capture transient resource fluctuations, ensure the timeliness and accuracy of scheduling decisions, and adapt to complex and variable AI workloads. When the current computing power utilization is greater than 90%, to avoid further exacerbating the situation by continuing to perform the unloading operation, the data unloading is suspended to ensure the computing performance.

[0099] It should be noted that the Scale-out network uses a CXL shared memory pool to transmit data. The sending host stores data in the CXL shared memory pool in a memory interleaved manner, and uses an RDMA channel to notify the data storage address. The host receives the data storage address notification through the RDMA protocol, and then transmits it to the corresponding GPU. The GPU can directly access the data in the CXL memory for calculation.

[0100] In summary, the embodiments of the present application form an efficient hybrid network congestion control and resource scheduling scheme for large-scale AI training and inference by combining multi-level heat query, dynamic threshold adjustment, multi-path backup switch activation and data unloading mechanism. The link bandwidth utilization and computing resource coordination efficiency in the multi-GPU multi-host system are significantly improved to ensure low latency and high throughput performance of the system under large-scale tasks.

[0101] The main performance is: (1) multi-level heat perception and standby switch selection mechanism, introduce heat table mechanism in PCIe Switch and CXLSwitch, real-time monitor bandwidth utilization, and through the ring order inquiry and heat value limit control standby path enable time, avoid frequent switching caused by path jitter. At the same time, according to the heat difference of the two sides of the main switch, the standby path enable threshold (50%~90%) is adaptively adjusted, which optimizes the link utilization and flow distribution under different congestion distribution; (2) cross-network transaction data offloading mechanism and computing-memory joint scheduling strategy, when the Scale-up network enters the full load state, the GPU transaction data is written into the CXL shared memory pool through the Memory-scale-up network, which relieves the system congestion pressure and forms the cross-domain bandwidth elastic scheduling capability. At the same time, according to the real-time calculation results of GPU computing utilization and memory utilization, it is dynamically decided whether to offload data to CXL memory in the form of memory interleaving, realizing the two-way optimization of memory and computing resources; (3) multi-GPU multi-Host global flow and resource cooperative control framework, combined with CXL shared memory pool and RDMA protocol, the sending Host writes data in the form of memory interleaving to CXL memory and sends address notification through RDMA, and the receiving Host's GPU can directly access the remote data, eliminating the CPU transit delay. The link congestion control, path switching, bandwidth threshold adjustment, transaction offloading, computing / memory scheduling and cross-host direct access are integrated into a unified control logic, realizing the overall optimization of multi-network domain and multi-computing resource.

[0102] Therefore, compared with the existing artificial intelligence server which only relies on single Scale-up or Scale-out network architecture, the application introduces multi-level heat perception, dynamic threshold adjustment, standby path switching and cross-network data offloading mechanism in the mixed interconnection architecture of Scale-up+Scale-out+CXL shared memory pool, realizes low congestion, high throughput and high resource utilization in large-scale AI training and inference tasks, and can achieve the following beneficial effects:

[0103] (1) Realize the efficient adaptation of large-scale AI hybrid architecture, the application embodiment introduces CXL shared memory pool instead of traditional RDMA cross-node communication in multi-GPU multi-host system, realizes high bandwidth and low delay data interaction between Hosts, and at the same time, takes into account the advantages of Scale-up and Scale-out architecture. In large-scale model training, inference and multi-modal processing tasks, the cross-node data access delay is significantly reduced, and the overall resource utilization is improved.

[0104] (2) Realize global congestion awareness and dynamic scheduling across networks, by deploying a unified link heat table and bandwidth utilization monitoring mechanism in the PCIe Switch and CXL Switch two-level networks, the application can realize collaborative congestion control between Scale-up and Scale-out networks. Data flow can be dynamically distributed between different network domains to avoid global performance bottlenecks caused by local link saturation, and ensure sustained high throughput under multi-network collaboration.

[0105] (3) Realize dynamic collaborative use of GPU memory and computing power, based on the monitoring results of GPU real-time memory utilization and computing power utilization, and CXL shared memory pool capacity, the application dynamically adjusts data offloading and backhaul strategy. When the memory is tight and the computing power utilization is low, the data is automatically interleaved and stored in the CXL memory to release the memory pressure; when the computing power demand is high, the offloading is suspended to prioritize computing performance, thereby reducing memory overflow and computing power idling.

[0106] (4) Improve the link bandwidth utilization and overall throughput of multiple switch chips, the application adopts a multi-path flow rerouting strategy, and combines a heat threshold dynamic adjustment mechanism, which can quickly switch to a backup switching path when link congestion occurs, balancing the bandwidth load of multiple switch chips. This mechanism effectively alleviates hot link congestion and improves the overall throughput and data transmission stability of the network.

[0107] According to the resource scheduling method provided by the embodiment of the application, the host obtains the bandwidth utilization of the main switch device in the plurality of switch devices, if the bandwidth utilization of the main switch device is greater than the first preset utilization, the first cyclic query strategy is used to query whether there is a standby switch device that meets the preset device condition in the remaining switch devices except the main switch device in the plurality of switch devices, if not, a target switch that meets the preset transmission condition is determined from the plurality of first switches, and the target switch is used to store the to-be-computed data to the shared memory pool. The technical problems of communication bottleneck caused by cross-network congestion control fragmentation, global resources cannot be collaboratively optimized, and low resource utilization in the AI system are solved, by introducing multi-level heat awareness, dynamic threshold adjustment, standby path switching and cross-network data offloading mechanism in the mixed interconnection architecture of Scale-up+Scale-out+CXL shared memory pool, low congestion, high throughput and high resource utilization are realized in large-scale AI training and inference tasks.

[0108] Figure 5 is a block diagram of the resource scheduling device of the embodiment of the application.

[0109] As Figure 5As shown, the resource scheduling apparatus 10 is applied to a heterogeneous computing system, the heterogeneous computing system comprising a host, a first switch and a shared memory pool, the host and the shared memory pool being connected with the first switch, the host comprising a plurality of switching devices, and the resource scheduling apparatus 10 comprises: an acquisition module 100, a query module 200 and a storage module 300.

[0110] The acquisition module 100 is configured to acquire a bandwidth utilization rate of a master switching device in the plurality of switching devices through the host.

[0111] The query module 200 is configured to, if the bandwidth utilization rate of the master switching device is greater than a first preset utilization rate, query whether there is a standby switching device meeting a preset device condition in the remaining switching devices except the master switching device based on a first cyclic query strategy.

[0112] The storage module 300 is configured to, if there is no standby switching device meeting the preset device condition, determine a target switching device meeting a preset transmission condition from the plurality of first switches, and store the to-be-computed data to the shared memory pool through the target switching device.

[0113] The features of the embodiments of the resource scheduling apparatus can refer to the related descriptions of the embodiments of the resource scheduling method, which will not be repeated here.

[0114] Figure 6 The electronic device provided by the embodiment of the present application is shown in the structural schematic diagram. The electronic device can comprise:

[0115] The memory 601, the processor 602 and the computer program stored in the memory 601 and executable on the processor 602.

[0116] The processor 602 executes the program to realize the resource scheduling method provided in the above embodiments.

[0117] Further, the electronic device further comprises:

[0118] The communication interface 603 is configured to communicate between the memory 601 and the processor 602.

[0119] The memory 601 is configured to store the computer program executable on the processor 602.

[0120] The memory 601 can contain a high-speed RAM memory, and can also include a non-volatile memory, for example, at least one disk memory.

[0121] If the memory 601, the processor 602 and the communication interface 603 are implemented independently, the communication interface 603, the memory 601 and the processor 602 can be connected with each other through a bus and complete communication between each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For convenience of representation, Figure 6 Only one thick line is used to represent the bus in the figure, but it does not mean that there is only one bus or only one type of bus.

[0122] Optionally, in a specific implementation, if the memory 601, the processor 602 and the communication interface 603 are integrated on a chip, the memory 601, the processor 602 and the communication interface 603 can complete communication between each other through an internal interface.

[0123] The processor 602 can be a Central Processing Unit (CPU), or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present application.

[0124] Embodiments of the present application also provide a computer readable storage medium, which stores a computer program, and the computer program is configured to execute the steps in any of the above resource scheduling method embodiments when running.

[0125] Embodiments of the present application also provide another computer program product, which includes a non-volatile computer readable storage medium, and the non-volatile computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the steps in any of the above resource scheduling method embodiments.

[0126] Those skilled in the art will further realize that the mere concepts, teachings, and embodiments described herein are merely meant to provide an enabling description of the claimed invention and are not intended to limit the scope of the claimed invention to these embodiments. Accordingly, the claims are not intended to be limited to the embodiments described herein. Rather, the claims should be read to include any embodiment that falls within the scope of the claims and their equivalents.

[0127] The above describes in detail the method of the present application. The principles and implementation of the present application are described by using specific examples. The above description of the examples is only to help understand the method of the present application and its core idea. It should be noted that those skilled in the art can make some improvements and modifications to the present application without departing from the principles of the present application. These improvements and modifications also fall within the scope of the claims of the present application.

Claims

1. A resource scheduling method, characterized in that, The method is applied to a heterogeneous computing system comprising a host, a first switch and a shared memory pool, the host and the shared memory pool are connected with the first switch, the host comprises a plurality of switching devices, and the method comprises the following steps: acquiring, by the host, a bandwidth utilization rate of a master switching device in the plurality of switching devices; if the bandwidth utilization rate of the master switching device is greater than a first preset utilization rate, querying, based on a first cyclic query strategy, whether there is a standby switching device meeting a preset device condition in the remaining switching devices except the master switching device in the plurality of switching devices; if there is no standby switching device meeting the preset device condition, determining a target switch meeting a preset transmission condition from a plurality of the first switches, and storing, by the target switch, to-be-computed data to the shared memory pool; wherein the switching device is used for data exchange within the host, and the first switch is used for data exchange between the host and the shared memory pool.

2. The method of claim 1, wherein, The querying, based on the first cyclic query strategy, whether there is a standby switching device meeting a preset device condition in the remaining switching devices except the master switching device in the plurality of switching devices comprises: determining, based on the first cyclic query strategy, a target switching device according to the master switching device, and acquiring a bandwidth utilization rate of the target switching device and a heat value of the target switching device; if the heat value of the target switching device is less than a first preset heat value, determining whether the bandwidth utilization rate of the target switching device is less than a second preset utilization rate; if the bandwidth utilization rate of the target switching device is less than the second preset utilization rate, it is determined that there is a standby switching device meeting the preset device condition.

3. The method of claim 2, wherein, After acquiring the bandwidth utilization rate of the target switching device and the heat value of the target switching device, further comprising: if the heat value of the target switching device is greater than or equal to the first preset heat value, reducing the heat value of the target switching device according to a preset heat reduction strategy, and determining a next switching device according to the master switching device based on the first cyclic query strategy; taking the next switching device as the target switching device, and re-executing the steps of acquiring the bandwidth utilization rate of the target switching device and the heat value of the target switching device until it is determined that the heat values of at least part of the switching devices are all greater than or equal to the first preset heat value or the bandwidth utilization rates of at least part of the switching devices are all greater than or equal to the second preset utilization rate, and it is determined that there is no standby switching device meeting the preset device condition.

4. The method of claim 2, wherein, After determining whether the bandwidth utilization rate of the target switching device is less than the second preset utilization rate, further comprising: if the bandwidth utilization rate of the target switching device is greater than or equal to the second preset utilization rate, determining a next switching device according to the master switching device based on the first cyclic query strategy; determining whether the backup switch device and the target acceleration unit have a data transmission path, and transmitting the to-be-computed data to the target acceleration unit through the backup switch device based on the data transmission path, wherein the target acceleration unit is a GPU.

5. The method of claim 2, wherein, Before the host obtains the bandwidth utilization of the main switch device in the plurality of switch devices, the method further includes: determining whether there is a data transmission requirement; if there is the data transmission requirement, determining a target acceleration unit based on the data transmission requirement, and calculating a first transmission path between the target acceleration unit and at least part of the switch devices; 6. The method of claim 1, wherein, based on the first transmission path between the target acceleration unit and at least part of the switch devices, determining a switch device having a shortest first transmission path between the target acceleration unit as the main switch device, wherein the target acceleration unit is a GPU. After the to-be-computed data is transmitted to the target acceleration unit through the backup switch device based on the data transmission path, the method further includes: obtaining a current transmission progress of the backup switch device in transmitting the to-be-computed data to the target acceleration unit; if the current transmission progress is complete transmission, reducing the heat value of the target switch device according to a preset heat reduction strategy.

7. The method of claim 5, wherein, The method of determining a target switch device satisfying a preset transmission condition from a plurality of first switches includes: determining a second transmission path between the host and at least part of the first switches; based on the second transmission path between the host and at least part of the first switches, determining a first switch having a shortest second transmission path between the host as a standby switch device; 8. The method of claim 1, wherein, obtaining a bandwidth utilization of the standby switch device and a heat value of an initial switch device, and determining whether the heat value of the standby switch device is less than a second preset heat value; if the heat value of the standby switch device is less than the second preset heat value, determining whether the bandwidth utilization of the standby switch device is less than a third preset utilization rate; if the bandwidth utilization of the standby switch device is less than the third preset utilization rate, determining the standby switch device as the target switch device. After determining whether the heat value of the standby switch device is less than the second preset heat value, the method further includes: if the heat value of the standby switch device is greater than or equal to the second preset heat value, reducing the heat value of the standby switch device according to a preset heat reduction strategy, and determining a next switch device based on a second loop query strategy according to the host; ​ 9. The method of claim 8, wherein, ​ ​ The next switch is taken as the pending switch, and the steps of obtaining the bandwidth utilization of the pending switch and the heat value of the pending switch are re-executed until it is determined that the heat value of the pending switch is less than the second preset heat value.

10. The method of claim 8, wherein, After judging whether the bandwidth utilization of the pending switch is less than a third preset utilization rate, the method further comprises: If the bandwidth utilization of the pending switch is greater than or equal to the third preset utilization rate, a next switch is determined based on a second cyclic query strategy according to the host; The next switch is taken as the pending switch, and the steps of obtaining the bandwidth utilization of the pending switch and the heat value of the pending switch are re-executed until it is determined that the bandwidth utilization of the pending switch is less than the third preset utilization rate, and the pending switch is taken as the target switch.

11. The method of claim 2, wherein, Before obtaining the bandwidth utilization of the target switching device and the heat value of the target switching device, the method further comprises: Based on the first cyclic query strategy, a first side of the master switching device and a second side of the master switching device are determined, and a heat value of the first side and a heat value of the second side are obtained; The first preset heat value is updated according to the heat value of the first side and the heat value of the second side.

12. The method of claim 11, wherein, The determination of the first side and the second side of the master switching device based on the first cyclic query strategy comprises: A query direction is determined according to the first cyclic query strategy; Based on the master switching device, a side opposite to the query direction is taken as the first side, and a side same as the query direction is taken as the second side.

13. The method according to claim 11 or 12, characterized in that, The updating of the first preset heat value according to the heat value of the first side and the heat value of the second side comprises: It is judged whether the heat value of the first side is greater than the heat value of the second side; If the heat value of the first side is greater than the heat value of the second side, the first preset bandwidth utilization is updated to a third preset bandwidth utilization, otherwise, the first preset bandwidth utilization is updated to a fourth preset bandwidth utilization; wherein, The third preset bandwidth utilization is less than the first preset bandwidth utilization, and the fourth preset bandwidth utilization is greater than the first preset bandwidth utilization.

14. The method of claim 1, wherein, After the to-be-calculated data is stored to the shared memory pool through the target switch, the method further comprises: It is judged whether the bandwidth utilization and the heat value of the master switching device both satisfy corresponding conditions; If the bandwidth utilization and the heat value of the master switching device both satisfy corresponding conditions, the to-be-transmitted transaction data is received through the master switching device, and the to-be-transmitted transaction data is transmitted to a target acceleration unit for transaction reception, wherein the target acceleration unit is a GPU.

15. The method of claim 1, wherein, The method further comprises: A current GPU memory utilization rate and a current computing power utilization rate of the target acceleration unit are obtained; It is judged whether the current GPU memory utilization rate is greater than a third preset utilization rate; If the current GPU memory utilization rate is greater than the third preset utilization rate, it is judged whether the current computing power utilization rate is less than a fourth preset utilization rate; If the current computing power utilization rate is less than the fourth preset utilization rate, a data unloading channel is opened, and data of the target acceleration unit is stored in the shared memory pool in a memory interleaving manner, wherein the target acceleration unit is a GPU.

16. The method of claim 15, wherein, After storing the data of the target acceleration unit in the shared memory pool in the memory interleaving manner, the method further comprises: reacquiring the computing power utilization rate of the target acceleration unit; If the reacquired computing power utilization rate is greater than a fifth preset utilization rate, the data unloading channel is closed.

17. The method of claim 15, wherein, After determining whether the current GPU memory utilization rate is greater than the third preset utilization rate, the method further comprises: If the current GPU memory utilization rate is less than or equal to the third preset utilization rate, the step of acquiring the current GPU memory utilization rate and the current computing power utilization rate of the target acceleration unit is re-executed.

18. An electronic device, comprising: comprise: a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the steps of the resource scheduling method according to any one of claims 1 to 17.

19. A computer-readable storage medium, characterized in that, The computer program is stored in the computer readable storage medium, and when the computer program is executed by the processor, the steps of the resource scheduling method according to any one of claims 1 to 17 are implemented.

20. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the resource scheduling method according to any one of claims 1 to 17.

Images