Load balancing system, method, device, medium and program product

Abstract

The invention discloses a load balancing system, method and device, a medium and a program product in the technical field of computers. Task distribution is carried out on the accelerator on the basis of the actual resource remaining amount of the accelerator, the actual operation condition of the accelerator is combined and referred, it can be guaranteed that the task distribution result conforms to the expectation as much as possible, and it is possible to balance loads of different accelerators; moreover, the network card of each heterogeneous resource node is synchronous with the data of the cache region of each heterogeneous accelerator, so that the data transmission and task distribution efficiency between the heterogeneous resource nodes and the heterogeneous accelerators can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a load balancing system, method, device, medium, and program product. Background Art

[0002] Currently, the task dispatching node infers the operating pressure of the accelerator based on its communication with the accelerator, and then dispatches tasks to the accelerator based on the inference result. However, the inferred and predicted operating pressure of the accelerator may not match the actual operating pressure of the accelerator, resulting in task dispatch not meeting expectations and difficulty in balancing the load sizes of different accelerators.

[0003] Therefore, how to balance the load sizes of different accelerators is a problem that those skilled in the art need to solve. Summary of the Invention

[0004] In view of this, the purpose of the present application is to provide a load balancing system, method, device, medium and program product to balance the load sizes of different accelerators.

[0005] In a first aspect, the present application provides a load balancing system, comprising: multiple heterogeneous resource nodes and multiple heterogeneous accelerators; the network card of each heterogeneous resource node is connected to the cache area of each heterogeneous accelerator using a cache consistency protocol; each heterogeneous accelerator is used to: detect whether the remaining amount of each type of resource in itself is sufficient, obtain a detection result, and update the detection result to its own cache area; each heterogeneous resource node is used to: use the cache consistency protocol to collect the detection results recorded in the cache area of each heterogeneous accelerator to its own network card, and use the network card to refer to the collected information to assign tasks to each heterogeneous accelerator to achieve load balancing of each heterogeneous accelerator.

[0006] In a second aspect, the present application provides a load balancing method, which is applied to a load balancing system, the load balancing system including: multiple heterogeneous resource nodes and multiple heterogeneous accelerators; the network card of each heterogeneous resource node is connected to the cache area of each heterogeneous accelerator using a cache consistency protocol; the load balancing method includes: each heterogeneous accelerator detects whether the remaining amount of each type of resource in itself is sufficient, obtains the detection result, and updates the detection result to its own cache area; each heterogeneous resource node uses the cache consistency protocol to collect the detection results recorded in the cache area of each heterogeneous accelerator to its own network card, and uses the network card to refer to the collected information to assign tasks to each heterogeneous accelerator to achieve load balancing of each heterogeneous accelerator.

[0007] In a third aspect, the present application provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the aforementioned disclosed load balancing method.

[0008] In a fourth aspect, the present application provides a non-volatile storage medium for storing a computer program, wherein the computer program implements the aforementioned disclosed load balancing method when executed by a processor.

[0009] In a fifth aspect, the present application provides a computer program product, comprising a computer program / instruction, which implements the steps of the aforementioned disclosed load balancing method when executed by a processor.

[0010] It can be seen from the above scheme that the present application provides a load balancing system, including: multiple heterogeneous resource nodes and multiple heterogeneous accelerators; the network card of each heterogeneous resource node is connected to the cache area of each heterogeneous accelerator using a cache consistency protocol; each heterogeneous accelerator is used to: detect whether the remaining amount of each type of resource in itself is sufficient, obtain the detection result, and update the detection result to the cache area in itself; each heterogeneous resource node is used to: use the cache consistency protocol to collect the detection results recorded in the cache area of each heterogeneous accelerator to the network card in itself, and use the network card to refer to the collected information to assign tasks to each heterogeneous accelerator to achieve load balancing of each heterogeneous accelerator.

[0011] It can be seen that the beneficial effects of this application are: the network cards of each heterogeneous resource node connect to the cache areas of each heterogeneous accelerator using a cache consistency protocol, thereby achieving data synchronization between the network cards of each heterogeneous resource node and the cache areas of each heterogeneous accelerator. On this basis, each heterogeneous resource node can use the network card to quickly obtain the detection results of whether the remaining amount of various resources recorded in the cache areas of each heterogeneous accelerator is sufficient, and accordingly assign tasks to each heterogeneous accelerator to achieve load balancing of each heterogeneous accelerator. This solution dispatches tasks to accelerators based on the actual remaining amount of accelerator resources, combined with the actual operation status of the accelerator, and can ensure that the task dispatch results meet expectations as much as possible, providing the possibility of balancing the load sizes of different accelerators; and, the data synchronization between the network cards of each heterogeneous resource node and the cache areas of each heterogeneous accelerator can improve the efficiency of data transmission and task dispatch between heterogeneous resource nodes and heterogeneous accelerators. Because the network card is closer to the network and has a faster response speed, the CPU of the heterogeneous resource node does not need to participate in the load balancing process.

[0012] Correspondingly, the load balancing method, device, medium and program product provided by this application also have the above technical effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0014] Figure 1 A schematic diagram of a load balancing system disclosed in this application; Figure 2 This is a schematic diagram of the second load balancing system disclosed in this application; Figure 3 A schematic diagram of the connection between a heterogeneous resource node and a heterogeneous accelerator pooling node disclosed in this application; Figure 4 This is a schematic diagram of the third load balancing system disclosed in this application; Figure 5 A schematic diagram of a load balancing process disclosed in this application; Figure 6 A resource usage status update flow chart disclosed in this application; Figure 7 A server structure diagram provided for this application; Figure 8 This is a terminal structure diagram provided for this application. DETAILED DESCRIPTION

[0015] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0016] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0017] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0018] Currently, task dispatching nodes infer the operating pressure of accelerators based on their communication with the accelerators, and then dispatch tasks to the accelerators based on the inference results. However, the inferred operating pressure of the accelerators may not match the actual operating pressure of the accelerators, resulting in task dispatching that does not meet expectations and difficulty in balancing the loads of different accelerators. To this end, this application provides a load balancing solution that can balance the loads of different accelerators.

[0019] See also Figure 1 As shown, the present invention discloses a load balancing system comprising: multiple heterogeneous resource nodes and multiple heterogeneous accelerators; the network interface card of each heterogeneous resource node is connected to the cache area of each heterogeneous accelerator using a cache coherence protocol. Each heterogeneous resource node can be a server or other device, and each heterogeneous accelerator can be an FPGA, GPU, etc.

[0020] It should be noted that each heterogeneous resource node and each heterogeneous accelerator are connected via Ethernet. Furthermore, each heterogeneous resource node and each heterogeneous accelerator communicate with each other via remote direct data access technology. This shows that when any heterogeneous resource node dispatches a task to any heterogeneous accelerator, it can directly read and write task-related data using remote direct data access technology, thereby improving task processing efficiency. Tasks can be storage-related, compute-related, or other tasks, such as model training tasks.

[0021] In this embodiment, each heterogeneous accelerator is used to detect whether the remaining amount of each type of resource in itself is sufficient, obtain a detection result, and update the detection result to a cache area in itself.

[0022] In this embodiment, each heterogeneous resource node is configured to collect the detection results recorded in the cache of each heterogeneous accelerator into its own network interface card using a cache coherence protocol, and use the network interface card to assign tasks to each heterogeneous accelerator based on the collected information, thereby achieving load balancing among the heterogeneous accelerators. For example, each heterogeneous resource node collects the detection results recorded in the cache of each heterogeneous accelerator into the memory area of its own network interface card using a cache coherence protocol for storage.

[0023] In one embodiment, each heterogeneous accelerator is configured to periodically collect usage information for various types of resources within itself and, based on this usage information, detect whether the remaining amount of each type of resource is sufficient. For example, each heterogeneous accelerator uses the resource usage rate of its own computing resources, memory resources, and cache resources as the usage information for each type of resource. In other words, each type of resource may include computing resources, memory resources, and cache resources.

[0024] In one embodiment, each heterogeneous accelerator is used to: use a time domain low-pass filter to smooth the usage information, and detect whether the remaining amount of each type of resource in itself is sufficient based on the smoothing result. d =a×X p +(1-a)×X d-1 , smoothing the usage information of any type of resources; where X d is the usage information of any type of resource after smoothing, a is the preset filter coefficient, X pis the usage information of the current class of resources before smoothing, X d-1 This is the usage information of the current type of resource after the previous smoothing process. For example: X d is the resource utilization rate after smoothing of the computing resources, a is the preset filter coefficient, X p is the resource usage before smoothing (i.e., the actual resource usage collected), X d-1 This is the resource usage after the previous smoothing process.

[0025] To reduce the amount of data synchronization between the network card memory of each heterogeneous resource node and the cache of each heterogeneous accelerator, this embodiment allows the cache of each heterogeneous accelerator to only record notification messages indicating whether the remaining amount of each resource is sufficient. The network card of each heterogeneous resource node can then receive this notification message and dispatch tasks accordingly. In one embodiment, each heterogeneous accelerator is configured to: detect whether the resource utilization rate of its own computing resources, memory resources, and cache resources exceeds the corresponding utilization rate threshold; if so, it determines that the remaining amount of this type of resource is insufficient and sends a corresponding resource insufficient message as the detection result; if not, it determines that the remaining amount of this type of resource is sufficient and sends a corresponding resource sufficient message as the detection result.

[0026] In this embodiment, each heterogeneous resource node uses its own network card to process the collected information with a load balancing algorithm and assigns tasks to each heterogeneous accelerator based on the corresponding processing results. The load balancing algorithm can refer to relevant technologies in the field and will not be described here in detail.

[0027] As can be seen in this embodiment, the network cards of each heterogeneous resource node connect to the caches of each heterogeneous accelerator using a cache coherence protocol, thereby achieving data synchronization between the network cards of each heterogeneous resource node and the caches of each heterogeneous accelerator. Furthermore, each heterogeneous resource node can use the network card to quickly obtain the detection results of whether the remaining amount of various resources recorded in the caches of each heterogeneous accelerator is sufficient, and accordingly assign tasks to each heterogeneous accelerator to achieve load balancing among the heterogeneous accelerators. This solution dispatches tasks to accelerators based on the actual remaining amount of accelerator resources, combined with reference to the actual operating conditions of the accelerators, to ensure that the task dispatch results meet expectations as much as possible, providing a possibility for balancing the load of different accelerators. Furthermore, the data synchronization between the network cards of each heterogeneous resource node and the caches of each heterogeneous accelerator can improve the efficiency of data transmission and task dispatch between the heterogeneous resource nodes and the heterogeneous accelerators. Because the network card is closer to the network, the response speed is faster, and the CPU of the heterogeneous resource node does not need to participate in the load balancing process.

[0028] See Figure 2Multiple heterogeneous accelerators are pooled into a heterogeneous acceleration pool, making these heterogeneous accelerators serve as heterogeneous accelerator pooling nodes in the heterogeneous acceleration pool. Each heterogeneous accelerator pooling node is connected to a heterogeneous resource node. The heterogeneous resource node places a load balancing module on the network interface card (NIC). The NIC memory synchronizes data with the cache of each heterogeneous accelerator pooling node using the CXL protocol. This allows the NIC memory to record the real-time resource usage status of each heterogeneous accelerator pooling node. The load balancing module then determines which heterogeneous accelerator pooling node to use for task processing based on this real-time resource usage status.

[0029] The real-time resource usage status of a heterogeneous accelerator pooling node is determined based on the node's own resource usage. The data synchronization between the network card of the heterogeneous resource node and the cache area of each heterogeneous accelerator pooling node is equivalent to the heterogeneous accelerator pooling node using a block of memory on the heterogeneous resource node and storing its real-time resource usage status in this memory.

[0030] See Figure 3 Real-time resource usage status is stored in the network card memory of the heterogeneous resource node. This is because the NIC is closer to the network, resulting in faster response times and eliminating the need for the CPU of the heterogeneous resource node to participate in the load balancing process. If the CPU of the heterogeneous resource node performs load balancing, load balancing performance will drop from milliseconds to microseconds, resulting in significant performance loss. The network card also supports Ethernet and Remote Direct Memory Access (RDMA) connections to heterogeneous accelerator pool nodes.

[0031] See Figure 4 If there are two heterogeneous resource nodes, each one is connected to all heterogeneous accelerator pooling nodes. In other words, each heterogeneous accelerator pooling node updates its real-time resource usage status to the network interface cards (NICs) of different heterogeneous resource nodes. This allows each heterogeneous resource node to be aware of the real-time resource usage status of all heterogeneous accelerator pooling nodes. The NICs of the heterogeneous resource nodes are CXL type 2 devices.

[0032] CXL's greatest performance advantage lies in its support for caching. Effective caching requires infrequent data changes. Frequent data changes require frequent data synchronization between heterogeneous resource nodes and heterogeneous accelerator pooling nodes. To reduce the amount of real-time resource usage data, real-time resource usage status can only record information about resource availability, rather than resource usage information.

[0033] In an example, the real-time resource usage status may be recorded with reference to Table 1.

[0034] Table 1

[0035] The heterogeneous accelerator pooling node determines resource sufficiency and then synchronizes it to the network interface cards (NICs) of the nodes using heterogeneous resources via CXL. Furthermore, the heterogeneous accelerator pooling node determines resource sufficiency based on resource utilization. This sufficiency is then smoothed using a time-domain low-pass filter before determining resource sufficiency, avoiding frequent status reporting due to transient changes.

[0036] The time domain low-pass filter is used for smoothing, which can be expressed as follows: , this formula is consistent with formula X d =a×X p +(1-a)×X d-1 The two are identical, differing only in the letters used to represent the parameters. 0<α<1 is the filter coefficient, which adjusts the smoothness of the filter. This filter combines the current resource utilization with the filtering results from the previous moment, using the value of α to control the emphasis on new data and the degree to which historical trends are retained. This smoothes resource utilization, filters out high-frequency noise, and retains low-frequency trend information. is the smoothed resource usage at the current moment, For the measured resource usage, It is the smoothed resource usage at the previous moment.

[0037] See Figure 5 A load balancing process includes: heterogeneous applications in heterogeneous resource nodes send heterogeneous accelerator requests (i.e., tasks that require processing by heterogeneous accelerator pooling nodes) to the heterogeneous resource node network card. The network card searches for a suitable heterogeneous accelerator pooling node based on the real-time resource usage status, and then dispatches the heterogeneous accelerator request to the appropriate heterogeneous accelerator pooling node, so that the heterogeneous accelerator pooling node can process it and return the corresponding processing results. In this example, the heterogeneous application is unaware of the relevant load balancing decisions, which is very different from traditional methods. Heterogeneous applications do not need to participate in the decision-making at all, which can improve call performance and is very important for heterogeneous computing with high real-time requirements. Traditional load balancing involves heterogeneous applications, which can meet the real-time requirements of web applications, but cannot meet the needs of high-real-time heterogeneous computing.

[0038] See Figure 6Heterogeneous accelerator pooling nodes periodically determine whether their resource usage status needs to be updated based on their own resource usage. If computing core resources were previously sufficient but are now insufficient, an update is required. If computing core resources were previously sufficient and are still sufficient, but the number of idle computing cores has changed, no update is required. If a status update is required, the resource usage status is updated to the network interface cards of the nodes using heterogeneous resources via CXL.

[0039] As can be seen, this embodiment uses the load balancing module on the heterogeneous resource nodes to obtain the real-time resource usage of the heterogeneous accelerator pooling nodes and then perform corresponding load balancing operations, thereby improving the accuracy of load balancing and subsequently improving the overall performance of the system. Implementing load balancing through hardware meets the requirements of high-real-time heterogeneous computing applications and improves the overall performance of the system.

[0040] The following introduces a load balancing method provided in an embodiment of the present application. The load balancing method described below can be referenced with other embodiments described in this document.

[0041] An embodiment of the present application discloses a load balancing method, which is applied to a load balancing system. The load balancing system includes: multiple heterogeneous resource nodes and multiple heterogeneous accelerators; the network interface card of each heterogeneous resource node is connected to the cache area of each heterogeneous accelerator using a cache consistency protocol; the load balancing method includes: each heterogeneous accelerator detects whether the remaining amount of each type of resource in itself is sufficient, obtains a detection result, and updates the detection result to its own cache area; each heterogeneous resource node uses the cache consistency protocol to collect the detection results recorded in the cache area of each heterogeneous accelerator to its own network interface card, and uses the network interface card to refer to the collected information to assign tasks to each heterogeneous accelerator to achieve load balancing for each heterogeneous accelerator.

[0042] In one embodiment, each heterogeneous accelerator is configured to periodically collect usage information of various resources within itself, and detect whether the remaining amount of various resources within itself is sufficient based on the usage information.

[0043] In one embodiment, each heterogeneous accelerator is configured to use the resource usage rate of its own computing resources, memory resources, and cache resources as the usage information of each type of resource.

[0044] In one embodiment, each heterogeneous accelerator is configured to: smooth the usage information using a time-domain low-pass filter, and detect whether the remaining amount of each type of resource in the accelerator is sufficient based on the smoothing result.

[0045] In one embodiment, each heterogeneous accelerator is used to: d =a×X p +(1-a)×X d-1 , smoothing the usage information of any type of resources; where Xd is the usage information of any type of resource after smoothing, a is the preset filter coefficient, X p is the usage information of the current class of resources before smoothing, X d-1 This is the usage information of the current type of resource after the previous smoothing process. For example: X d is the resource utilization rate after smoothing of the computing resources, a is the preset filter coefficient, X p is the resource usage before smoothing (i.e., the actual resource usage collected), X d-1 This is the resource usage after the previous smoothing process.

[0046] In one embodiment, each heterogeneous accelerator is used to: respectively detect whether the resource utilization rate of its own computing resources, memory resources and cache resources exceeds the corresponding utilization rate threshold; if exceeded, it is confirmed that the remaining amount of such resources is insufficient, and the corresponding resource insufficient message is used as the detection result; if not exceeded, it is confirmed that the remaining amount of such resources is sufficient, and the corresponding resource sufficient message is used as the detection result.

[0047] In one embodiment, each heterogeneous resource node is configured to collect the detection results recorded in the cache area of each heterogeneous accelerator into the memory area of the network card in the node by using a cache coherence protocol for storage.

[0048] In one embodiment, each heterogeneous resource node and each heterogeneous accelerator are further connected via Ethernet.

[0049] In one embodiment, each heterogeneous resource node and each heterogeneous accelerator further communicates via remote direct data access technology.

[0050] In one embodiment, each heterogeneous resource node is configured to: utilize its own network card to process the collected information using a load balancing algorithm, and allocate tasks to each heterogeneous accelerator according to corresponding processing results.

[0051] In one embodiment, the network card memory of each heterogeneous resource node synchronizes data with the cache area of each heterogeneous accelerator using a cache coherence protocol.

[0052] Among them, for more specific working processes of each module and unit in this embodiment, reference can be made to the corresponding contents disclosed in the aforementioned embodiments, which will not be repeated here.

[0053] As can be seen, this embodiment provides a load balancing method that dispatches tasks to accelerators based on their actual remaining resources. This, combined with reference to the accelerators' actual operating conditions, ensures that task dispatch results meet expectations, making it possible to balance the loads of different accelerators. Furthermore, data synchronization between the network cards of each heterogeneous resource node and the caches of each heterogeneous accelerator improves the efficiency of data transmission and task dispatch between the heterogeneous resource nodes and the heterogeneous accelerators. Because the network cards are closer to the network and respond more quickly, the CPUs of the heterogeneous resource nodes do not need to participate in the load balancing process.

[0054] An electronic device provided in an embodiment of the present application is introduced below. The electronic device described below can be referenced with other embodiments described herein.

[0055] An embodiment of the present application discloses an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the method disclosed in any of the above embodiments.

[0056] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: detect whether the remaining amount of various resources in itself is sufficient, obtain the detection result, and update the detection result to the cache area in itself.

[0057] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: using a cache consistency protocol to collect the detection results recorded in the cache area of each heterogeneous accelerator to the network card in the processor, and using the network card to refer to the collected information to assign tasks to each heterogeneous accelerator to achieve load balancing among the heterogeneous accelerators.

[0058] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: regularly collect usage information of various resources in itself, and detect whether the remaining amount of various resources in itself is sufficient based on the usage information.

[0059] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: using the resource usage rates of its own computing resources, memory resources, and cache resources as usage information of various types of resources.

[0060] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: use a time domain low-pass filter to smooth the usage information, and detect whether the remaining amount of each type of resource in itself is sufficient based on the smoothing result.

[0061] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: respectively detect the resource utilization rate of the computing resources, memory resources and cache resources in itself to see whether it exceeds the corresponding utilization rate threshold; if it exceeds, confirm that the remaining amount of this type of resources is insufficient, and use the corresponding insufficient resource message as the detection result; if it does not exceed, confirm that the remaining amount of this type of resources is sufficient, and use the corresponding sufficient resource message as the detection result.

[0062] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: using a cache consistency protocol to collect the detection results recorded in the cache area of each heterogeneous accelerator into the memory area of the network card in the processor for storage.

[0063] In this embodiment, when the processor executes the computer program stored in the memory, it can specifically implement the following steps: using the network card in the processor to process the collected information with a load balancing algorithm, and assigning tasks to each heterogeneous accelerator according to the corresponding processing results.

[0064] Furthermore, the embodiment of the present application also provides an electronic device. The electronic device can be Figure 7 The server shown can also be Figure 8 The terminal shown. Figure 7 and Figure 8 Each of the diagrams is a structural diagram of an electronic device according to an exemplary embodiment, and the contents in the diagrams cannot be considered as any limitation on the scope of use of the present application.

[0065] Figure 7 This is a schematic diagram of the structure of a server provided in an embodiment of the present application. The server may specifically include: at least one processor, at least one memory, a power supply, a communication interface, an input / output interface, and a communication bus. The memory is used to store a computer program, which is loaded and executed by the processor to implement the relevant steps of load balancing disclosed in any of the aforementioned embodiments.

[0066] In this embodiment, the power supply is used to provide operating voltage for each hardware device on the server; the communication interface can create a data transmission channel between the server and external devices. The communication protocol it follows is any communication protocol that can be applied to the technical solution of this application and is not specifically limited here; the input and output interface is used to obtain external input data or output data to the outside world. The specific interface type can be selected according to specific application needs and is not specifically limited here.

[0067] In addition, the memory as a carrier for resource storage can be a read-only memory, random access memory, disk or CD, etc. The resources stored thereon include operating system, computer programs and data, etc. The storage method can be temporary storage or permanent storage.

[0068] The operating system is used to manage and control the hardware devices and computer programs on the server to enable the processor to calculate and process data in the memory. It can be Windows Server, NetWare, Unix, Linux, etc. In addition to including computer programs capable of implementing the load balancing method disclosed in any of the aforementioned embodiments, computer programs can also include computer programs capable of performing other specific tasks. Data can include data such as application update information and other data such as application developer information.

[0069] Figure 8 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present application. The terminal may specifically include but is not limited to a smartphone, tablet computer, laptop computer or desktop computer.

[0070] Generally, the terminal in this embodiment includes: a processor and a memory.

[0071] The processor may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor may be implemented in at least one of the following hardware forms: a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), or a PLA (Programmable Logic Array). The processor may also include a main processor and a coprocessor. The main processor is used to process data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing content required to be displayed on the display. In some embodiments, the processor may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0072] The memory may include one or more computer non-volatile storage media, which may be non-transitory. The memory may also include high-speed random access memory, and non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory is used to store at least the following computer program, wherein, after the computer program is loaded and executed by the processor, it can implement the relevant steps in the load balancing method performed by the terminal side disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory may also include an operating system and data, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system may include Windows, Unix, Linux, etc. The data may include but is not limited to update information of the application.

[0073] In some embodiments, the terminal may further include a display screen, an input and output interface, a communication interface, a sensor, a power supply, and a communication bus.

[0074] Those skilled in the art will understand that Figure 8 The structure shown in the figure does not constitute a limitation to the terminal, and may include more or fewer components than shown in the figure.

[0075] A non-volatile storage medium provided in an embodiment of the present application is introduced below. The non-volatile storage medium described below can be referenced with other embodiments described herein.

[0076] A non-volatile storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the load balancing method disclosed in the aforementioned embodiment. The non-volatile storage medium is a computer-readable non-volatile storage medium that, as a carrier for resource storage, may be a read-only memory, random access memory, magnetic disk, or optical disk. The resources stored thereon include an operating system, computer programs, and data, and the storage method may be either temporary or permanent.

[0077] In this embodiment, when the processor executes the computer program stored in the non-volatile storage medium, it can specifically implement the following steps: detect whether the remaining amount of various resources in itself is sufficient, obtain the detection result, and update the detection result to the cache area in itself.

[0078] In this embodiment, when the processor executes a computer program stored in a non-volatile storage medium, it can specifically implement the following steps: using a cache consistency protocol to collect the detection results recorded in the cache area of each heterogeneous accelerator to its own network card, and using the network card to refer to the collected information to assign tasks to each heterogeneous accelerator to achieve load balancing among the heterogeneous accelerators.

[0079] In this embodiment, when the processor executes the computer program stored in the non-volatile storage medium, it can specifically implement the following steps: regularly collect usage information of various resources in itself, and detect whether the remaining amount of various resources in itself is sufficient based on the usage information.

[0080] In this embodiment, when the processor executes the computer program stored in the non-volatile storage medium, it can specifically implement the following steps: using the resource utilization rate of its own computing resources, memory resources and cache resources as the usage information of each type of resources.

[0081] In this embodiment, when the processor executes the computer program stored in the non-volatile storage medium, it can specifically implement the following steps: use a time domain low-pass filter to smooth the usage information, and detect whether the remaining amount of each type of resource in itself is sufficient based on the smoothing result.

[0082] In this embodiment, when the processor executes a computer program stored in a non-volatile storage medium, it can specifically implement the following steps: respectively detect whether the resource utilization rate of its own computing resources, memory resources and cache resources exceeds the corresponding utilization rate threshold; if exceeded, confirm that the remaining amount of such resources is insufficient, and use the corresponding insufficient resource message as the detection result; if not exceeded, confirm that the remaining amount of such resources is sufficient, and use the corresponding sufficient resource message as the detection result.

[0083] In this embodiment, when the processor executes the computer program stored in the non-volatile storage medium, the following steps may be specifically implemented: using a cache consistency protocol to collect the detection results recorded in the cache areas of each heterogeneous accelerator into the memory area of the network card in the processor for storage.

[0084] In this embodiment, when the processor executes the computer program stored in the non-volatile storage medium, it can specifically implement the following steps: using its own network card to process the collected information with a load balancing algorithm, and assigning tasks to each heterogeneous accelerator according to the corresponding processing results.

[0085] A computer program product provided in an embodiment of the present application is introduced below. The computer program product described below can be referenced with other embodiments described herein.

[0086] A computer program product includes a computer program / instruction, which implements the steps of the aforementioned load balancing method when executed by a processor.

[0087] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the steps in any of the above embodiments are implemented.

[0088] In this embodiment, when the processor executes the computer program stored in the non-volatile storage medium, it can specifically implement the following steps: detect whether the remaining amount of various resources in itself is sufficient, obtain the detection result, and update the detection result to the cache area in itself.

[0089] In this embodiment, when the processor executes a computer program stored in a non-volatile storage medium, it can specifically implement the following steps: using a cache consistency protocol to collect the detection results recorded in the cache area of each heterogeneous accelerator to its own network card, and using the network card to refer to the collected information to assign tasks to each heterogeneous accelerator to achieve load balancing among the heterogeneous accelerators.

[0090] In this embodiment, when the processor executes the computer program stored in the non-volatile storage medium, it can specifically implement the following steps: regularly collect usage information of various resources in itself, and detect whether the remaining amount of various resources in itself is sufficient based on the usage information.

[0091] In this embodiment, when the processor executes the computer program stored in the non-volatile storage medium, it can specifically implement the following steps: using the resource utilization rate of its own computing resources, memory resources and cache resources as the usage information of each type of resources.

[0092] In this embodiment, when the processor executes the computer program stored in the non-volatile storage medium, it can specifically implement the following steps: use a time domain low-pass filter to smooth the usage information, and detect whether the remaining amount of each type of resource in itself is sufficient based on the smoothing result.

[0093] In this embodiment, when the processor executes a computer program stored in a non-volatile storage medium, it can specifically implement the following steps: respectively detect whether the resource utilization rate of its own computing resources, memory resources and cache resources exceeds the corresponding utilization rate threshold; if exceeded, confirm that the remaining amount of such resources is insufficient, and use the corresponding insufficient resource message as the detection result; if not exceeded, confirm that the remaining amount of such resources is sufficient, and use the corresponding sufficient resource message as the detection result.

[0094] In this embodiment, when the processor executes the computer program stored in the non-volatile storage medium, the following steps may be specifically implemented: using a cache consistency protocol to collect the detection results recorded in the cache areas of each heterogeneous accelerator into the memory area of the network card in the processor for storage.

[0095] In this embodiment, when the processor executes the computer program stored in the non-volatile storage medium, it can specifically implement the following steps: using its own network card to process the collected information with a load balancing algorithm, and assigning tasks to each heterogeneous accelerator according to the corresponding processing results.

[0096] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0097] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of non-volatile storage medium known in the art.

[0098] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A load balancing system, characterized in that: include: Multiple heterogeneous resource nodes and multiple heterogeneous accelerators; The network cards of each heterogeneous resource node connect to the cache areas of each heterogeneous accelerator using a cache consistency protocol; Each heterogeneous accelerator is used to detect whether the remaining amount of various resources in itself is sufficient, obtain the detection result, and update the detection result to the cache area in itself; Each heterogeneous resource node is used to collect the detection results recorded in the cache area of each heterogeneous accelerator to its own network card using the cache consistency protocol, and use the network card to refer to the collected information to allocate tasks to each heterogeneous accelerator to achieve load balancing of each heterogeneous accelerator.

2. The system according to claim 1, wherein: Each heterogeneous accelerator is used to regularly collect usage information of various resources in itself, and detect whether the remaining amount of various resources in itself is sufficient based on the usage information.

3. The system according to claim 2, characterized in that Each heterogeneous accelerator is used to use the resource utilization rate of its own computing resources, memory resources, and cache resources as the utilization information of each type of resource.

4. The system according to claim 2, wherein: Each heterogeneous accelerator is configured to: perform smoothing processing on the usage information by using a time-domain low-pass filter, and detect whether the remaining amount of each type of resource in the accelerator is sufficient based on the smoothing processing result.

5. The system according to claim 4, characterized in that Each heterogeneous accelerator is used to: use the formula X d =a×X p +(1-a)×X d-1 , smoothing the usage information of any type of resources; where X d is the usage information of any type of resource after smoothing, a is the preset filter coefficient, X p is the usage information of the current class of resources before smoothing, X d-1 This is the usage information of the current type of resource after the previous smoothing process.

6. The system according to claim 2, wherein: Each heterogeneous accelerator is used to detect whether the resource utilization of its own computing resources, memory resources, and cache resources exceeds the corresponding utilization threshold; If it exceeds, it is determined that the remaining amount of such resources is insufficient, and a corresponding message of insufficient resources is used as the detection result; If not, it is confirmed that the remaining amount of this type of resources is sufficient, and the corresponding resource sufficiency message is used as the detection result.

7. The system according to claim 1, wherein: Each heterogeneous resource node is used to collect the detection results recorded in the cache area of each heterogeneous accelerator into the memory area of the network card in the heterogeneous resource node by using the cache consistency protocol for storage.

8. The system according to claim 1, wherein: Each heterogeneous resource node and each heterogeneous accelerator are also connected via Ethernet.

9. The system according to claim 1, wherein: The heterogeneous resource nodes and heterogeneous accelerators also communicate with each other through remote direct data access technology.

10. The system according to claim 1, wherein: Each heterogeneous resource node is used to: utilize its own network card to process the collected information with a load balancing algorithm, and allocate tasks to each heterogeneous accelerator according to the corresponding processing results.

11. The system according to any one of claims 1 to 10, characterized in that The network card memory of each heterogeneous resource node synchronizes data with the cache area of each heterogeneous accelerator using a cache consistency protocol.

12. A load balancing method, characterized in that: Applied to a load balancing system, the load balancing system includes: multiple heterogeneous resource nodes and multiple heterogeneous accelerators; the network card of each heterogeneous resource node is connected to the cache area of each heterogeneous accelerator using a cache consistency protocol; The load balancing method includes: Each heterogeneous accelerator detects whether the remaining amount of each type of resource in itself is sufficient, obtains a detection result, and updates the detection result to its own cache area; Each heterogeneous resource node collects the detection results recorded in the cache area of each heterogeneous accelerator to its own network card using the cache consistency protocol, and uses the network card to refer to the collected information to allocate tasks to each heterogeneous accelerator to achieve load balancing of each heterogeneous accelerator.

13. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to execute the computer program to implement the method according to claim 12.

14. A non-volatile storage medium, characterized in that: Used for storing a computer program, wherein the computer program implements the method according to claim 12 when executed by a processor.

15. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method of claim 12 is implemented.

Images