A load balancing system, method, device, medium and program product

Abstract

The application discloses a load balancing system, method, device, medium and program product in the computer technology field. The application performs task dispatching on an accelerator based on the actual resource remaining amount of the accelerator, combines the actual operation of the accelerator, can guarantee that the task dispatching result meets the expectation as much as possible, and provides the possibility for balancing the load sizes of different accelerators. In addition, the data synchronization of the network cards of various heterogeneous resource nodes and the cache areas of various heterogeneous accelerators can improve the data transmission and task dispatching efficiency between the heterogeneous resource nodes and the heterogeneous accelerators.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and particularly relates to a load balancing system, method, device, medium and program product. BACKGROUND

[0002] At present, a task dispatching node infers running pressure of an accelerator according to communication between the task dispatching node and the accelerator, and then performs task dispatching on the accelerator according to the inference result. However, the inferred running pressure of the accelerator may not be consistent with the actual running pressure of the accelerator, which leads to unexpected task dispatching and difficulty in balancing the load sizes of different accelerators.

[0003] Therefore, how to balance the load sizes of different accelerators is a problem to be solved by those skilled in the art. SUMMARY

[0004] Therefore, how to balance the load sizes of different accelerators is a problem to be solved by those skilled in the art.

[0005] In a first aspect, the present application provides a load balancing system, comprising: a plurality of heterogeneous resource nodes and a plurality of heterogeneous accelerators; a network card of each of the heterogeneous resource nodes is connected to a cache area of each of the heterogeneous accelerators by a cache consistency protocol; each of the heterogeneous accelerators is configured to detect whether the remaining amount of each type of resource in the heterogeneous accelerator is sufficient, obtain a detection result, and update the detection result to the cache area in the heterogeneous accelerator; and each of the heterogeneous resource nodes is configured to collect the detection result recorded by the cache area in each of the heterogeneous accelerators to the network card in the heterogeneous resource node by using the cache consistency protocol, and allocate tasks to each of the heterogeneous accelerators by using the network card to refer to the collected information, so as to balance the loads of the heterogeneous accelerators.

[0006] In a second aspect, the present application provides a load balancing method applied to a load balancing system, wherein the load balancing system comprises: a plurality of heterogeneous resource nodes and a plurality of heterogeneous accelerators; a network card of each of the heterogeneous resource nodes is connected to a cache area of each of the heterogeneous accelerators by a cache consistency protocol; and the load balancing method comprises: each of the heterogeneous accelerators detects whether the remaining amount of each type of resource in the heterogeneous accelerator is sufficient, obtains a detection result, and updates the detection result to the cache area in the heterogeneous accelerator; and each of the heterogeneous resource nodes collects the detection result recorded by the cache area in each of the heterogeneous accelerators to the network card in the heterogeneous resource node by using the cache consistency protocol, and allocates tasks to each of the heterogeneous accelerators by using the network card to refer to the collected information, so as to balance the loads of the heterogeneous accelerators.

[0007] In a third aspect, the present application provides an electronic device, comprising: a memory configured to store a computer program; and a processor configured to execute the computer program to implement the load balancing method disclosed above.

[0008] In a fourth aspect, the present application provides a non-transitory storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the load balancing method disclosed above.

[0009] In a fifth aspect, the present application provides a computer program product comprising computer programs / instructions which, when executed by a processor, implement the steps of the load balancing method disclosed above.

[0010] According to the above solution, the present application provides a load balancing system, comprising: a plurality of heterogeneous resource nodes and a plurality of heterogeneous accelerators; the network card of each heterogeneous resource node is connected to the cache area of each heterogeneous accelerator by a cache consistency protocol; each heterogeneous accelerator is configured to detect whether the remaining amount of each type of resource in itself is sufficient, obtain a detection result, and update the detection result to the cache area in itself; and each heterogeneous resource node is configured to collect the detection result recorded by the cache area in each heterogeneous accelerator to the network card in itself by using the cache consistency protocol, and allocate tasks for each heterogeneous accelerator by using the network card in reference to the collected information, so as to realize load balancing of each heterogeneous accelerator.

[0011] It can be seen that the present application has the following beneficial effects: the network card of each heterogeneous resource node is connected to the cache area of each heterogeneous accelerator by a cache consistency protocol, so that data synchronization between the network card of each heterogeneous resource node and the cache area of each heterogeneous accelerator can be realized. On this basis, each heterogeneous resource node can quickly obtain the detection result of whether the remaining amount of each type of resource recorded in the cache area of each heterogeneous accelerator is sufficient by using the network card, and allocate tasks for each heterogeneous accelerator accordingly, so as to realize load balancing of each heterogeneous accelerator. This solution allocates tasks for accelerators based on the actual resource remaining amount of the accelerators, and takes into account the actual running condition of the accelerators, so as to ensure that the task allocation result meets the expectation as much as possible, and provides the possibility of balancing the load size of different accelerators; and the data synchronization between the network card of each heterogeneous resource node and the cache area of each heterogeneous accelerator can improve the data transmission and task allocation efficiency between the heterogeneous resource node and the heterogeneous accelerator. Because the network card is closer to the network and has a faster response speed, the CPU of the heterogeneous resource node does not need to participate in the load balancing process.

[0012] Correspondingly, the load balancing method, device, medium and program product provided by the present application also have the above technical effects. BRIEF DESCRIPTION OF DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of the provided drawings.

[0014] Figure 1 A schematic diagram of a load balancing system disclosed in the present application;

[0015] Figure 2 A schematic diagram of a second load balancing system disclosed in the present application;

[0016] Figure 3 A schematic diagram of a connection of a heterogeneous resource node and a heterogeneous accelerator pooling node disclosed in the present application;

[0017] Figure 4 A schematic diagram of a third load balancing system disclosed in the present application;

[0018] Figure 5 A schematic diagram of a load balancing process disclosed in the present application;

[0019] Figure 6 A flow chart of a resource usage state updating process disclosed in the present application;

[0020] Figure 7 A schematic diagram of a server structure provided in the present application;

[0021] Figure 8 A schematic diagram of a terminal structure provided in the present application. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, any other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0023] It should be noted that, in the description of the present application, the terms “comprise”, “contain” or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. The terms “first”, “second” and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0024] In order for those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0025] Currently, a task dispatching node infers running pressure of an accelerator according to communication between the task dispatching node and the accelerator, and then dispatches tasks to the accelerator according to the inference result. However, the inferred running pressure of the accelerator may not be consistent with the actual running pressure of the accelerator, which leads to unexpected task dispatching and difficulty in balancing the load sizes of different accelerators. To this end, the present application provides a load balancing scheme that can balance the load sizes of different accelerators.

[0026] Referring to Figure 1 As shown in the figure, the embodiments of the present application disclose a load balancing system, comprising: a plurality of heterogeneous resource nodes and a plurality of heterogeneous accelerators; the network cards of each heterogeneous resource node are connected to the cache areas of each heterogeneous accelerator by a cache coherence protocol. Each heterogeneous resource node can be a server or the like, and each heterogeneous accelerator can be an FPGA, a GPU or the like.

[0027] It should be noted that each heterogeneous resource node and each heterogeneous accelerator are also connected by Ethernet. In addition, each heterogeneous resource node and each heterogeneous accelerator also communicate through remote direct memory access technology. Therefore, when any heterogeneous resource node dispatches tasks to any heterogeneous accelerator, the direct reading and writing of task-related data can be realized based on the remote direct memory access technology, thereby improving the task processing efficiency. The tasks can be storage type, computing type, etc., such as model training tasks, etc.

[0028] In this embodiment, each heterogeneous accelerator is configured to detect whether the remaining amount of each type of resource in the accelerator is sufficient, obtain a detection result, and update the detection result to the cache area in the accelerator.

[0029] In this embodiment, each heterogeneous resource node is configured to collect the detection results recorded by the cache areas in each heterogeneous accelerator to the network card in the heterogeneous resource node by using the cache coherence protocol, and allocate tasks to each heterogeneous accelerator by using the network card to refer to the collected information, so as to realize load balancing of each heterogeneous accelerator. For example, each heterogeneous resource node stores the detection results recorded by the cache areas in each heterogeneous accelerator to the memory area of the network card in the heterogeneous resource node by using the cache coherence protocol.

[0030] In one embodiment, each heterogeneous accelerator is configured to collect usage information of each type of resource in the accelerator at regular intervals, and detect whether the remaining amount of each type of resource in the accelerator is sufficient according to the usage information. For example, the resource usage rate of the computing resource, the memory resource and the cache resource in the accelerator is used as the usage information of each type of resource. That is, each type of resource can include a computing resource, a memory resource and a cache resource.

[0031] In an embodiment, each heterogeneous accelerator is configured to: smooth the usage information using a time domain low-pass filter, and detect whether the remaining amount of each type of resource in the heterogeneous accelerator is sufficient based on the smoothed result. In this embodiment, each heterogeneous accelerator is configured to: smooth the usage information of any type of resource using the formula X d =a×X p +(1-a)×X d-1 , where X d is the smoothed usage information of any type of resource, a is a preset filter coefficient, X p is the usage information of the current type of resource before smoothing, and X d-1 is the usage information of the current type of resource after smoothing in the previous time. For example, X d is the smoothed resource usage rate of the computing resource, a is a preset filter coefficient, X p is the usage information of the computing resource before smoothing (i.e., the actual resource usage rate collected), and X d-1 is the usage information of the computing resource after smoothing in the previous time.

[0032] In order to reduce the amount of data synchronization between the network card memory of each heterogeneous resource node and the cache area of each heterogeneous accelerator, in this embodiment, the cache area of the heterogeneous accelerator only records the notification message of whether the remaining amount of each type of resource is sufficient. Thus, the network card of each heterogeneous resource node can obtain the notification message, and accordingly dispatch tasks based on the notification message. In an embodiment, each heterogeneous accelerator is configured to: detect whether the resource usage rate of the computing resource, the memory resource and the cache resource in the heterogeneous accelerator exceeds the corresponding usage rate threshold, respectively; if yes, determine that the remaining amount of the type of resource is insufficient, and take the corresponding resource insufficient message as the detection result; if no, determine that the remaining amount of the type of resource is sufficient, and take the corresponding resource sufficient message as the detection result.

[0033] In this embodiment, each heterogeneous resource node uses the network card therein to process the collected information using a load balancing algorithm, and allocates tasks to each heterogeneous accelerator according to the corresponding processing result. The load balancing algorithm can refer to the related art in the field, and will not be described herein.

[0034] It can be seen that in the embodiment, the network cards of the heterogeneous resource nodes are connected to the cache areas of the heterogeneous accelerators by the cache consistency protocol, so that the data synchronization between the network cards of the heterogeneous resource nodes and the cache areas of the heterogeneous accelerators can be realized. On this basis, the heterogeneous resource nodes can quickly obtain the detection results of whether the remaining amounts of various resources recorded in the cache areas of the heterogeneous accelerators are sufficient by using the network cards, and allocate tasks to the heterogeneous accelerators according to the detection results, so as to realize the load balancing of the heterogeneous accelerators. The scheme allocates tasks to the accelerators on the basis of the actual resource remaining amounts of the accelerators, and combines the actual running conditions of the accelerators, so as to ensure that the task allocation results meet the expectations as much as possible, and provide the possibility of balancing the load sizes of different accelerators. In addition, the data synchronization between the network cards of the heterogeneous resource nodes and the cache areas of the heterogeneous accelerators can improve the data transmission and task allocation efficiency between the heterogeneous resource nodes and the heterogeneous accelerators. Because the network cards are closer to the network and have a faster response speed, the CPU of the heterogeneous resource node does not need to participate in the load balancing process.

[0035] Please refer to Figure 2 The plurality of heterogeneous accelerators are pooled into a heterogeneous acceleration pool, so that the heterogeneous accelerators serve as heterogeneous accelerator pooling nodes in the heterogeneous acceleration pool. Each heterogeneous accelerator pooling node is connected to a heterogeneous resource node. The heterogeneous resource node places a load balancing module in a network card, and the memory in the network card synchronizes data with the cache of each heterogeneous accelerator pooling node by the CXL protocol, so that the real-time resource usage states of each heterogeneous accelerator pooling node are recorded in the memory of the network card. Then, the load balancing module determines which heterogeneous accelerator pooling node to use for task processing according to the real-time resource usage states.

[0036] The real-time resource usage states of the heterogeneous accelerator pooling nodes are determined based on the resource usage of the heterogeneous accelerator pooling nodes themselves. The data synchronization between the network cards of the heterogeneous resource nodes and the cache areas of the heterogeneous accelerator pooling nodes is equivalent to that the heterogeneous accelerator pooling nodes use a piece of memory on the heterogeneous resource node and store their real-time resource usage states on the piece of memory.

[0037] Please refer to Figure 3 The real-time resource usage states exist on the memory of the network card of the heterogeneous resource node. The reason for placing them on the memory of the network card is that the network card is closer to the network and has a faster response speed, so that the CPU of the heterogeneous resource node does not need to participate in the load balancing process. If the CPU of the heterogeneous resource node performs load balancing, the load balancing performance will change from milliseconds to microseconds, and the performance loss is huge. The network card also supports the connection of Ethernet, RDMA (Remote Direct Memory Access) and the heterogeneous accelerator pooling nodes.

[0038] Please refer to Figure 4If there are two heterogeneous resource nodes, each of the heterogeneous resource nodes is connected to all the heterogeneous accelerator pooling nodes. That is, each of the heterogeneous accelerator pooling nodes updates its real-time resource usage state to the network card of different heterogeneous resource nodes, so that each of the heterogeneous resource nodes can know the real-time resource usage state of all the heterogeneous accelerator pooling nodes. The network card of the heterogeneous resource node is a CXL type2 device.

[0039] The biggest performance advantage of CXL is that it can support cache. The efficiency of cache requires that data is not frequently changed. If the change is too frequent, data needs to be frequently synchronized between the heterogeneous resource node and the heterogeneous accelerator pooling node. In order to reduce the amount of data of the real-time resource usage state, only the sufficient or insufficient message of each type of resource can be recorded in the real-time resource usage state, and the resource usage information is not recorded.

[0040] In an example, the real-time resource usage state can be recorded with reference to Table 1.

[0041] Table 1

[0042]

[0043] The sufficiency or insufficiency is judged by the heterogeneous accelerator pooling node, and then synchronized to the network card of the heterogeneous resource node through CXL. In addition, the heterogeneous accelerator pooling node judges the sufficiency or insufficiency of each type of resource based on the resource usage rate of each type of resource. The resource usage rate of each type of resource can be smoothed by a time domain low pass filter before judging the sufficiency or insufficiency, so as to avoid frequent state reporting caused by instantaneous change.

[0044] The smoothing is performed by using a time domain low pass filter, which is expressed by the following formula: The formula is consistent with formula X d =a×X p +(1-a)×X d-1 , only the letters representing the parameters are different. Wherein, 0<α<1 is the filter coefficient, which is used to adjust the smoothing degree of the filter. The filter combines the resource usage rate at the current time and the filtering result at the previous time, and uses the size of a to control the importance of new data and the retention degree of historical trend, so as to smooth the resource usage rate, filter out high frequency noise and retain low frequency trend information. is the smoothed resource usage at the current time, is the measured resource usage, is the smoothed resource usage at the previous time.

[0045] Please refer to Figure 5A load balancing process includes: a heterogeneous application in a heterogeneous resource node sends a heterogeneous accelerator request (i.e., a task that needs to be processed by a heterogeneous accelerator pooling node) to a heterogeneous resource node network card, the network card finds a suitable heterogeneous accelerator pooling node according to a real-time resource usage state, and then schedules the heterogeneous accelerator request to the suitable heterogeneous accelerator pooling node, so that the heterogeneous accelerator pooling node processes and returns a corresponding processing result. In this example, the heterogeneous application does not know the related load balancing decision, which is very different from the traditional way. The heterogeneous application does not participate in the decision at all, which can improve the call performance and is very important for high real-time heterogeneous computing. Traditional load balancing is all participated by heterogeneous applications, which can meet the needs of WEB applications in real-time, but cannot meet the needs of high real-time heterogeneous computing.

[0046] Please refer to Figure 6 The heterogeneous accelerator pooling node determines whether to update the resource usage state according to its own resource usage state periodically. If the computing core resources were sufficient before, but are insufficient now, it needs to be updated. If the computing core resources were sufficient before, but are still sufficient now, but the number of idle computing cores has changed, it does not need to be updated. If the state is updated, the resource usage state is updated to the heterogeneous resource node network card through CXL.

[0047] It can be seen that the load balancing module on the heterogeneous resource node can perform corresponding load balancing operations after obtaining the real-time resource usage state of the heterogeneous accelerator pooling node, thereby improving the accuracy of load balancing and improving the overall performance of the system. The load balancing is realized by hardware, which meets the needs of high real-time heterogeneous computing applications and improves the overall performance of the system.

[0048] A load balancing method provided by the embodiments of the present application will be described below. The load balancing method described below can be mutually referred to with other embodiments described herein.

[0049] The embodiments of the present application disclose a load balancing method, which is applied to a load balancing system. The load balancing system includes: a plurality of heterogeneous resource nodes and a plurality of heterogeneous accelerators. The network card of each heterogeneous resource node is connected to the cache area of each heterogeneous accelerator in a cache coherence protocol. The load balancing method includes: each heterogeneous accelerator detects whether the remaining amount of each type of resource in itself is sufficient, obtains a detection result, and updates the detection result to the cache area in itself. Each heterogeneous resource node collects the detection result recorded by the cache area in each heterogeneous accelerator to the network card in itself by using the cache coherence protocol, and allocates tasks for each heterogeneous accelerator by using the network card to refer to the collected information, so as to realize load balancing of each heterogeneous accelerator.

[0050] In an embodiment, each heterogeneous accelerator is configured to: collect usage information of each type of resource in itself, and detect whether the remaining amount of each type of resource in itself is sufficient based on the usage information.

[0051] In an embodiment, each heterogeneous accelerator is configured to: use the resource usage rate of the computing resource, the memory resource and the cache resource in itself as the usage information of each type of resource.

[0052] In an embodiment, each heterogeneous accelerator is configured to: smooth the usage information by using a time domain low-pass filter, and detect whether the remaining amount of each type of resource in itself is sufficient based on the smoothing result.

[0053] In an embodiment, each heterogeneous accelerator is configured to: smooth the usage information of any type of resource by using the formula X d = a x X p + (1-a) x X d-1 ; wherein, X d is the smoothed usage information of any type of resource, a is a preset filter coefficient, X p is the usage information of the current type of resource before smoothing, and X d-1 is the usage information of the current type of resource after smoothing in the previous time. For example, X d is the smoothed resource usage rate of the computing resource, a is a preset filter coefficient, X p is the resource usage rate of the computing resource before smoothing (i.e. the actual resource usage rate collected), and X d-1 is the resource usage rate of the computing resource after smoothing in the previous time.

[0054] In an embodiment, each heterogeneous accelerator is configured to: respectively detect whether the resource usage rate of the computing resource, the memory resource and the cache resource in itself exceeds the corresponding usage rate threshold, and if so, confirm that the remaining amount of the type of resource is insufficient, and take the corresponding resource insufficient message as the detection result; and if not, confirm that the remaining amount of the type of resource is sufficient, and take the corresponding resource sufficient message as the detection result.

[0055] In an embodiment, each heterogeneous resource node is configured to: collect the detection results of the cache area records in each heterogeneous accelerator to the memory area of the network card in itself by using the cache coherence protocol, and store the detection results.

[0056] In an embodiment, each heterogeneous resource node and each heterogeneous accelerator are further connected by an Ethernet.

[0057] In an embodiment, each heterogeneous resource node and each heterogeneous accelerator are further connected by a remote direct data access technology.

[0058] In an implementation, each heterogeneous resource node is configured to process the collected information by using the network card therein with a load balancing algorithm, and assign tasks to each heterogeneous accelerator according to the corresponding processing result.

[0059] In an implementation, the network card memory of each heterogeneous resource node is configured to synchronize data with the cache area of each heterogeneous accelerator with a cache consistency protocol.

[0060] In the embodiments, the working processes of the modules and units are specifically described above, and thus no further repeated description is provided herein.

[0061] It can be seen that the embodiments provide a load balancing method, which assigns tasks to accelerators based on the actual resource remaining amount of the accelerators, and combines the actual running condition of the accelerators, so as to ensure that the task assignment result meets the expectation as much as possible, and provides a possibility for balancing the load sizes of different accelerators. In addition, the data synchronization between the network card of each heterogeneous resource node and the cache area of each heterogeneous accelerator can improve the data transmission and task assignment efficiency between the heterogeneous resource node and the heterogeneous accelerator. Because the network card is closer to the network and has a faster response speed, the CPU of the heterogeneous resource node does not need to participate in the load balancing process.

[0062] An electronic device provided by the embodiments of the present application is described below, and the electronic device described below can be combined with other embodiments described herein.

[0063] The embodiments of the present application disclose an electronic device, which comprises a memory configured to save a computer program, and a processor configured to execute the computer program to implement the method disclosed in any of the embodiments.

[0064] In the embodiments, when the processor executes the computer program saved in the memory, the following steps can be specifically implemented: detecting whether the remaining amount of each type of resource in the processor is sufficient, obtaining a detection result, and updating the detection result to the cache area in the processor.

[0065] In the embodiments, when the processor executes the computer program saved in the memory, the following steps can be specifically implemented: collecting the detection result recorded in the cache area of each heterogeneous accelerator to the network card in the processor by using a cache consistency protocol, and assigning tasks to each heterogeneous accelerator by using the network card to refer to the collected information, so as to implement the load balancing of each heterogeneous accelerator.

[0066] In the embodiments, when the processor executes the computer program saved in the memory, the following steps can be specifically implemented: collecting the usage information of each type of resource in the processor at a regular time, and detecting whether the remaining amount of each type of resource in the processor is sufficient according to the usage information.

[0067] In this embodiment, when the processor executes the computer program stored in the memory, the following steps can be specifically implemented: taking the resource usage of the computing resource, the memory resource and the cache resource in itself as the usage information of various resources.

[0068] In this embodiment, when the processor executes the computer program stored in the memory, the following steps can be specifically implemented: smoothing the usage information by using a time domain low-pass filter, and detecting whether the remaining amount of various resources in itself is sufficient based on the smoothing result.

[0069] In this embodiment, when the processor executes the computer program stored in the memory, the following steps can be specifically implemented: respectively detecting whether the resource usage of the computing resource, the memory resource and the cache resource in itself exceeds the corresponding usage threshold value; if yes, confirming that the remaining amount of the resource is insufficient, and taking the corresponding resource insufficient message as the detection result; if not, confirming that the remaining amount of the resource is sufficient, and taking the corresponding resource sufficient message as the detection result.

[0070] In this embodiment, when the processor executes the computer program stored in the memory, the following steps can be specifically implemented: collecting the detection results of the cache area records in various heterogeneous accelerators to the memory area of the network card in itself by using the cache consistency protocol.

[0071] In this embodiment, when the processor executes the computer program stored in the memory, the following steps can be specifically implemented: processing the collected information by using the network card in itself by using a load balancing algorithm, and allocating tasks to various heterogeneous accelerators according to the corresponding processing result.

[0072] Further, the embodiment of the present application also provides an electronic device. Wherein, the above-mentioned electronic device can be a server as shown in Figure 7 , or a terminal as shown in Figure 8 . Figure 7 and Figure 8 are structural diagrams of electronic devices according to an exemplary embodiment, and the contents in the diagrams cannot be considered as any limitation to the use range of the present application.

[0073] Figure 7 A structural diagram of a server provided by the embodiment of the present application. The server can specifically include: at least one processor, at least one memory, a power supply, a communication interface, an input / output interface and a communication bus. Wherein, the memory is used to store a computer program, the computer program is loaded and executed by the processor to realize the related steps in the load balancing disclosed in any of the preceding embodiments.

[0074] In this embodiment, the power supply is used to provide working voltage for each hardware device on the server; the communication interface can create a data transmission channel between the server and external devices, and the communication protocol followed by the communication interface is any communication protocol applicable to the technical solution of the present application, which is not specifically limited herein; the input and output interface is used to obtain external input data or output data to the outside world, and the specific interface type can be selected according to the specific application needs, which is not specifically limited herein.

[0075] In addition, the memory as a carrier of resource storage can be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the resources stored thereon include an operating system, a computer program and data, etc., and the storage mode can be temporary storage or permanent storage.

[0076] The operating system is used to manage and control each hardware device and computer program on the server to realize the operation and processing of the processor on the data in the memory, and can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program capable of completing the load balancing method disclosed in any of the preceding embodiments, the computer program can further include a computer program capable of completing other specific work. In addition to the data including the update information of the application program and other data, the data can also include the developer information of the application program and other data.

[0077] Figure 8 A structure schematic diagram of a terminal provided by the embodiment of the present application, the terminal specifically can include but is not limited to a smart phone, a tablet computer, a notebook computer or a desktop computer, etc.

[0078] Generally, the terminal in the embodiment includes a processor and a memory.

[0079] The processor can include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor can be implemented in at least one of a hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), a PLA (Programmable Logic Array). The processor can also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also referred to as a CPU (Central Processing Unit). The coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor can be integrated with a GPU (Graphics Processing Unit) that is responsible for rendering and drawing the content required to be displayed on the display screen. In some embodiments, the processor can also include an AI (Artificial Intelligence) processor for processing machine learning-related computing operations.

[0080] The memory can include one or more computer non-volatile storage media that can be non-transitory. The memory can also include a high-speed random access memory and a non-volatile memory such as one or more disk storage devices, flash memory devices. In this embodiment, the memory is at least used to store the following computer programs, wherein the computer programs are loaded and executed by the processor, and can realize the related steps in the load balancing method executed by the terminal side disclosed in any of the preceding embodiments. In addition, the resources stored by the memory can also include an operating system and data, and the storage mode can be temporary storage or permanent storage. The operating system can include Windows, Unix, Linux, and the like. The data can include but is not limited to application update information.

[0081] In some embodiments, the terminal can also include a display screen, an input / output interface, a communication interface, a sensor, a power supply, and a communication bus.

[0082] Those skilled in the art can understand that the structure shown in the above embodiments does not constitute a limitation on the terminal, and can include more or fewer components than the illustrated components. Figure 8 The terminal can include more or fewer components than those shown in the above embodiments.

[0083] A non-volatile storage medium provided by an embodiment of the present application is described below. The non-volatile storage medium described below can be mutually referred to with other embodiments described herein.

[0084] A non-volatile storage medium for storing a computer program, wherein the computer program is executed by a processor to implement the load balancing method disclosed in the foregoing embodiments. The non-volatile storage medium is a computer-readable non-volatile storage medium, which is a carrier for storing resources, and can be a read-only memory, a random access memory, a magnetic disk, an optical disk, or the like. The resources stored on the non-volatile storage medium include an operating system, a computer program, and data, etc. The storage mode can be temporary storage or permanent storage.

[0085] In the embodiment, when the processor executes the computer program stored in the non-volatile storage medium, the following steps can be specifically implemented: detecting whether the remaining amount of each type of resource in the processor is sufficient, obtaining a detection result, and updating the detection result to a cache area in the processor.

[0086] In the embodiment, when the processor executes the computer program stored in the non-volatile storage medium, the following steps can be specifically implemented: collecting the detection results recorded in the cache areas of each heterogeneous accelerator to a network card in the processor by using a cache consistency protocol, and allocating tasks to each heterogeneous accelerator by using the network card to refer to the collected information, so as to achieve load balancing of each heterogeneous accelerator.

[0087] In the embodiment, when the processor executes the computer program stored in the non-volatile storage medium, the following steps can be specifically implemented: collecting usage information of each type of resource in the processor at a regular time, and detecting whether the remaining amount of each type of resource in the processor is sufficient according to the usage information.

[0088] In the embodiment, when the processor executes the computer program stored in the non-volatile storage medium, the following steps can be specifically implemented: taking the resource usage rates of the computing resources, the memory resources, and the cache resources in the processor as the usage information of each type of resource.

[0089] In the embodiment, when the processor executes the computer program stored in the non-volatile storage medium, the following steps can be specifically implemented: smoothing the usage information by using a time-domain low-pass filter, and detecting whether the remaining amount of each type of resource in the processor is sufficient based on the smoothing result.

[0090] In the embodiment, when the processor executes the computer program stored in the non-volatile storage medium, the following steps can be specifically implemented: respectively detecting whether the resource usage rates of the computing resources, the memory resources, and the cache resources in the processor exceed corresponding usage rate thresholds; if yes, confirming that the remaining amount of the corresponding type of resource is insufficient, and taking a corresponding resource insufficient message as the detection result; and if no, confirming that the remaining amount of the corresponding type of resource is sufficient, and taking a corresponding resource sufficient message as the detection result.

[0091] In the embodiment, when the processor executes the computer program stored in the nonvolatile storage medium, the following steps can be specifically implemented: collecting the detection results of the cache area records in each heterogeneous accelerator to the memory area of the network card in the processor by using the cache consistency protocol and storing the detection results.

[0092] In the embodiment, when the processor executes the computer program stored in the nonvolatile storage medium, the following steps can be specifically implemented: processing the collected information by using the network card in the processor according to the load balancing algorithm, and allocating tasks to each heterogeneous accelerator according to the corresponding processing results.

[0093] The following introduces a computer program product provided by the embodiment of the application, and the computer program product described below can be mutually referred to with other embodiments described herein.

[0094] A computer program product comprises computer programs / instructions, which, when executed by a processor, implement the steps of the load balancing method disclosed above.

[0095] The embodiment of the application further provides another computer program product, which comprises a nonvolatile computer readable storage medium, and the nonvolatile computer readable storage medium is used for storing a computer program, and the computer program, when executed by a processor, implements the steps in any of the above embodiments.

[0096] In the embodiment, when the processor executes the computer program stored in the nonvolatile storage medium, the following steps can be specifically implemented: detecting whether the remaining amount of each type of resource in the processor is sufficient, obtaining a detection result, and updating the detection result to the cache area in the processor.

[0097] In the embodiment, when the processor executes the computer program stored in the nonvolatile storage medium, the following steps can be specifically implemented: collecting the detection results of the cache area records in each heterogeneous accelerator to the network card in the processor by using the cache consistency protocol, and allocating tasks to each heterogeneous accelerator by using the network card according to the collected information, so as to realize the load balancing of each heterogeneous accelerator.

[0098] In the embodiment, when the processor executes the computer program stored in the nonvolatile storage medium, the following steps can be specifically implemented: collecting the usage information of each type of resource in the processor at a regular time, and detecting whether the remaining amount of each type of resource in the processor is sufficient according to the usage information.

[0099] In the embodiment, when the processor executes the computer program stored in the nonvolatile storage medium, the following steps can be specifically implemented: taking the resource usage rates of the computing resource, the memory resource and the cache resource in the processor as the usage information of each type of resource.

[0100] In the embodiment, the processor executes the computer program stored in the nonvolatile storage medium, and the following steps can be implemented: the time domain low pass filter is used to smooth the usage information, and the remaining amount of each type of resource in the processor is detected based on the smoothing result.

[0101] In the embodiment, the processor executes the computer program stored in the nonvolatile storage medium, and the following steps can be implemented: the resource usage rate of the computing resource, the memory resource and the cache resource in the processor is detected respectively, whether the resource usage rate exceeds the corresponding threshold value; if yes, it is confirmed that the remaining amount of the resource is insufficient, and the corresponding resource insufficient message is taken as the detection result; if no, it is confirmed that the remaining amount of the resource is sufficient, and the corresponding resource sufficient message is taken as the detection result.

[0102] In the embodiment, the processor executes the computer program stored in the nonvolatile storage medium, and the following steps can be implemented: the detection results of the cache area records in each heterogeneous accelerator are collected to the memory area of the network card in the processor by using the cache consistency protocol.

[0103] In the embodiment, the processor executes the computer program stored in the nonvolatile storage medium, and the following steps can be implemented: the collected information is processed by using the network card in the processor according to the load balancing algorithm, and each heterogeneous accelerator is allocated with a task according to the corresponding processing result.

[0104] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments, and the same or similar parts of each embodiment can be referred to each other.

[0105] The steps of the method or algorithm described in combination with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in a random access memory (RAM), a memory, a read only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of nonvolatile storage medium known in the art.

[0106] The principles and implementation modes of the application are described by using specific examples in the present disclosure, and the above embodiment description is only used to help understand the method and core idea of the application; meanwhile, for those skilled in the art, according to the idea of the application, the specific implementation mode and application range can be changed, and the above description should not be understood as a limitation of the application.

Claims

1. A load balancing system, characterized by, The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. Each isomerization accelerator is configured to utilize the formula X d = a * X p + (1-a) * X d-1 to smooth usage information for any type of resource, and to detect whether a sufficient amount of each type of resource is available in the isomerization accelerator based on the smoothed usage information. In this formula, X d is the smoothed usage information for any type of resource, a is a predetermined filter coefficient, X p is the usage information for the current type of resource before smoothing, and X d-1 is the usage information for the current type of resource after smoothing.

2. The system of claim 1, wherein, The application relates to a load balancing method and system.

3. The system of claim 2, wherein, The application relates to a load balancing method and system.

4. The system of claim 2, wherein, The application relates to a load balancing method and system.

5. The system of claim 2, wherein, The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system.

6. The system of claim 1, wherein, The application relates to a load balancing method and system.

7. The system of claim 1, wherein, The application relates to a load balancing method and system.

8. The system of claim 1, wherein, The application relates to a load balancing method and system.

9. The system according to any one of claims 1 to 8, characterized in that, The application relates to a load balancing method and system.

10. A load balancing method, characterized by, The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application relates to a load balancing method and system. The application The isomerous resource nodes collect the detection results of the cache area records in the isomerous accelerators to the network cards in the isomerous resource nodes by using the cache consistency protocol, and allocate tasks for the isomerous accelerators by using the network cards to refer to the collected information, so as to realize the load balancing of the isomerous accelerators; wherein the network card memory of the isomerous resource nodes is synchronized with the cache of each isomerous accelerator, and the real-time resource usage state of each isomerous accelerator is recorded in the network card memory; The isomerous resource nodes process the collected information by using the network cards in the isomerous resource nodes with a load balancing algorithm, and allocate tasks for the isomerous accelerators according to the corresponding processing results; Each heterogeneous accelerator is configured to: utilize the formula X d = a * X p + (1-a) * X d-1 smooth usage information of any type of resource, and detect whether the remaining amount of each type of resource in the self is sufficient based on a result of the smoothing; wherein X d is the smoothed usage information of any type of resource, a is a preset filter coefficient, X p is the usage information of the current type of resource before smoothing, and X d-1 is the usage information of the current type of resource after the previous smoothing.

11. An electronic device, comprising: Comprise: A memory for storing a computer program; A processor for executing the computer program to implement the method of claim 10.

12. A non-volatile storage medium, comprising: A computer program for saving, wherein the computer program is executed by a processor to implement the method of claim 10.

13. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instruction is executed by the processor to implement the method of claim 10. The computer program / instruction is executed by the processor to implement the method of claim 10.

Images