Load balancing method and device for hybrid heterogeneous system, equipment, storage medium and product

Abstract

The application discloses a load balancing method and device for a hybrid heterogeneous system, equipment, a storage medium and a product. The method comprises the following steps: performing performance evaluation on each device in a working cluster of the hybrid heterogeneous system based on a pre-trained performance evaluation model, and generating a performance evaluation value; wherein the performance evaluation value represents the comprehensive performance of the corresponding device under multiple types of tasks; dividing all devices in the working cluster into a set number of device groups based on the performance evaluation values of the devices; and distributing a plurality of tasks to be processed to each device group respectively; wherein the set number is the same as the number of tasks, and each device group corresponds to each task to be processed; and performing load balancing control on each task based on the device group. In this way, intelligent grouping can be performed based on the performance evaluation values of each device in the working cluster, and the heterogeneous hardware can be uniformly managed and scheduled based on the device group, so that the utilization rate of computing power resources and the overall energy efficiency of the hybrid heterogeneous system can be effectively improved.

Description

Technical Field

[0001] This application relates to the field of high-performance computing technology, and in particular to a load balancing method, apparatus, device, storage medium and product for hybrid heterogeneous systems. Background Technology

[0002] With the rapid development of emerging technologies such as 5G, big data, and cloud computing, computing scenarios are becoming increasingly diverse, further increasing the demand for large-scale computing power and highlighting the growing contradiction between computing power supply and demand. Modern intelligent computing centers and cloud computing increasingly rely on hybrid heterogeneous systems (HHS) to handle various complex and diverse computing tasks. These systems typically include different types of computing resources, such as central processing units (CPUs), graphics processing units (GPUs), field-programmable gate arrays (FPGAs), and dedicated accelerators.

[0003] Heterogeneous computing has become an effective way to break through the performance limits of CPUs, alleviate GPU production capacity shortages, and improve the supply level of computing power. It provides high-performance, low-latency, and large-scale computing power in conjunction with CPUs through various XPU accelerators to meet the application needs of different scenarios. However, heterogeneous computing power has placed higher demands on the management of diversified and large-scale computing resources, leading to the further manifestation of problems such as performance bottlenecks, difficulty in balancing high performance and flexibility, and "computing silos". Summary of the Invention

[0004] In view of this, embodiments of this application provide a load balancing method, apparatus, device, storage medium, and product for hybrid heterogeneous systems, aiming to improve the utilization rate of computing resources and overall energy efficiency of hybrid heterogeneous systems.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] In a first aspect, embodiments of this application provide a load balancing method for hybrid heterogeneous systems, including:

[0007] The performance of each device in the working cluster of a hybrid heterogeneous system is evaluated based on a pre-trained performance evaluation model, and performance evaluation values ​​are generated; wherein, the performance evaluation values ​​represent the comprehensive performance of the corresponding device under multiple types of tasks.

[0008] Based on the performance evaluation values ​​of each device, all devices in the working cluster are divided into a set number of device groups;

[0009] Multiple tasks to be processed are assigned to various device groups; wherein, the set number is the same as the number of tasks, and each device group corresponds one-to-one with each task to be processed;

[0010] Load balancing control is performed on each task based on the aforementioned equipment group.

[0011] In the above scheme, the load balancing control of each task based on the device group includes at least one of the following:

[0012] If the set dynamic adjustment conditions are met, then perform at least one of the following load balancing controls:

[0013] Based on load migration, a portion of the workload of tasks in a high-load device group is migrated to a low-load device group;

[0014] Based on the devices in the backup cluster, replace the faulty devices in each device group;

[0015] The work cluster can be elastically expanded and / or elastically reduced.

[0016] In the above scheme, determining whether the set dynamic adjustment conditions are met includes:

[0017] Determine that the load rate of at least one device group is greater than or equal to a set load rate threshold; and / or,

[0018] A fault is determined to exist based on the current performance values ​​of each device in the work cluster.

[0019] In the above scheme, the performance of each device in the working cluster of the hybrid heterogeneous system is evaluated based on a pre-trained performance evaluation model, generating performance evaluation values, including:

[0020] For each device in the working cluster of a hybrid heterogeneous system, a feature vector of the device is constructed based on a first feature information characterizing the hardware configuration and a second feature information characterizing the performance under any task.

[0021] The feature vector of the device is input into a pre-trained performance evaluation model to obtain the performance evaluation value of the device.

[0022] The method in the above scheme further includes:

[0023] The multilayer perceptron is trained based on the training sample set using a loss function to obtain the pre-trained performance evaluation model.

[0024] The training sample set includes a feature vector of the device, the true performance value of the device, and a domain label representing the task processed by the device. The loss function is determined based on a first loss function representing the regression task loss and a second loss function representing the domain classification loss. The first loss function is used to determine the loss value between the predicted performance evaluation value and the true performance value, and the second loss function is used to determine the loss value between the predicted task processed by the device and the domain label.

[0025] The method in the above scheme further includes:

[0026] Obtain historical operating data of each device in the work cluster, and construct a training sample set based on the historical operating data;

[0027] The pre-trained performance evaluation model is trained and updated based on the newly constructed training sample set.

[0028] In the above scheme, dividing all devices in the work cluster into a set number of device groups based on the performance evaluation values ​​of each device includes:

[0029] Sort all devices in the work cluster based on the performance evaluation value;

[0030] The sorted devices are added to each device group sequentially using a greedy algorithm until the difference in the sum of the performance evaluation values ​​between a set number of device groups is less than or equal to a set difference threshold.

[0031] Secondly, embodiments of this application also provide a load balancing device for hybrid heterogeneous systems, comprising:

[0032] The performance evaluation module is used to evaluate the performance of each device in the working cluster of a hybrid heterogeneous system based on a pre-trained performance evaluation model and generate performance evaluation values; wherein, the performance evaluation values ​​represent the comprehensive performance of the corresponding device under multiple types of tasks;

[0033] A grouping module is used to divide all devices in the working cluster into a set number of device groups based on the performance evaluation value of each device.

[0034] The task allocation module is used to allocate multiple tasks to be processed to various device groups; wherein, the set number is the same as the number of tasks, and each device group corresponds one-to-one with each task to be processed;

[0035] The control module is used to perform load balancing control on each task based on the device group.

[0036] Thirdly, embodiments of this application also provide an electronic device, including: a processor and a memory for storing a computer program capable of running on the processor, wherein, when the processor is used to run the computer program, it executes the steps of the method described in the first aspect of embodiments of this application.

[0037] Fourthly, embodiments of this application also provide a computer storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described in the first aspect of embodiments of this application.

[0038] Fifthly, embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect of embodiments of this application.

[0039] The technical solution provided in this application provides performance evaluation of each device in a hybrid heterogeneous system's working cluster based on a pre-trained performance evaluation model, generating performance evaluation values. These performance evaluation values ​​characterize the comprehensive performance of the corresponding device under multiple types of tasks. Based on the performance evaluation values ​​of each device, all devices in the working cluster are divided into a predetermined number of device groups. Multiple tasks to be processed are then assigned to each device group. The predetermined number is the same as the number of tasks, and each device group corresponds one-to-one with each task to be processed. Load balancing control is then performed on each task based on the device groups. In this way, intelligent grouping based on the performance evaluation values ​​of each device in the working cluster and unified management and scheduling of heterogeneous hardware based on device groups can effectively improve the utilization rate of computing resources and the overall energy efficiency of the hybrid heterogeneous system. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating the load balancing method for hybrid heterogeneous systems according to an embodiment of this application.

[0041] Figure 2 This is a schematic diagram illustrating the principle of generating performance evaluation values ​​for a device based on a multilayer perceptron, as described in an embodiment of this application.

[0042] Figure 3 This is a schematic diagram illustrating the principle of intelligent grouping of devices in a working cluster of a hybrid heterogeneous system, as described in an application embodiment of this application.

[0043] Figure 4 This is a flowchart illustrating a load balancing method for hybrid heterogeneous systems, as described in an application embodiment of this application.

[0044] Figure 5 This is a schematic diagram of the structure of a load balancing device for hybrid heterogeneous systems according to an embodiment of this application;

[0045] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0046] The present application will now be described in further detail with reference to the accompanying drawings and embodiments.

[0047] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0048] In related technologies, load balancing is a technique that evenly distributes computing tasks across multiple accelerators. By load balancing, request pressure can be effectively distributed, improving the overall throughput of the system while enhancing its fault tolerance and scalability. Load balancing technology is widely used in various scenarios such as websites, database clusters, and cloud services, and is an indispensable part of modern intelligent computing center cluster management.

[0049] Heterogeneous computing based on hybrid heterogeneous systems refers to computing power services provided by heterogeneous devices with different computing architectures. It often involves computing resources supporting heterogeneous devices such as CPUs, GPUs, FPGAs, and ASICs (Application-Specific Integrated Circuits). Hybrid heterogeneous systems require intelligent load balancing algorithms to ensure efficient task allocation. Common load balancing strategies include the following:

[0050] Average distribution strategy: Distribute tasks evenly across all available accelerator cards. This strategy is simple to implement, but it cannot fully utilize the computing power of the accelerator cards;

[0051] Priority-based allocation strategy: Tasks are assigned to different accelerator cards based on their priority. This strategy can better meet the needs of important tasks, but may result in low-priority tasks lacking computing power.

[0052] Load-based allocation strategy: This strategy distributes tasks to different accelerator cards based on the current task load, which can improve the utilization efficiency of computing resources.

[0053] It should be noted that as the application scope and scenarios of heterogeneous computing expand, the coexistence of accelerators such as CPUs, GPUs, DAS (Domain-Specific Architectures), and FPGAs will lead to an increasingly prominent "island" problem. Furthermore, CPUs and DPUs, as central units of heterogeneous systems, need to coordinate and manage other hardware accelerators. When the number of accelerators increases or the complexity of tasks increases, it can lead to uneven resource load distribution, exacerbating the performance burden on the central unit. Clearly, heterogeneous computing places higher demands on the management of diversified and large-scale computing resources, further highlighting problems such as performance bottlenecks, the difficulty of balancing high performance and flexibility, and "computing islands."

[0054] Based on this, in various embodiments of this application, heterogeneous devices are intelligently grouped based on performance evaluation values, and multiple tasks to be processed are allocated to each device group. Load balancing control is performed on each task based on the device group, thereby realizing unified management and scheduling of heterogeneous devices to improve the utilization rate of computing resources and overall energy efficiency of the hybrid heterogeneous system.

[0055] For example, embodiments of this application provide a load balancing method for hybrid heterogeneous systems, such as... Figure 1 As shown, the method includes:

[0056] Step 101: Based on the pre-trained performance evaluation model, perform performance evaluation on each device in the working cluster of the hybrid heterogeneous system and generate performance evaluation values; wherein, the performance evaluation values ​​characterize the comprehensive performance of the corresponding device under multiple types of tasks.

[0057] Here, a working cluster refers to a cluster of devices used to process multiple computing tasks based on heterogeneous computing power. The pre-trained performance evaluation model is used to generate a comprehensive performance evaluation value of the device under multiple types of tasks based on the domain-invariant features and specific domain features of each device. That is, it generates a performance evaluation value representing the comprehensive performance of the device under multiple types of tasks based on the first feature information (i.e., domain-invariant features) representing the hardware configuration and the second feature information (i.e., specific domain features) representing the performance under any task.

[0058] Understandably, pre-trained performance evaluation models have domain generalization capabilities. They can evaluate the overall performance of a device under various types of tasks based on the first feature information corresponding to the device's hardware configuration (which is relatively stable across different tasks) and the second feature information corresponding to the device's real-time performance data when performing a certain type of task (which varies with different tasks).

[0059] Step 102: Based on the performance evaluation values ​​of each device, divide all devices in the working cluster into a set number of device groups.

[0060] Here, all devices in the work cluster can be grouped based on the performance evaluation values ​​of each device in the work cluster, resulting in device groups that correspond one-to-one with multiple tasks to be processed. This facilitates unified management and scheduling of heterogeneous devices in the work cluster based on intelligent grouping.

[0061] Step 103: Assign the multiple tasks to be processed to each device group; wherein the set number is the same as the number of tasks, and each device group corresponds one-to-one with each task to be processed.

[0062] Here, the multiple tasks to be processed are paired with the divided device groups, so that each device group corresponds one-to-one with each task to be processed. This makes the computing power performance of the device groups corresponding to each task more balanced, which is conducive to improving the energy efficiency of the entire working cluster.

[0063] Step 104: Perform load balancing control on each task based on the device group.

[0064] It is understandable that by intelligently grouping heterogeneous devices based on performance evaluation values ​​and allocating multiple tasks to be processed to each device group, and then performing load balancing control on each task based on the device group, it is possible to achieve unified management and scheduling of heterogeneous devices in the work cluster based on intelligent grouping. This enables unified management and scheduling of heterogeneous devices, thereby improving the utilization rate of computing resources and overall energy efficiency of the hybrid heterogeneous system. The method of this application embodiment can adapt to the management needs of large-scale heterogeneous hardware, ensure the efficient operation of the hybrid heterogeneous system, and effectively reduce operation and maintenance costs.

[0065] For example, the performance of each device in a working cluster of a hybrid heterogeneous system is evaluated based on a pre-trained performance evaluation model, generating performance evaluation values, including:

[0066] For each device in the working cluster of a hybrid heterogeneous system, a feature vector of the device is constructed based on a first feature information characterizing the hardware configuration and a second feature information characterizing the performance under any task.

[0067] The feature vector of the device is input into a pre-trained performance evaluation model to obtain the performance evaluation value of the device.

[0068] Here, the first feature information can be obtained by converting the device's hardware configuration into a One-Hot vector. A One-Hot vector, also known as a one-bit valid encoding, is a method for converting categorical variables into a form that machine learning algorithms can understand. In One-Hot encoding, each categorical variable is assigned a unique binary bit to represent its value. If the variable's value is 1, the corresponding binary bit is 1; if the value is 0, the corresponding binary bit is 0. In One-Hot encoding, each variable is encoded only once, hence the term "one-bit valid." The hardware configuration information includes at least one of the following: CPU model, GPU type, memory size, storage type, network bandwidth, etc. Each categorical variable corresponding to this configuration information has a corresponding encoding bit, thus allowing the device's hardware configuration to be represented based on the One-Hot vector.

[0069] Here, the second feature information can be understood as the real-time performance data of the device when processing any task. For example, the real-time performance data includes at least one of the following: CPU utilization, GPU utilization, and memory utilization. The second feature information can be obtained by normalizing each real-time performance data. For example, based on minimum-maximum normalization, each real-time performance data can be scaled to the range of [0,1]. Specifically, the normalization can be performed using the following quantization formula:

[0070]

[0071] Where X represents real-time performance data, X min X is the minimum value of this real-time performance data. max X represents the maximum value of this real-time performance data. normalized This is the value obtained after normalizing the real-time performance data.

[0072] It is understandable that by concatenating the first feature information with the second feature information, the feature vector of the device can be obtained. This feature vector can be understood as the feature representation of the device and any task, including domain-invariant features (corresponding to the first feature information) and domain-specific features (corresponding to the second feature information).

[0073] It should be noted that the pre-trained performance evaluation model has domain generalization capabilities. It can evaluate the overall performance of a device across various task types based on a first feature information corresponding to the device's hardware configuration (which is relatively stable across different tasks) and a second feature information corresponding to the device's real-time performance data when performing a specific type of task (which varies with the task). Therefore, by inputting the device's feature vector into this pre-trained performance evaluation model, the device's performance evaluation value can be obtained. This pre-trained performance evaluation model can also be called a multi-dimensional performance evaluation system. The performance evaluation values ​​it generates comprehensively consider the heterogeneity of devices and the diversity of tasks, giving the performance evaluation values ​​the comprehensive characteristics of cross-task and multi-dimensional indicators, without relying on human experience, thus providing a reliable basis for subsequent device grouping.

[0074] Exemplarily, the method further includes:

[0075] The multilayer perceptron is trained based on the training sample set using a loss function to obtain the pre-trained performance evaluation model.

[0076] The training sample set includes a feature vector of the device, the true performance value of the device, and a domain label representing the task processed by the device. The loss function is determined based on a first loss function representing the regression task loss and a second loss function representing the domain classification loss. The first loss function is used to determine the loss value between the predicted performance evaluation value and the true performance value, and the second loss function is used to determine the loss value between the predicted task processed by the device and the domain label.

[0077] Here, the Multilayer Perceptron (MLP) is a feedforward artificial neural network composed of multiple neurons (neural nodes) arranged in a hierarchical structure, including an input layer, hidden layers, and an output layer. Neurons between layers are connected by weights, and information propagates sequentially from the input layer to the output layer without feedback connections. In regression tasks, the output layer directly uses the values ​​of the output nodes as the predicted values, i.e., to obtain the device's performance evaluation values. Furthermore, the neurons between layers are connected by weights, which are continuously adjusted and optimized during model training to enable the model to learn the mapping relationship between input data and the output target.

[0078] For example, the formula for the regression task of this multilayer perceptron can be expressed as follows:

[0079]

[0080] Among them, Q j Let h represent the weight matrix of the j-th layer of the multilayer perceptron, M be the total number of layers in the multilayer perceptron, and h be the weight matrix of the j-th layer. i Let y represent the feature vector of device i. i This represents the performance evaluation value of device i.

[0081] It is understandable that the feature vector h of device i is constructed based on the first feature information and the second feature information. i Then, the feature vector h i The input layer of the multilayer perceptron is fed into the hidden layers of the multilayer perceptron, where the weighted summation calculation is repeated. Finally, a sigmoid activation function is used for nonlinear mapping, mapping the output of each neuron to the interval [0-1], thus obtaining the predicted value y. i .

[0082] For example, during the training of a multilayer perceptron, a domain generalization mechanism is introduced. This mechanism allows the model to learn high-level abstract features in the hidden layers. These features can capture common information across domains, thereby enhancing the model's generalization ability. Specifically, this mechanism includes two parts: a gradient reversal layer and a domain classifier. The gradient reversal layer keeps the input unchanged during forward propagation and reverses the direction of the gradient during backward propagation, enabling the model to learn cross-domain features. The domain classifier is used to distinguish data from different domains (i.e., different task types) based on the extracted features. This is the opposite of the feature extraction objective, thus creating an adversarial relationship to eliminate differences in feature distribution between different domains.

[0083] For example, the formula for the backpropagation process can be expressed as follows:

[0084]

[0085] Where L is the loss value, λ represents the hyperparameter controlling the degree of gradient reversal, x represents the input feature vector of the gradient reversal layer, and y represents the output of the forward propagation of the gradient reversal layer. The gradient reversal layer includes forward propagation and backward propagation. Forward propagation is the process of generating feature vectors by the feature extractor (multilayer perceptron), and backward propagation is the process of reversing and scaling the gradient direction, optimizing the feature extractor through gradient reversal.

[0086] For example, the processing procedure of the domain classifier is shown in the following formula:

[0087] p = softmax(h′)

[0088] Here, softmax() is the domain classifier, and p is the output of the domain classifier, representing the probability that sample h′ belongs to a certain domain.

[0089] In one application example, the loss function of a multilayer perceptron is defined as follows:

[0090]

[0091] L total =L reg +λL dom

[0092] Where N is the number of samples in the training sample set. It should be noted that the same device may correspond to different samples under different tasks. i y is the true performance value of the i-th sample. i This represents the performance value predicted by the model, where C is the number of domains, i.e., the number of task types, and y is the performance value predicted by the model. ij p is the label (0 or 1) of the i-th sample belonging to the j-th domain (task). ij L is the probability predicted by the domain classifier that the i-th sample belongs to the j-th domain. reg It is the loss from the regression task, L dom It is a domain classification loss, where λ is a hyperparameter, and L total It is the total loss function.

[0093] The true performance value of the sample can be the average value after normalization of specific domain features such as CPU utilization, GPU utilization, and memory utilization.

[0094] Understandably, by introducing a domain generalization mechanism during model training, the multilayer perceptron can eliminate the distribution differences of heterogeneous devices under multiple tasks, enabling the model trained on the source domain to better adapt to the target domain and improve the model's generalization ability on the target domain.

[0095] Exemplarily, the method further includes:

[0096] Obtain historical operating data of each device in the work cluster, and construct a training sample set based on the historical operating data;

[0097] The pre-trained performance evaluation model is trained and updated based on the newly constructed training sample set.

[0098] Understandably, in practical applications, operational data from various devices can be continuously collected, and a training sample set can be built based on the collected operational data. The model can then be retrained or fine-tuned periodically to ensure that the performance evaluation model maintains optimal performance.

[0099] For example, the principle of generating performance evaluation values ​​for devices based on multilayer perceptrons is as follows: Figure 2 As shown in the diagram. Data preprocessing includes converting the device's hardware configuration information into a One-Hot vector (corresponding to the first feature information) and converting the device's real-time performance data into a normalized vector (corresponding to the second feature information). The One-Hot vector and the normalized vector are concatenated to obtain a feature representation vector (i.e., the aforementioned feature vector). This feature representation vector is then input into a multilayer perceptron, where it undergoes repeated weighted summation calculations through the hidden layers of the multilayer perceptron. Finally, a sigmoid activation function is used for nonlinear mapping to obtain the predicted value, which is the device's performance evaluation value.

[0100] For example, the division of all devices in the work cluster into a set number of device groups based on the performance evaluation values ​​of each device includes:

[0101] Sort all devices in the work cluster based on the performance evaluation value;

[0102] The sorted devices are added to each device group sequentially using a greedy algorithm until the difference in the sum of the performance evaluation values ​​between a set number of device groups is less than or equal to a set difference threshold.

[0103] Here, the performance evaluation value obtained by each device in the device cluster based on the performance evaluation model can be used as the weight value of the corresponding device. All devices are sorted according to their weight values ​​to obtain a device queue. For example, the devices are sorted in descending order of performance evaluation values. Devices are added to each group of the device queue using a greedy algorithm. The number of groups corresponds to the number of tasks to be processed. In order to achieve performance balance among device groups, the difference between the sum of performance evaluation values ​​(i.e., the sum of weight values) of each device group should be less than or equal to a set difference threshold. In this way, the intelligently grouped device groups can be used to perform load balancing control on multiple tasks during subsequent task execution.

[0104] For example, the load balancing control of each task based on the device group includes at least one of the following:

[0105] If the set dynamic adjustment conditions are met, then perform at least one of the following load balancing controls:

[0106] Based on load migration, a portion of the workload of tasks in a high-load device group is migrated to a low-load device group;

[0107] Based on the devices in the backup cluster, replace the faulty devices in each device group;

[0108] The work cluster can be elastically expanded and / or elastically reduced.

[0109] Here, load balancing among the device groups in the device cluster can be achieved through load migration, device replacement, and elastic scaling, thereby enabling dynamic adjustment of resource allocation in a hybrid heterogeneous system, improving resource utilization, and releasing overall efficiency.

[0110] For example, determining that the set dynamic adjustment conditions are met includes:

[0111] Determine that the load rate of at least one device group is greater than or equal to a set load rate threshold; and / or,

[0112] A fault is determined to exist based on the current performance values ​​of each device in the work cluster.

[0113] For example, if it is determined that the load rate of at least one device is greater than or equal to a set load rate threshold, a portion of the task load of the high-load device group can be migrated to the low-load device group. Specifically, on the high-load device group, the low-priority task load (e.g., non-real-time tasks) can be determined as the migration target based on a priority strategy, and the migration target can be migrated to the device group with the lowest current total load rate based on a greedy selection, thereby achieving load balancing among the device groups.

[0114] For example, if it is determined that a task fails, heartbeat is lost, or service is unavailable on a device node, it can be determined that the device is faulty. Then, based on the devices in the backup cluster, the faulty devices in each device group can be replaced, thereby achieving balanced distribution of the load cluster by dynamically adjusting the devices in the device cluster.

[0115] For example, if it is determined that the load rate of all device groups is greater than or equal to a first set load rate threshold, or less than a second set load rate threshold, then the work cluster can be elastically scaled up or down. It is understood that load migration cannot achieve overall performance optimization at this point. Therefore, elastic scaling up or down operations can be performed on the work cluster. For example, for devices with significantly degraded performance, such as those with a significant increase in node response time (latency increasing from 50ms to 500ms), elastic scaling up can be performed, or for devices with consistently low load, elastic scaling down can be performed to prevent some nodes from being idle while others are overloaded.

[0116] The present application will be further described in detail below with reference to application examples.

[0117] In one application embodiment, devices in a working cluster of a hybrid heterogeneous system are intelligently grouped, such as... Figure 3 , refer to Figure 3 The multi-dimensional performance evaluation subsystem, namely the aforementioned performance evaluation model, is based on which the performance evaluation of each device in the work cluster (i.e., Figure 3 Each device in the cluster performs performance evaluation based on its hardware configuration information (i.e., the first characteristic information) and real-time performance information (i.e., the second characteristic information), obtaining a performance evaluation score (i.e., a performance evaluation value). Each device's performance evaluation score is then used as its weight value, and the intelligent grouping subsystem performs intelligent grouping based on these weight values. (Refer to...) Figure 4 The load balancing method includes the following steps:

[0118] Step 401: Assign weights.

[0119] Here, based on Figure 3 The multi-dimensional performance evaluation subsystem shown assigns weights to devices in the work cluster. The performance evaluation score (within the [0,1] interval) output by the multi-dimensional performance evaluation subsystem is the weight value of each device.

[0120] Step 402: Initialize the device queue.

[0121] Here, the devices are sorted in descending order of performance evaluation scores to obtain the device queue.

[0122] Step 403: Use a greedy algorithm to generate device groups.

[0123] Here, the device with the highest performance evaluation score is taken from the device queue and assigned to the group with the lowest current total performance score (i.e., the sum of the performance evaluation scores of devices within the group), and the total performance score of that group is updated. Specifically, for each device i (starting from the highest performance), it is assigned to the group j with the lowest current total performance score, and the total performance score of group j is updated: S[j] += p[i], where S[j] is the total performance score of group j, and p[i] is the performance evaluation score of device i. The above steps are repeated until all devices have been assigned. After each assignment, it is checked whether the total performance score of each device group meets the condition. If the condition is not met, the devices are reassigned. The condition for the total performance score of each device group to meet the condition is that the difference between the total performance scores of the device groups is less than or equal to a set difference threshold. For example, the difference between the total performance scores does not exceed 0.2. If not, the device groups can be recursively checked to optimize the device grouping scheme until a grouping scheme that meets the condition is obtained.

[0124] Step 404: Evaluate the grouping scheme and check the load and performance indicators of each device group.

[0125] Here, the load rate of each device group and the performance indicators of each device (i.e., node) in the device group can be obtained based on the device group.

[0126] Step 405: Dynamically adjust to ensure system load balancing and efficient operation.

[0127] Here, load balancing control can be performed based on the load rate of each device group, or device replacement or elastic scaling can be performed based on node performance indicators. This enables unified management of functions such as device grouping and performance monitoring, improving resource allocation efficiency and monitoring capabilities, and adapting to complex and ever-changing workload demands. Through intelligent grouping and monitoring, the potential of heterogeneous computing is unleashed, meeting the resource optimization needs of data center and cloud computing environments.

[0128] In order to implement the method of the embodiments of this application, the embodiments of this application also provide a load balancing device for hybrid heterogeneous systems. The load balancing device for hybrid heterogeneous systems corresponds to the load balancing method for hybrid heterogeneous systems described above. The steps in the embodiments of the load balancing method for hybrid heterogeneous systems described above are also fully applicable to the embodiments of the load balancing device for hybrid heterogeneous systems described above.

[0129] like Figure 5As shown, the load balancing device for hybrid heterogeneous systems includes: a performance evaluation module 501, a grouping module 502, a task allocation module 503, and a control module 504. The performance evaluation module 501 evaluates the performance of each device in the working cluster of the hybrid heterogeneous system based on a pre-trained performance evaluation model, generating performance evaluation values; wherein, the performance evaluation values ​​characterize the comprehensive performance of the corresponding device under multiple types of tasks; the grouping module 502 divides all devices in the working cluster into a set number of device groups based on the performance evaluation values ​​of each device; the task allocation module 503 allocates multiple tasks to be processed to each device group; wherein, the set number is the same as the number of tasks, and each device group corresponds one-to-one with each task to be processed; the control module 504 performs load balancing control on each task based on the device groups.

[0130] For example, the control module 504 is specifically used for:

[0131] If the set dynamic adjustment conditions are met, then perform at least one of the following load balancing controls:

[0132] Based on load migration, a portion of the workload of tasks in a high-load device group is migrated to a low-load device group;

[0133] Based on the devices in the backup cluster, replace the faulty devices in each device group;

[0134] The work cluster can be elastically expanded and / or elastically reduced.

[0135] For example, the control module 504 determines that the set dynamic adjustment conditions are met, including:

[0136] Determine that the load rate of at least one device group is greater than or equal to a set load rate threshold; and / or,

[0137] A fault is determined to exist based on the current performance values ​​of each device in the work cluster.

[0138] For example, the performance evaluation module 501 is specifically used for:

[0139] For each device in the working cluster of a hybrid heterogeneous system, a feature vector of the device is constructed based on a first feature information characterizing the hardware configuration and a second feature information characterizing the performance under any task.

[0140] The feature vector of the device is input into a pre-trained performance evaluation model to obtain the performance evaluation value of the device.

[0141] For example, the load balancing device for hybrid heterogeneous systems further includes: a training module 505, used to train the multilayer perceptron based on a loss function using a training sample set to obtain the pre-trained performance evaluation model;

[0142] The training sample set includes a feature vector of the device, the true performance value of the device, and a domain label representing the task processed by the device. The loss function is determined based on a first loss function representing the regression task loss and a second loss function representing the domain classification loss. The first loss function is used to determine the loss value between the predicted performance evaluation value and the true performance value, and the second loss function is used to determine the loss value between the predicted task processed by the device and the domain label.

[0143] For example, the training module 505 is also used for:

[0144] Obtain historical operating data of each device in the work cluster, and construct a training sample set based on the historical operating data;

[0145] The pre-trained performance evaluation model is trained and updated based on the newly constructed training sample set.

[0146] For example, the grouping module 502 is specifically used for:

[0147] Sort all devices in the work cluster based on the performance evaluation value;

[0148] The sorted devices are added to each device group sequentially using a greedy algorithm until the difference in the sum of the performance evaluation values ​​between a set number of device groups is less than or equal to a set difference threshold.

[0149] In practical applications, the performance evaluation module 501, grouping module 502, task allocation module 503, control module 504, and training module 505 can be implemented by a processor in an electronic device. Of course, the processor needs to run a computer program in memory to perform its functions.

[0150] It should be noted that the load balancing device for hybrid heterogeneous systems provided in the above embodiments is only illustrated by the division of the above-described program modules when performing load balancing for hybrid heterogeneous systems. In practical applications, the above processing can be distributed to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the load balancing device for hybrid heterogeneous systems provided in the above embodiments and the load balancing method embodiments for hybrid heterogeneous systems belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0151] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiments of this application, the embodiments of this application also provide an electronic device. Figure 6 The diagram shows only an exemplary structure of the electronic device, not the entire structure; implementation is possible as needed. Figure 6 The structure shown may be part or all of the structure.

[0152] like Figure 6 As shown, the electronic device 600 provided in this application embodiment includes: at least one processor 601, a memory 602, a user interface 603, and at least one network interface 604. The various components in the electronic device 600 are coupled together via a bus system 605. It can be understood that the bus system 605 is used to implement communication between these components. In addition to a data bus, the bus system 605 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 6 The general designated all buses as Bus System 605.

[0153] The user interface 603 may include a monitor, keyboard, mouse, trackball, click wheel, buttons, touchpad, or touch screen.

[0154] The memory 602 in this embodiment is used to store various types of data to support the operation of the electronic device. Examples of such data include any computer program used to operate on the electronic device.

[0155] The load balancing method for hybrid heterogeneous systems disclosed in this application can be applied to or implemented by processor 601. Processor 601 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the load balancing method for hybrid heterogeneous systems can be completed by integrated logic circuits in the hardware of processor 601 or by instructions in software form. The processor 601 can be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 601 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules can be located in a storage medium, specifically memory 602. Processor 601 reads information from memory 602 and, in conjunction with its hardware, completes the steps of the load balancing method for hybrid heterogeneous systems provided in the embodiments of this application.

[0156] In an exemplary embodiment, the electronic device may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0157] It is understood that memory 602 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Sync Link Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable types of memories.

[0158] In an exemplary embodiment, this application also provides a computer storage medium, specifically a computer-readable storage medium, such as a memory 602 storing a computer program, which can be executed by a processor 601 of an electronic device to complete the steps described in the method of this application embodiment. The computer-readable storage medium can be a ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM, etc.

[0159] In an exemplary embodiment, this application also provides a computer program product, including a computer program that can be executed by a processor 601 of an electronic device 600 to perform the steps described in the method of this application embodiment.

[0160] It should be noted that terms such as "first" and "second" are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0161] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0162] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A load balancing method for hybrid heterogeneous systems, characterized in that, include: The performance of each device in the working cluster of a hybrid heterogeneous system is evaluated based on a pre-trained performance evaluation model, and performance evaluation values ​​are generated; wherein, the performance evaluation values ​​represent the comprehensive performance of the corresponding device under multiple types of tasks. Based on the performance evaluation values ​​of each device, all devices in the working cluster are divided into a set number of device groups; Multiple tasks to be processed are assigned to various device groups; wherein, the set number is the same as the number of tasks, and each device group corresponds one-to-one with each task to be processed; Load balancing control is performed on each task based on the aforementioned equipment group.

2. The method according to claim 1, characterized in that, The load balancing control of each task based on the device group includes at least one of the following: If the set dynamic adjustment conditions are met, then perform at least one of the following load balancing controls: Based on load migration, a portion of the workload of tasks in a high-load device group is migrated to a low-load device group; Based on the devices in the backup cluster, replace the faulty devices in each device group; The work cluster can be elastically expanded and / or elastically reduced.

3. The method according to claim 2, characterized in that, The determination that the set dynamic adjustment conditions are met includes: Determine that the load rate of at least one device group is greater than or equal to a set load rate threshold; and / or, A fault is determined to exist based on the current performance values ​​of each device in the work cluster.

4. The method according to claim 1, characterized in that, The performance of each device in a working cluster of a hybrid heterogeneous system is evaluated based on a pre-trained performance evaluation model, generating performance evaluation values, including: For each device in the working cluster of a hybrid heterogeneous system, a feature vector of the device is constructed based on a first feature information characterizing the hardware configuration and a second feature information characterizing the performance under any task. The feature vector of the device is input into a pre-trained performance evaluation model to obtain the performance evaluation value of the device.

5. The method according to claim 4, characterized in that, The method further includes: The multilayer perceptron is trained based on the training sample set using a loss function to obtain the pre-trained performance evaluation model. The training sample set includes a feature vector of the device, the true performance value of the device, and a domain label representing the task processed by the device. The loss function is determined based on a first loss function representing the regression task loss and a second loss function representing the domain classification loss. The first loss function is used to determine the loss value between the predicted performance evaluation value and the true performance value, and the second loss function is used to determine the loss value between the predicted task processed by the device and the domain label.

6. The method according to claim 5, characterized in that, The method further includes: Obtain historical operating data of each device in the work cluster, and construct a training sample set based on the historical operating data; The pre-trained performance evaluation model is trained and updated based on the newly constructed training sample set.

7. The method according to claim 1, characterized in that, The performance evaluation values ​​of each device are used to divide all devices in the working cluster into a set number of device groups, including: Sort all devices in the work cluster based on the performance evaluation value; The sorted devices are added to each device group sequentially using a greedy algorithm until the difference in the sum of the performance evaluation values ​​between a set number of device groups is less than or equal to a set difference threshold.

8. A load balancing device for hybrid heterogeneous systems, characterized in that, include: The performance evaluation module is used to evaluate the performance of each device in the working cluster of a hybrid heterogeneous system based on a pre-trained performance evaluation model and generate performance evaluation values; wherein, the performance evaluation values ​​represent the comprehensive performance of the corresponding device under multiple types of tasks; A grouping module is used to divide all devices in the working cluster into a set number of device groups based on the performance evaluation value of each device. The task allocation module is used to allocate multiple tasks to be processed to various device groups; wherein, the set number is the same as the number of tasks, and each device group corresponds one-to-one with each task to be processed; The control module is used to perform load balancing control on each task based on the device group.

9. An electronic device, characterized in that, include: A processor and memory for storing computer programs that can run on the processor, wherein, The processor, when running a computer program, performs the steps of the method according to any one of claims 1 to 7.

10. A computer storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images