Load balancing method and system of storage and calculation integrated hyper-converged server

Abstract

The invention provides a load balancing method and system for a storage and calculation integrated hyper-converged server, and the method comprises the steps: screening an overload container and an idle container when a container resource pool gives an overload alarm; obtaining overload process information of the overload container, and determining a plurality of transfer containers from the idle containers according to the overload process information; detecting whether fusion parameters are stored among the plurality of transfer containers, if the fusion parameters are not stored, acquiring node associated parameters from the overload containers, and generating the fusion parameters according to the node associated parameters; starting a plurality of mirror image working processes in the transfer container according to the fusion parameters and the overload process information; and performing correct rate evaluation on the mirror image working process, and determining a target working process which is used for taking over the released working process of the overload container.

Description

Technical Field

[0001] The present application relates to the field of server technology, and in particular to a load balancing method and system for a storage-computing integrated hyper-converged server. Background Art

[0002] Hyperconverged servers provide efficient data processing capabilities by integrating computing, storage, and network resources. However, in a multi-tenant or multi-tasking environment, when the workload on some nodes suddenly increases to exceed their processing capacity, node overload occurs. At this time, in order to maintain the stability and performance of the overall system, resource scheduling operations are usually required, such as migrating virtual machines or adjusting resource allocation to different tasks. Although this dynamic resource management helps balance the load of the entire cluster, it may cause a series of problems for certain types of tasks, especially neural network training, which runs for a long time and requires high computing resources. Summary of the invention

[0003] The present application provides a load balancing method and system for a storage-computing hyper-converged server, which are used to reduce the adverse effects on neural network training when performing resource scheduling in the storage-computing hyper-converged server.

[0004] In a first aspect, an embodiment of the present application provides a load balancing method for a storage-computing integrated hyper-converged server, the method comprising: When an overload alarm occurs in the container resource pool, overloaded containers and idle containers are screened; Acquiring overload progress information of the overloaded container, and determining a plurality of transfer containers from the idle containers according to the overload progress information; Detecting whether fusion parameters are stored between the plurality of transfer containers, and if the fusion parameters are not stored, acquiring node association parameters from the overload container, and generating the fusion parameters according to the node association parameters; Starting a plurality of mirror work processes in the transfer container according to the fusion parameter and the overload process information; The correctness of the mirror work process is evaluated to determine a target work process, where the target work process is used to take over the released work process of the overloaded container.

[0005] In a second aspect, an embodiment of the present application provides a load balancing system for a storage-computing integrated hyper-converged server, the device comprising: The load warning module is used to screen overloaded containers and idle containers when an overload alarm occurs in the container resource pool; a container determination module, configured to obtain overloading process information of the overloaded container, and determine a plurality of transfer containers from the idle containers according to the overloading process information; a parameter determination module, configured to detect whether fusion parameters are stored between the plurality of transfer containers, and if the fusion parameters are not stored, to obtain node association parameters from the overload container, and to generate the fusion parameters according to the node association parameters; A process mirroring module, used for starting a plurality of mirroring working processes in the transfer container according to the fusion parameter and the overloaded process information; The process transfer module is used to evaluate the accuracy of the mirror work process and determine the target work process, where the target work process is used to take over the released work process of the overloaded container.

[0006] In a third aspect, an embodiment of the present application provides a storage-computing integrated hyper-converged server, wherein the storage-computing integrated hyper-converged server includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the load balancing method of the storage and computing integrated hyper-converged server as described in any one of the embodiments of the present application when executing the computer program.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements a load balancing method for a storage-computing hyper-converged server as described in any one of the embodiments of the present application.

[0008] An embodiment of the present application provides a load balancing method for a storage-computing integrated hyper-converged server, the method comprising: when an overload alarm occurs in a container resource pool, screening overloaded containers and idle containers; obtaining overload process information of the overloaded container, and determining multiple transfer containers from the idle containers according to the overload process information; detecting whether fusion parameters are stored between the multiple transfer containers, and if no fusion parameters are stored, obtaining node association parameters from the overloaded container, and generating fusion parameters according to the node association parameters; starting multiple mirror work processes in the transfer container according to the fusion parameters and the overload process information; evaluating the accuracy of the mirror work process, and determining a target work process, the target work process being used to take over the released work process of the overloaded container. Through the above method, when a container is overloaded, multiple transfer containers are determined from the idle containers according to the overload process information of the overloaded container, and a transfer adaptation is performed. The adaptability of the transfer container is judged according to the fusion parameters. If the transfer containers are not adapted, they need to be adapted according to the node association parameters of the overloaded container, and the fusion parameters are generated. According to the determined transfer container, multiple mirror work processes corresponding to the overload process information are started, and then a second transfer adaptation is performed. By evaluating the accuracy of the mirror work process, three transfer adaptations are completed. Through the above multiple transfer adaptations, the impact on the neural network training is reduced as much as possible to prevent the occurrence of hallucination problems during the neural network training process. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0010] Figure 1 A schematic flow chart of a load balancing method for a storage-computing integrated hyper-converged server provided in an embodiment of the present application; Figure 2 A schematic block diagram of a load balancing system for a storage-computing hyper-converged server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0011] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0012] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0013] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0014] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0015] Hyper-Converged Infrastructure (HCI) is an IT infrastructure solution that integrates computing, storage, network resources, and software, usually in a single device or a tightly integrated group of devices. By merging these traditional data center components into a shared resource pool, hyper-converged infrastructure simplifies management and deployment, improves efficiency, and enables faster response to changes in business needs.

[0016] It should be noted that the "hyper" in hyper-convergence means "Hyper" rather than "Super". It specifically refers to the software-defined technology architecture, highlighting the meaning of software definition, including software-defined computing (Hypervisor), software-defined storage (Server SAN) and software-defined network (NFV: Network Function Virtualization).

[0017] However, although the storage and computing hyper-converged server has many advantages, when the neural network training task needs to be transferred, the impact of the transfer still needs to be considered, especially when mixing hardware and software from different manufacturers.

[0018] See also Figure 1 , Figure 1 The embodiment of the present application also provides a schematic flow chart of a load balancing method for a storage-computing integrated hyper-converged server. Figure 1 As shown, the load balancing method for the storage and computing integrated hyper-converged server provided in the embodiment of the present application specifically includes the steps of: S101-S105.

[0019] S101. When an overload alarm occurs in a container resource pool, overloaded containers and idle containers are screened.

[0020] Exemplarily, a storage-computing hyper-converged server supports multiple types of nodes, such as converged nodes (computing + block storage), computing nodes (computing), and storage nodes (block, file, and object storage). The CPU, memory, and hard disk capacity of the converged nodes are balanced. The computing nodes mainly focus on the CPU and memory (for example, 4U 4-way integrated machines), and the storage nodes are mainly high-density (48-disk) integrated storage machines. Different containers are formed according to different types, and multiple containers are managed under a unified platform. A single container is compatible with multiple types of nodes and managed in a unified manner.

[0021] In addition, considering the limited resources, for example, in a single container, there are different requirements for computing and storage resources at the same time, and it is necessary to focus on one resource. In this case, a single container is required to manage two or more types of nodes at the same time, such as half of the nodes are computing nodes and half of the nodes are fusion nodes. The computing resources in this container are obviously relatively abundant and can carry high computing requirements, so it can be given priority as an idle container.

[0022] During the operation and maintenance of a storage and computing hyper-converged server, it is necessary to monitor the load of each container and issue warnings for container overload conditions, which include: excessively high hardware temperature, software lag, and abnormal workload.

[0023] The container with the above overload situation is set as the overload container, and the container with hardware resources or software resources greater than the preset resource threshold is set as the pre-selected container. Then, the idle container is determined from the pre-selected container according to the node type or hardware configuration of the overloaded container, so that the overloaded container and the idle container can be highly adapted to the hardware as much as possible.

[0024] S102: Obtain overload process information of the overloaded container, and determine a plurality of transfer containers from the idle containers according to the overload process information.

[0025] For example, in a container, a node undertakes the running tasks of a process. When the task volume of the process is greater than the capacity of the node, resulting in node lag and insufficient resources, the process can be set as an overloaded process.

[0026] In other embodiments, the overloaded node may be determined by whether the training progress of the neural network meets the standard, and the process whose training progress does not meet the standard may be set as an overloaded process.

[0027] After the overloaded process is determined, the overloaded process information corresponding to the overloaded process is obtained, including: node hardware configuration, neural network training task type corresponding to the process, and association between processes.

[0028] When a single idle container is insufficient to handle an overloaded process, multiple transfer containers need to be determined from the idle containers. Specifically, the idle containers are matched according to the hardware adaptation information, including the neural network type and the corresponding hardware adaptation information in the overloaded process information, and the successfully matched ones are used as transfer containers.

[0029] S103, detecting whether fusion parameters are stored between the multiple transfer containers, if no fusion parameters are stored, obtaining node association parameters from the overload container, and generating fusion parameters according to the node association parameters.

[0030] Exemplarily, after a plurality of transfer containers are determined, it is necessary to consider whether the plurality of transfer containers can be merged to form a virtual large container.

[0031] Generally, a storage-computing hyper-converged server is equipped with redundant containers, which are often used to undertake overloaded processes. Therefore, after each load is completed, the fusion parameters are recorded and can be used directly when the same problem is encountered again in the future, such as the same neural network training progress not meeting the standard. When new problems arise or new transfer containers are introduced, the fusion parameters need to be rebuilt.

[0032] In the process of reconstructing the fusion parameters, it is necessary to open the node association parameters of the overload container, determine the task types and association relationships of multiple transfer containers according to the node types and node association relationships in the node association parameters, and establish the fusion parameters.

[0033] S104: Start multiple mirroring work processes in the transfer container according to the fusion parameters and the overloaded process information.

[0034] Exemplarily, after obtaining the fusion parameters, the work process is rebuilt in the transfer container, that is, multiple mirror work processes are started. These mirror work processes are used to take over the overloaded process. It should be noted that the number of mirror work processes should be greater than the number of corresponding overloaded processes, so as to facilitate secondary hardware selection and prevent the influence of hardware matching problems on neural network training.

[0035] S105: Evaluate the accuracy of the mirrored working process and determine a target working process, where the target working process is used to take over the released working process of the overloaded container.

[0036] Exemplarily, after starting the mirror working process, the overload process still needs to be continued to run. The first training parameter of the overload process is recorded, and the second training parameter of the mirror working process is recorded. The target working process is determined from the mirror working process through the similarity between the first training parameter and the second training parameter, for example, the similarity threshold is 98%, which is conducive to reducing the interference of hardware matching problems.

[0037] An embodiment of the present application provides a load balancing method for a storage-computing integrated hyper-converged server, the method comprising: when an overload alarm occurs in a container resource pool, screening overloaded containers and idle containers; obtaining overload process information of the overloaded container, and determining multiple transfer containers from the idle containers according to the overload process information; detecting whether fusion parameters are stored between the multiple transfer containers, and if no fusion parameters are stored, obtaining node association parameters from the overloaded container, and generating fusion parameters according to the node association parameters; starting multiple mirror work processes in the transfer container according to the fusion parameters and the overload process information; evaluating the accuracy of the mirror work process, and determining a target work process, the target work process being used to take over the released work process of the overloaded container. Through the above method, when a container is overloaded, multiple transfer containers are determined from the idle containers according to the overload process information of the overloaded container, and a transfer adaptation is performed. The adaptability of the transfer container is judged according to the fusion parameters. If the transfer containers are not adapted, they need to be adapted according to the node association parameters of the overloaded container, and the fusion parameters are generated. According to the determined transfer container, multiple mirror work processes corresponding to the overload process information are started, and then a second transfer adaptation is performed. By evaluating the accuracy of the mirror work process, three transfer adaptations are completed. Through the above multiple transfer adaptations, the impact on the neural network training is reduced as much as possible to prevent the occurrence of hallucination problems during the neural network training process.

[0038] In order to more clearly introduce the technical solution of the present application, the technical solution of the present application will be introduced through specific embodiments below. It should be noted that the specific embodiments are used to expand the technical solution of the present application, but are not intended to limit the present application.

[0039] In some embodiments, overloading process information of the overloaded container is obtained, and multiple transfer containers are determined from the idle containers according to the overloading process information. The specific steps include: S1021-S1023.

[0040] S1021. Acquire first hardware configuration information and first software configuration information of an overloaded container, and acquire second hardware configuration information and second software configuration information of an idle container.

[0041] Exemplarily, the first hardware configuration information includes: CPU architecture and number of cores, memory capacity and speed, storage performance, and network bandwidth and latency. The first software configuration information includes: framework version and environment settings, model structure and parameters, optimizer configuration, data preprocessing and enhancement strategy, and batch size and epoch number.

[0042] Framework version and environment setup: Confirm the deep learning framework (such as TensorFlow, PyTorch) used and its version number, because there may be API changes or performance differences between different versions. Check the version of Python or other programming languages, as well as other necessary dependent libraries, to ensure that all packages are compatible.

[0043] Model structure and parameters: Collect specific architectural details about the neural network being trained, including the number of layers, number of nodes per layer, activation function type, etc. Record key training statistics such as the current number of iterations, loss value, accuracy, etc., so that the training process can be seamlessly continued after migration.

[0044] Optimizer configuration: Get the optimization algorithm used (e.g. Adam, SGD) and its hyperparameter settings. This information is critical to maintain the same gradient update rule.

[0045] Data preprocessing and enhancement strategy: Describe the standardization method, cropping method, flipping, rotation and other enhancement techniques of input data to ensure the consistency of data before and after migration.

[0046] Batch size and number of epochs: Knowing how many samples each mini-batch contains and how many epochs the entire training plan is expected to complete will help plan the training progress after migration.

[0047] It should be noted that the content of the second hardware configuration information is the same as that of the first hardware configuration information, and the content of the second software configuration information is the same as that of the first software configuration information in the non-operating environment, for example, the adapted framework version and environment settings.

[0048] S1022. Determine overload process information according to the first hardware configuration information and the first software configuration information.

[0049] Exemplarily, based on the first hardware configuration information and the first software configuration information, an overloaded node is determined, and then a process on the overloaded node is set as an overloaded process, and the overloaded process is matched.

[0050] S1023: Determine a plurality of transfer containers from the idle containers according to the overload process information, the second hardware configuration information, and the second software configuration information.

[0051] Exemplarily, after determining the overloaded process information, it is necessary to select a container with a higher degree of adaptability from the idle containers according to the overloaded process information, including hardware adaptation and software adaptation. Different neural network models have different training effects on different hardware. Therefore, when transferring tasks, it is necessary to focus on this issue and try to keep the hardware type of the overloaded container and the corresponding transfer container the same, and the same is true for the software configuration. Therefore, it is necessary to select the idle container based on the overloaded process information, the second hardware configuration information, and the second software configuration information to determine the transfer container.

[0052] Through the above process, it is ensured that the hardware environment and software environment supporting the training of the neural network model are basically the same before and after the transfer, so as to avoid unsatisfactory training of the neural network training model caused by deviation in the training environment.

[0053] In some embodiments, determining the overload process information according to the first hardware configuration information and the first software configuration information includes: S201-S206.

[0054] S201. Determine a neural network type and a training instruction set of an overload container according to first software configuration information, where the training instruction set is compiled and generated according to the neural network type and a preset neural network.

[0055] Specifically, the first software configuration information includes an architecture definition file of a neural network model, and the neural network model of the neural network model corresponding to the first software configuration information is determined by disassembling the architecture definition file, and the preset neural network is the neural network model.

[0056] By parsing the model architecture definition file, we can identify the basic types of neural networks (such as convolutional neural network CNN, recurrent neural network RNN, transformer Transformer, etc.), as well as their specific layer structures (such as fully connected layer, convolutional layer, pooling layer, etc.).

[0057] Using the preset compilation tool chain, according to the determined neural network type and existing configuration information, the high-level training code is converted into a low-level operation sequence, namely the training instruction set. These instruction sets include but are not limited to the specific implementation of forward propagation, backpropagation, weight update and other operations.

[0058] S202. Determine multiple current processes and process architecture parameters according to the first hardware configuration information and the neural network type.

[0059] For example, different types of neural networks have different preferences for hardware resources. For example, some models may be more suitable for GPU acceleration, while other models can run efficiently on CPU. In addition, process architecture parameters determine the way and efficiency of task execution.

[0060] The current process is the process of training the neural network, including: single-threaded, multi-threaded, distributed training, etc.

[0061] The task allocation strategy involved in each current process (such as data parallelism, model parallelism, etc.) is also part of the process architecture parameters. The process architecture parameters also include configuring the necessary parameters for each selected process, such as batch size, number of gradient accumulations, communication mechanism, etc.

[0062] S203, compile the training instruction set into an event queue.

[0063] Each operation in the training instruction set is converted into an independent event and arranged in a logical order to form a clear event queue. Taking into account resource availability and dependencies, the priority of each event in the event queue is adjusted to avoid unnecessary waiting time or conflicts.

[0064] S204: Determine a target process for executing each event in the event queue from the current process according to the event queue and process architecture parameters.

[0065] Based on the resources required for each event in the event queue (such as CPU / GPU cores, memory space, etc.), combined with the process architecture parameters, the target process that is most suitable for executing the event is selected to ensure that the load between all target processes is distributed as evenly as possible to prevent a process from being overloaded or idle.

[0066] Each event may require different computing resources to execute, and proper allocation of these resources can improve overall efficiency.

[0067] S205. Determine the timing cycle information and running power consumption information required for the target process to execute the corresponding event according to a preset timing database and a preset power consumption database, wherein the timing database and the power consumption database are obtained by simulating according to preset types of events and process architecture parameters.

[0068] Exemplarily, access the pre-built timing database and power consumption database, which contain simulation results of different types of events under various hardware conditions. According to the event type and the current hardware configuration, find the corresponding timing cycle information (such as the time required to complete a matrix multiplication) and running power consumption information (such as the energy units consumed per second).

[0069] S206: Determine overload process information according to the timing cycle information and the operation power consumption information.

[0070] For example, the timing cycle and power consumption information are used to evaluate the actual execution of each event in each process to determine whether there is an overload phenomenon. If it is found that some processes are indeed overloaded, a detailed migration or resource reallocation plan is formulated to ensure that the ongoing neural network training task is not affected.

[0071] In some embodiments, multiple transfer containers are determined from the idle containers according to the overload process information, the second hardware configuration information, and the second software configuration information, including: S1021-S1023.

[0072] S1021. Determine the running bearing information of the idle container according to the second hardware configuration information and the second software configuration information.

[0073] For example, check the hardware configuration of idle container A: it is equipped with 2 GPUs (NVIDIA V100), 64GB memory, 500GB SSD storage, and Gigabit Ethernet connection. Check the software configuration of idle container A: TensorFlow2.4.0, CUDA 11.0 and related dependent libraries have been installed, and the environment variables are set correctly. Based on the above hardware and software configurations, it can be inferred that idle container A can support more complex deep learning model training tasks, especially those that rely on GPU acceleration.

[0074] S1022: Determine the transfer adaptability of the idle container according to the running load information and the overloaded process information.

[0075] S1023. Determine a transfer container from the idle containers according to the transfer adaptability.

[0076] For example, when finally selecting a transfer container, in addition to considering the adaptability, factors such as network latency and geographic location need to be considered comprehensively to ensure that the migrated workload can run efficiently and stably. Assume that there are three idle containers (A, B, and C) in the cluster, and they have all been evaluated in the first two steps.

[0077] Container A: has high transfer adaptability, but is located on a different physical rack, which may increase network transmission latency.

[0078] Container B: The transfer adaptability is moderate, but the network latency is extremely low within the same physical rack.

[0079] Container C: Low transfer fitness, although in the same rack, not suitable for selection due to other reasons.

[0080] Finally, we decided to choose container B as the transfer container because although it is not the most adaptable, it combines good network conditions and sufficient hardware resources to provide better overall performance.

[0081] By considering factors at the hardware and software levels, we systematically evaluate and select the most suitable idle container to take over the overloaded process, achieving more intelligent and efficient resource scheduling.

[0082] In some embodiments, generating fusion parameters according to node association parameters includes: S1031-S1033.

[0083] S1031. Determine a transfer node and corresponding first node function information according to node association parameters.

[0084] For example, in a hyper-converged server environment, there may be multiple associations between different nodes, such as data dependency, network connection, etc. These association parameters are crucial to understanding the roles of each node and the interactions between them.

[0085] Node-related parameters may include, but are not limited to, the frequency of communication between nodes, the amount of shared data, dependent services or applications, etc. Based on the current system status (e.g., a node is overloaded with resources), identify the node where the workload that needs to be migrated is located as the transfer node. Obtain the functional characteristics of the transfer node, such as the type of application it runs (e.g., neural network training), the required computing resources (CPU / GPU), memory usage, storage requirements, and network bandwidth, etc.

[0086] S1032: Generate a node topology network according to the node transfer network and the first node function information.

[0087] Exemplarily, the node topology network depicts the structure of the entire system, including the various nodes and their relationships. Constructing the node topology network can find the optimal migration target.

[0088] Specifically, using known node association parameters, a graph is drawn that reflects the logical connections between all nodes. The functional information of the transferred nodes is embedded into this graph to identify which nodes are suitable for receiving specific types of tasks. Considering factors such as network latency and bandwidth, the topology structure is adjusted to ensure the optimal performance of the system after migration.

[0089] S1033. Determine fusion parameters according to the node topology network.

[0090] For example, the convergence parameters refer to a series of configurations and settings necessary to ensure the consistency and stability of the new and old environments after the migration is completed. This step ensures that the migration is not only a physical move, but also a logically seamless process.

[0091] Specifically, analyze the impact on the overall system before and after the migration, especially the changes in network traffic, resource utilization, etc. According to the node topology network, formulate a set of detailed fusion parameters, including but not limited to software environment consistency, hardware resource allocation strategy, network configuration, etc. Gradually adjust the system according to the fusion parameters to ensure that the migrated workload can smoothly transition to the new environment and maintain the original performance and service level.

[0092] In one example, in a hyperconverged cluster, node N1 is overloaded because it is running a large-scale image classification task. Analyze the node association parameters and find that node N1 has frequent data exchanges with nodes N2 and N3, mainly transferring training data sets. Determine node N1 as a migration node and collect its functional information, such as it is running the TensorFlow framework for ResNet50 model training, which occupies a large amount of GPU resources. Build a more comprehensive node topology network, adding information about other idle containers, such as node N4 (with strong GPU support) and node N5 (with sufficient memory and fast disk I / O). Combined with the functional information of node N1, mark which nodes are most suitable for image classification tasks, such as node N4 is the preferred choice due to its strong GPU capabilities. Optimize the topology network and further confirm that node N4 is the optimal migration target considering the low-latency connection between node N4 and existing data sources. Predict network congestion or other problems that may occur during the migration process and prepare corresponding mitigation measures. Ensure that the deep learning framework version on node N4 is the same as that on node N1, and adjust the number of GPU cores, memory allocation, etc. to match the original task requirements. First, synchronize the necessary files and data, and then gradually migrate the training tasks from node N1 to node N4, while monitoring various indicators to ensure a smooth transition.

[0093] Through the above method, not only the physical and logical relationships between nodes are considered, but also full attention is paid to the workload compatibility and system stability after migration, thereby achieving efficient and reliable resource scheduling.

[0094] In some embodiments, based on the fusion parameters and the overloaded process information, multiple mirroring work processes are started in the transfer container, including: S1041-S1044.

[0095] S1041. Obtain the second node function information and work cycle information of the transfer container.

[0096] For example, before preparing for migration, the functional characteristics and working mode of the target transfer container must be understood in detail to ensure that the requirements of the original task can be supported in the new environment.

[0097] Specifically, the second node function information is collected, including hardware resources (such as the number of CPU / GPU cores, memory size, storage type, etc.) and software environment (such as operating system version, installed libraries and frameworks, environment variable settings, etc.). Work cycle information is obtained by collecting the workload pattern of the target container, such as the time distribution of peak and off-peak periods, and possible regular maintenance or update windows.

[0098] S1042. Generate a process transfer network according to the fusion parameters and the overloaded process information.

[0099] Exemplarily, the process transfer network is used to plan the specific migration path from the overloaded node to the transfer container, ensuring that the migration process is efficient and has minimal impact on existing services. During the execution process, review the previously defined fusion parameters, especially the requirements on hardware resource allocation, network configuration, etc. Integrate the overloaded process information based on the task type (such as neural network training), required computing resources, dependent services, etc. involved in the overloaded process. Draw a map reflecting all relevant nodes and their connection relationships, identify the best migration path and key nodes, and build a process transfer network.

[0100] S1043. Determine a receiving node and a corresponding target transfer process from the nodes of the transfer container according to the process transfer network, the second node function information and the work cycle information.

[0101] Exemplarily, based on the process transfer network and the second node function information, evaluate which nodes are most suitable for receiving a specific type of overloaded process and evaluate node adaptability. Combined with the work cycle information, select nodes with low load and sufficient resources in the future as receiving nodes. Clarify which overloaded processes will be migrated to the selected receiving nodes, and formulate a detailed transfer plan for each process.

[0102] S1044. Start the mirror work process corresponding to the target transfer process in the receiving node.

[0103] For example, starting the mirroring work process means rebuilding the original workflow in the new environment to ensure that the migrated tasks can continue seamlessly.

[0104] The specific implementation includes: Prepare the environment: adjust the software environment of the receiving node according to the fusion parameters, and ensure that all dependencies are correctly installed and configured. Synchronize data and status: copy the data set, model parameters and other information involved in the overload process to the receiving node, and restore the latest training status. Start the mirroring work process: according to the pre-defined transfer plan, start a new instance on the receiving node that is completely consistent with the original task and start executing the scheduled task.

[0105] Through the above method, combined with the functional characteristics and working cycle of the target node, the optimal migration path is planned by constructing a detailed process transfer network, and ultimately efficient and reliable resource scheduling is achieved.

[0106] In some embodiments, according to the process transfer network, the second node function information and the work cycle information, the receiving node and the corresponding target transfer process are determined from the nodes of the transfer container, including: S1051-S1055.

[0107] S1051. Determine a transfer node, a corresponding transfer process, and a corresponding congestion period according to a process transfer network.

[0108] Exemplarily, based on the process transfer network, identify which nodes are overloaded nodes (i.e., nodes that need to migrate tasks). Assign specific transfer processes to each transfer node, which include the tasks they are running and their resource requirements. Evaluate the load of each transfer node at different time periods of the day, find out the peak and off-peak periods, and determine the best time for migration.

[0109] S1052: According to the transfer process, the transfer nodes are divided into first-class nodes and second-class nodes.

[0110] For example, according to the characteristics of the transfer process (such as computation-intensive vs. I / O-intensive), the transfer nodes are divided into two categories. The first category of nodes refers to those nodes that run computation-intensive tasks, which usually occupy a large amount of CPU / GPU resources. The second category of nodes refers to those tasks that mainly involve data reading and writing or other I / O operations, which may occupy more memory or storage bandwidth.

[0111] S1053. Determine a first idle node from the nodes of the transfer container according to the first-class nodes and the node function information, wherein the ratio of the first idle node to the first-class nodes is 2:1.

[0112] For example, in order to ensure the successful migration of a type of nodes, a sufficient number of idle nodes with matching performance must be found as receiving points. Usually, the number of idle nodes selected is greater than the number of transfer nodes. In this way, in the subsequent process, the specific receiving nodes are finally determined by observing the training quality of the idle nodes.

[0113] Specifically, check the functional information of all nodes in the transfer container, especially the hardware configuration similar to the first-class nodes (such as the number of GPU cores, memory size, etc.). According to the ratio of 2:1, find two first idle nodes suitable as the receiving points for each first-class node.

[0114] S1054. Determine a second idle node from the nodes of the transfer container according to the second-category nodes and the node function information, wherein the ratio of the second idle node to the second-category nodes is 1.2:1.

[0115] For example, for the second type of nodes, although the resource requirements are relatively small, it is still necessary to ensure that there are enough idle nodes to support their smooth migration.

[0116] Specifically, pay special attention to the hardware characteristics related to the second-class nodes, such as memory capacity, storage performance, etc. According to the ratio of 1.2:1, find at least one idle node suitable as a takeover point for each second-class node. Consider the working cycle and resource status of the node, and select the most suitable idle node as the second idle node.

[0117] S1055. Determine a receiving node from the first idle node and the second idle node according to the congestion cycle and the working cycle information, and set the transfer process corresponding to the transfer node adapted by the receiving node as the target transfer process.

[0118] For example, when finally determining the receiving node, in addition to considering resource matching, it is also necessary to combine congestion cycle and work cycle information to ensure that the migrated workload can run efficiently and stably.

[0119] Specifically, by comparing the congestion cycle and work cycle information, select nodes with low load and sufficient resources in the future as receiving nodes. Clarify which transfer processes will be migrated to the selected receiving nodes, and formulate a detailed transfer plan for each process.

[0120] Through the above method, combined with the characteristics and resource requirements of different types of nodes, a detailed migration plan was constructed to optimize resource utilization to ensure that the migration process is efficient and does not affect ongoing tasks.

[0121] See also Figure 2 , Figure 2 The embodiment of the present application also provides a schematic block diagram of a load balancing system of a storage-computing integrated hyper-converged server, wherein the load balancing system 200 of the storage-computing integrated hyper-converged server is used to execute the aforementioned load balancing method of the storage-computing integrated hyper-converged server. The load balancing system 200 of the storage-computing integrated hyper-converged server can be configured in a server.

[0122] Among them, the server can be an independent server or a server cluster, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN) and big data and artificial intelligence platforms.

[0123] like Figure 2 As shown, the load balancing system 200 of the storage and computing integrated hyper-converged server includes: a load warning module 201, a container determination module 202, a parameter determination module 203, a process mirroring module 204 and a process transfer module 205.

[0124] The load warning module 201 is used to screen overloaded containers and idle containers when an overload alarm occurs in the container resource pool.

[0125] The container determination module 202 is used to obtain overload process information of the overloaded container and determine a plurality of transfer containers from the idle containers according to the overload process information.

[0126] The parameter determination module 203 is used to detect whether fusion parameters are stored between multiple transfer containers. If no fusion parameters are stored, the node association parameters are obtained from the overload container, and the fusion parameters are generated according to the node association parameters.

[0127] The process mirroring module 204 is used to start multiple mirroring working processes in the transfer container according to the fusion parameters and the overloaded process information.

[0128] The process transfer module 205 is used to evaluate the accuracy of the mirror work process and determine the target work process, which is used to take over the released work process of the overloaded container.

[0129] An embodiment of the present application provides a storage-computing hyper-converged server, which includes a memory and a processor; the memory is used to store computer programs; the processor is used to execute the computer programs and implement a load balancing method for the storage-computing hyper-converged server such as any one of the embodiments of the present application when executing the computer programs.

[0130] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements a load balancing method for a storage-computing integrated hyper-converged server such as any one of the embodiments of the present application.

[0131] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for load balancing of a storage-computing integrated hyper-converged server, characterized in that: The method comprises: When an overload alarm occurs in the container resource pool, overloaded containers and idle containers are screened; Acquiring overload progress information of the overloaded container, and determining a plurality of transfer containers from the idle containers according to the overload progress information; Detecting whether fusion parameters are stored between the plurality of transfer containers, and if the fusion parameters are not stored, acquiring node association parameters from the overload container, and generating the fusion parameters according to the node association parameters; Starting a plurality of mirror work processes in the transfer container according to the fusion parameter and the overload process information; The correctness of the mirror work process is evaluated to determine a target work process, where the target work process is used to take over the released work process of the overloaded container.

2. The load balancing method for a storage-computing integrated hyper-converged server according to claim 1, characterized in that: The obtaining of the overload process information of the overloaded container and determining a plurality of transfer containers from the idle containers according to the overload process information comprises: Acquire first hardware configuration information and first software configuration information of the overloaded container, and acquire second hardware configuration information and second software configuration information of the idle container; Determine the overload process information according to the first hardware configuration information and the first software configuration information; A plurality of transfer containers are determined from the idle containers according to the overload process information, the second hardware configuration information and the second software configuration information.

3. The load balancing method for a storage-computing integrated hyper-converged server according to claim 2, characterized in that: The determining the overload process information according to the first hardware configuration information and the first software configuration information includes: Determining the neural network type and training instruction set of the overload container according to the first software configuration information, wherein the training instruction set is generated by compiling the neural network type and a preset neural network; Determining a plurality of current processes and process architecture parameters based on the first hardware configuration information and the neural network type; Compiling the training instruction set into an event queue; Determining, from the current process, a target process for executing each event in the event queue based on the event queue and the process architecture parameter; Determine the timing cycle information and running power consumption information required by the target process for executing the corresponding event according to a preset timing database and a preset power consumption database, wherein the timing database and the power consumption database are obtained by simulation according to the preset type of event and the process architecture parameters; The overload process information is determined according to the timing cycle information and the operation power consumption information.

4. The load balancing method for a storage-computing integrated hyper-converged server according to claim 3, characterized in that: The step of determining a plurality of transfer containers from the idle containers according to the overload process information, the second hardware configuration information, and the second software configuration information comprises: Determine the operation bearing information of the idle container according to the second hardware configuration information and the second software configuration information; Determining the transfer adaptability of the idle container according to the operation load information and the overload process information; A transfer container is determined from the free containers according to the transfer fitness.

5. The load balancing method for a storage-computing integrated hyper-converged server according to claim 1, characterized in that: The generating the fusion parameter according to the node association parameter comprises: Determine the transfer node and the corresponding first node function information according to the node association parameter; generating a node topology network according to the node transfer network and the first node function information; The fusion parameter is determined according to the node topology network.

6. The load balancing method for a storage-computing integrated hyper-converged server according to claim 5, characterized in that: The starting of multiple mirror work processes in the transfer container according to the fusion parameter and the overload process information includes: Acquire second node function information and working cycle information of the transfer container; generating a process transfer network according to the fusion parameter and the overloaded process information; Determine a receiving node and a corresponding target transfer process from the nodes of the transfer container according to the process transfer network, the second node function information and the work cycle information; The mirror work process corresponding to the target transfer process is started in the receiving node.

7. The load balancing method for a storage-computing integrated hyper-converged server according to claim 6, characterized in that: The step of determining a receiving node and a corresponding target transfer process from the nodes of the transfer container according to the process transfer network, the second node function information and the work cycle information includes: Determining a transfer node, a corresponding transfer process, and a corresponding congestion period according to the process transfer network; According to the transfer process, the transfer nodes are divided into first-class nodes and second-class nodes; Determine a first idle node from the nodes of the transfer container according to the first type of nodes and the node function information, wherein the ratio of the first idle node to the first type of nodes is 2:1; Determine a second idle node from the nodes of the transfer container according to the second type of nodes and the node function information, wherein the ratio of the second idle node to the second type of nodes is 1.2:1; According to the congestion cycle and the working cycle information, the receiving node is determined from the first idle node and the second idle node, and the transfer process corresponding to the transfer node adapted by the receiving node is set as the target transfer process.

8. A load balancing system for a storage and computing integrated hyper-converged server, characterized in that: The load balancing system of the storage-computing integrated hyper-converged server includes: The load warning module is used to screen overloaded containers and idle containers when an overload alarm occurs in the container resource pool; a container determination module, configured to obtain overloading process information of the overloaded container, and determine a plurality of transfer containers from the idle containers according to the overloading process information; a parameter determination module, configured to detect whether fusion parameters are stored between the plurality of transfer containers, and if the fusion parameters are not stored, to obtain node association parameters from the overload container, and to generate the fusion parameters according to the node association parameters; A process mirroring module, used for starting a plurality of mirroring working processes in the transfer container according to the fusion parameter and the overloaded process information; The process transfer module is used to evaluate the accuracy of the mirror work process and determine the target work process, where the target work process is used to take over the released work process of the overloaded container.

Images