Target node determination method, container migration method and electronic equipment

Abstract

The invention provides a target node determination method, a container migration method and electronic equipment. According to the method, the historical operation data, the container type classification and the real-time resource utilization rate are combined, so that the nodes with unbalanced loads can be accurately identified, and the most suitable target node is selected for container migration. Through the method, which containers need to be migrated can be accurately decided, and according to the resource requirements of the containers and the resource availability of the target node, the migration times are reduced, and the allocation of cluster resources is optimized, so that the resource utilization efficiency and the load balancing effect of the cluster are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of container technology, and in particular to a target node determination method, a container migration method and an electronic device. BACKGROUND

[0002] Container technology is an emerging virtualization solution that can package an application and its required resources (operating system, library, configuration file, etc.) together, so that it can run on different operating systems without the need to reconfigure the environment. After the container is deployed on the container cluster, different application services in the cloud environment based on container technology have different usage preferences for resources, and will dynamically change over time. This will cause dynamic changes in the load of running containers, which may lead to the phenomenon of uneven load in the cluster. Container migration refers to migrating a deployed container to run on other nodes, adjusting the mapping relationship between the container and the cloud server to achieve load balancing.

[0003] In related technologies, in a dynamic load environment, the existing container migration method of the node cluster has the technical problem of low decision accuracy. SUMMARY

[0004] The purpose of the present application is to overcome the above technical deficiencies and provide a target node determination method, a container migration method and an electronic device to solve the technical problem of low decision accuracy of the existing container migration method of the node cluster in related technologies in a dynamic load environment.

[0005] To achieve the above technical purpose, the present application adopts the following technical solutions: In a first aspect, the present application provides a target node determination method, which is applied to a controller in a node cluster, the node cluster comprising at least two nodes, and the nodes being deployed with at least two containers; the method comprising: In response to detecting a target event, obtaining running data of a first target node; wherein the target event is an event for indicating that container migration is triggered on the first target node, and the running data comprises historical running data and real-time running data in a preset time period; Based on the historical running data of the first target node and a preset container type classification algorithm, determining the container type of each container in the first target node; Based on the container type, real-time running data and target event, determining a to-be-migrated container identifier; Based on the to-be-migrated container identifier, obtaining its corresponding resource requirement data and real-time resource availability data of the node cluster; Determine a second target node based on the resource demand data, real-time resource availability data, and a preset container migration decision model, wherein the second target node is a node for receiving the container to be migrated.

[0006] Further, before the step of obtaining the running data of the first target node in response to detecting the target event, the method further comprises: Traverse the current running data of each node in the node cluster. Determine whether a node triggers a target event, wherein the target event includes at least a first target event and a second target event, wherein the first target event indicates that the CPU utilization in the node is greater than a CPU utilization threshold and / or the memory utilization is greater than a CPU utilization threshold, and the second target event indicates that the absolute value of the difference between the CPU utilization and the memory utilization is greater than a difference threshold. In the case where it is determined that a node triggers a target event, determine the corresponding node as the first target node.

[0007] Further, based on the historical running data of the first target node and a preset container type classification algorithm, determine the container type of each container in the first target node, comprising: Input the historical running data of the first target node into a preset K-Means clustering algorithm to obtain the container type of each container in the first target node, wherein the container type includes at least a first container type and a second container type, wherein the first container type indicates a container with urgent demand for CPU resources, and the second container type indicates a container with urgent demand for memory resources.

[0008] Further, the step of determining the container to be migrated based on the container type, real-time running data, and target event, comprising: Use a preset maximum resource utilization strategy, and the container type, real-time running data, and target event to determine the container to be migrated, wherein the maximum resource utilization strategy indicates a strategy of selecting a container with the highest real-time resource utilization from a container set of the corresponding container type as the object to be migrated according to the resource type triggered by the target event.

[0009] Further, the step of determining the container to be migrated based on the container type, real-time running data, and target event, comprising: determining the type of the target event, and determining the container type to be selected based on the type of the target event; wherein, if the target event is a first target event, determining the container type to be selected according to the resource type triggered in the first target event; if the target event is a second target event, determining the container type to be selected according to the difference between the CPU utilization and the memory utilization in the first target node; wherein, if the first target event is triggered by the CPU utilization being greater than a CPU utilization threshold, determining the container type to be selected as the first container type; if the first target event is triggered by the memory utilization being greater than a memory utilization threshold, determining the container type to be selected as the second container type; if the difference is positive and the absolute value is greater than a difference threshold, determining the container type to be selected as the first container type; if the difference is negative and the absolute value is greater than a difference threshold, determining the container type to be selected as the second container type; from the determined container type corresponding container, selecting the container with the highest corresponding resource utilization in the real-time running data, and recording its container identifier as the container identifier to be migrated; wherein, when the container type to be selected is the first container type, the corresponding resource is CPU; when the container type to be selected is the second container type, the corresponding resource is memory.

[0010] Further, the step of determining the second target node based on the resource requirement data, the real-time resource availability data, and the preset container migration decision model comprises: inputting the resource requirement data corresponding to the container identifier to be migrated and the real-time resource availability data of the node cluster into the pre-trained container migration decision model based on Dueling DQN respectively, to obtain the identifier of the second target node.

[0011] Further, the step of inputting the resource requirement data corresponding to the container identifier to be migrated and the real-time resource availability data of the node cluster into the pre-trained container migration decision model based on Dueling DQN respectively, to obtain the identifier of the second target node comprises: constructing a state space based on the resource requirement data and the real-time resource availability data; wherein, in the state space, the real-time resource availability data is the real-time resource availability data of other nodes except the first target node; input the state space into a pre-trained container migration decision model based on a Dueling DQN to obtain an action number; wherein the container migration decision model based on the Dueling DQN is trained by a preset reward function and an optimization objective function in a pre-training process; wherein the reward function is a joint reward function for representing a node internal load balancing reward and a cluster overall load balancing reward, and the optimization objective function is used for jointly minimizing a first index, a second index and a third index under the conditions of meeting a first constraint, a second constraint, a third constraint, a fourth constraint and a fifth constraint; wherein the first index is an index for representing a cluster node CPU utilization rate standard deviation, the second index is an index for representing a cluster node memory utilization rate standard deviation, and the third index is an index for representing a node internal load imbalance degree; the first constraint is used for representing a CPU resource capacity constraint, the second constraint is used for representing a memory resource capacity constraint, the third constraint is used for representing a migration number consistency constraint, the fourth constraint is used for representing a migration operation legality constraint, and the fifth constraint is used for representing a migration path legality constraint; map the action number to a node identifier as an identifier of the second target node.

[0012] Further, the step of inputting the state space into the pre-trained container migration decision model based on the Dueling DQN to obtain the action number comprises: an input layer of the container migration decision model based on the Dueling DQN receives a state space vector in the state space; a shared feature extraction layer of the container migration decision model based on the Dueling DQN performs feature extraction on the state space vector, and then outputs a state value function through a preset first branch and outputs an advantage function through a second branch; wherein the state value function is used for evaluating a cluster overall state value, and the advantage function is used for evaluating a relative advantage of each migration action; a merging layer of the container migration decision model based on the Dueling DQN merges the state value function and the advantage function to obtain an action value function; select an action number that makes an output value of the action value function maximum as an output.

[0013] In a second aspect, the present application provides a container migration method, comprising: determining a first target node and a second target node; wherein the first target node and the second target node are determined by using the above-mentioned target node determination method; migrating a to-be-migrated container in the first target node to the second target node.

[0014] In a third aspect, the present application provides an electronic device, comprising: a memory, and one or more processors connected in communication with the memory; instructions stored in the memory and executable by the one or more processors, the instructions being executed by the one or more processors to cause the one or more processors to implement the method described above.

[0015] Advantages: The present application can accurately identify the nodes with uneven load and select the most suitable target node for container migration by combining historical operation data, container type classification and real-time resource utilization. Through this method, it can accurately decide which containers need to be migrated, and according to the resource requirements of the containers and the resource availability of the target node, reduce the number of migrations, optimize the allocation of cluster resources, and thus improve the resource utilization efficiency and load balancing effect of the cluster. In addition, this method can dynamically adapt to the load changes of the cluster environment, avoid the lag problem of traditional static rule systems, and significantly improve the overall flexibility and response speed. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 is a flowchart of a method for determining a target node provided by an embodiment of the present application; Figure 2 is a schematic diagram of one of the container migration scenarios (uneven load of the cluster) provided by an embodiment of the present application; Figure 3 is a schematic diagram of another of the container migration scenarios (uneven load of different resource dimensions within a node) provided by an embodiment of the present application; Figure 4 is a schematic diagram of a container migration system model provided by an embodiment of the present application; Figure 5 is a schematic diagram of a container migration selection process provided by an embodiment of the present application; Figure 6 is a structure diagram of a Dueling DQN network provided by an embodiment of the present application; Figure 7 is a schematic diagram of a cumulative reward curve provided by an embodiment of the present application; Figure 8 is a schematic diagram of the standard deviation of the CPU utilization rate of the cluster before and after migration provided by an embodiment of the present application; Figure 9 is a schematic diagram of the standard deviation of the memory utilization rate of the cluster before and after migration provided by an embodiment of the present application; Figure 10 is a schematic diagram of the internal load imbalance degree provided by an embodiment of the present application. DETAILED DESCRIPTION

[0017] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application, so that those skilled in the art can better understand the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work should be within the scope of the present application.

[0018] In the related art, containerized applications are deployed in a node cluster by an orchestration system such as Kubernetes. The node cluster can be composed of multiple physical or virtual servers, and containers run various application services through these nodes. Each node in the cluster can carry tens to hundreds of containers, forming a highly centralized resource usage scenario.

[0019] In this scenario, on the one hand, the resource requirements of containers fluctuate over time. Taking an e-commerce platform as an example, during a promotion, the user access volume surges, resulting in a significant increase in CPU resource requirements of Web service containers. Scheduled batch processing tasks can also cause a substantial increase in CPU or memory requirements during certain periods. These dynamic changes in demand make the allocation of cluster resources more complex.

[0020] On the other hand, there are significant differences in resource requirements among containers of different service types. For example, AI inference services can be CPU-intensive, while memory-intensive services such as graph databases have higher memory resource requirements. Such differences mean that different containers on the same node need to be treated differently in terms of priority and handling in resource allocation.

[0021] On the other hand, modern clusters are often very large, involving tens to thousands of nodes, and the number of containers running on each node can reach 10 to 100. Such a large scale increases the difficulty of cluster resource scheduling and container migration decisions.

[0022] In the related art, in existing container migration techniques, although there are some heuristic algorithms or optimization methods to address the load imbalance problem, these methods still have significant limitations and are difficult to achieve efficient and accurate container migration in complex and dynamic environments.

[0023] Specifically, in some possible embodiments, heuristic algorithms can be used to make container migration decisions. For example, Best-Fit algorithm and improved ant colony algorithm can be applied to the optimization of cluster load balancing. Although these heuristic algorithms can provide certain optimization effects in some static environments, there are still significant limitations, especially when facing dynamic changing loads. Heuristic algorithms usually decide container migration strategies based on pre-set static rules. For example, the Best-Fit algorithm migrates containers to the most suitable node, and the ant colony algorithm selects a migration path by simulating the foraging behavior of ants. The parameters of these algorithms are fixed at the time of system design, and lack the ability to adaptively adjust according to real-time load changes. When the cluster load fluctuates sharply (for example, e-commerce platforms will experience a surge in traffic during promotions, or timed batch processing tasks are executed), these algorithms cannot respond to load fluctuations in time. For example, when the CPU resources on a node suddenly overload, heuristic algorithms usually rely on predetermined rules and parameters to make decisions, which makes the system unable to quickly adapt to rapid changes in load, resulting in a lag in migration decisions. This not only increases the cost of migration, but also may cause delays in system response time, affecting the performance and efficiency of the entire cluster.

[0024] In some possible embodiments, the different resource requirements of container types are often overlooked. Most existing technologies assume that all containers have the same resource requirements and adopt a unified strategy for migration decisions. However, in actual scenarios, containers have different resource requirements depending on the type of service they run, for example, AI inference services are usually CPU-intensive, while graph database services are memory-intensive. The differences in resource requirements between different types of containers are very obvious, but existing methods usually do not distinguish between these container types, resulting in inaccuracy in migration decisions.

[0025] In some possible implementation solutions, only the load balancing between nodes is concerned, for example, global load balancing is achieved by optimizing the CPU utilization standard deviation and the memory utilization standard deviation of each node in the cluster. This method attempts to improve the use efficiency of cluster resources by balancing the load of each node. However, this approach ignores the fragmentation of internal resources of a node, that is, there may be significant load imbalance between different resources (such as CPU and memory) in the same node. Specifically, the CPU resource utilization of a node can reach 90%, while the memory utilization is only 30%. This fragmentation of internal resources of a node causes the cluster resources to be underutilized. In this case, even if the cluster as a whole appears to achieve load balancing, there is still a large amount of resource waste in a single node. For example, the memory resource is insufficient while the CPU resource is excessive, which causes the memory of the node to be underutilized, ultimately affecting the throughput and performance of the entire cluster. This internal load imbalance of a node greatly affects the improvement of resource utilization.

[0026] It can be understood that the existing system cannot adapt to the dynamic changes of the cluster environment, resulting in a lag in migration decisions. For example, the ant colony algorithm and other heuristic methods are usually parameterized and cannot adjust the decision strategy in real time. The existing migration strategy fails to effectively distinguish the resource usage patterns of containers (such as CPU-intensive and memory-intensive containers), resulting in a high invalid migration rate. The blindness of migration decisions reduces the efficiency of migration and fails to fully utilize the resources of the cluster.

[0027] In summary, in the related art, in a dynamic load environment, the existing container migration method of a node cluster has the technical problem of low decision accuracy.

[0028] As shown in Figure 1 The embodiment provides a target node determination method. The method is applied to a controller in a node cluster. The node cluster includes at least two nodes. The nodes are deployed with at least two containers.

[0029] In the embodiment, the node cluster can include a plurality of nodes (which can be physical servers or virtual servers) and a controller in communication connection with the nodes. The controller can be a node in the node cluster or a special controller (for example, an independent physical server which can be in communication connection with each node in the node cluster for data exchange).

[0030] In the embodiment, the node cluster can be a node cluster of a cloud computing data center.

[0031] In the embodiment, the node cluster can be a node cluster of an edge computing network.

[0032] In the embodiment, the node cluster can be a node cluster of a real-time data processing platform.

[0033] The execution subject of the method can be a controller in the node cluster, and the method can include: Step S10: In response to detecting a target event, obtaining running data of the first target node; wherein the target event is an event for representing triggering the container migration of the first target node, and the running data includes historical running data and real-time running data in a preset time period.

[0034] In the embodiment, the target event can include a first target event and a second target event; wherein the first target event indicates that the CPU utilization in the node is greater than a CPU utilization threshold and / or the RAM utilization is greater than the CPU utilization threshold; and the second target event indicates that the absolute value of the difference between the CPU utilization and the RAM utilization is greater than a difference threshold.

[0035] In the embodiment, the target event can also include a third target event, which can be that the processing delay or response time of the node exceeds a threshold.

[0036] In the embodiment, the target event can also include a fourth target event, which can be that the storage space of the node is insufficient.

[0037] In the embodiment, the target event can also include a fifth target event, which can be that the network delay of the node exceeds a threshold.

[0038] In the embodiment, the first target node can be a node that triggers the target event, and the first target node needs to migrate some containers to other nodes to achieve load balancing. In other words, the first target node can be a node to be migrated. It can be understood that the first target node can be a node to be migrated due to high resource utilization or internal resource load imbalance (such as extreme inconsistency of CPU and memory usage).

[0039] In the embodiment, the preset time period can be half an hour, one hour, one day, etc.

[0040] In this embodiment, the execution subject (controller of the node cluster) can obtain the state information of the nodes in real time by establishing communication channels with each node in the node cluster. Each node in the cluster can run a node agent program or a monitoring module, which is responsible for reporting the running state of the node to the controller periodically. For example, the controller can access the running data of the nodes through the API Server of Kubernetes. Kubernetes provides comprehensive monitoring and management functions for nodes and containers, and the controller can obtain the resource utilization, load information, and container state of the nodes by calling the API interface.

[0041] In this embodiment, the real-time running data can represent the resource utilization of the nodes at the current time, such as CPU usage, memory usage, disk I / O, and network traffic, etc. The above-mentioned data can be collected in real time by a cluster monitoring system (e.g., Prometheus, Kubernetes Metrics Server, cluster management tools, etc.) and provided to the controller through an API.

[0042] In this embodiment, the historical running data can represent the resource utilization of the nodes in the past period of time, which can help the controller evaluate the long-term resource consumption trend of the nodes to make more accurate migration decisions. For example, the controller can obtain the historical data of node resource usage in the past 1 hour, 1 day, or longer. These data can help determine whether the node has periodic load fluctuations or a long-term load trend.

[0043] Step S12: determining the container type of each container in the first target node based on the historical running data of the first target node and a preset container type classification algorithm.

[0044] In this embodiment, the preset container type classification algorithm can be a machine learning-based container type classification algorithm. Specifically, a supervised learning model can be pre-trained to classify the container type based on historical running data. First, a large amount of historical running data with labels needs to be collected, including resource usage of the container (CPU utilization, memory occupation, disk I / O, etc.) and corresponding container type labels (e.g., CPU-intensive, memory-intensive, hybrid, etc.). Then, features (e.g., resource consumption pattern, load fluctuation, etc.) are extracted from the historical running data as input to the machine learning model. Next, the labeled historical data is used to train the preset machine learning algorithm (e.g., support vector machine (SVM), random forest (Random Forest), etc.) to generate a classification model.

[0045] In the embodiment, the preset container type classification algorithm can be a container type classification algorithm based on deep learning. Specifically, first, a large amount of historical running data with labels is collected, including resource usage (CPU utilization, memory occupation, disk I / O, etc.) of the container and corresponding container type labels (for example, CPU-intensive, memory-intensive, mixed, etc.). Then, the historical running data and the container type labels are used to train a deep learning model (convolutional neural network, long short-term memory network), to obtain a classification model.

[0046] In the embodiment, the preset container type classification algorithm can be a container type classification algorithm based on clustering algorithm. Specifically, the container type classification method based on clustering algorithm belongs to an unsupervised learning method, which classifies containers with similar resource usage patterns into a class through clustering analysis of historical running data, without relying on existing label data.

[0047] In the embodiment, the container type classification algorithm based on clustering algorithm can be a K-Means clustering algorithm.

[0048] In the embodiment, the container type classification algorithm based on clustering algorithm can be a DBSCAN clustering algorithm.

[0049] In the embodiment, the container type classification algorithm based on clustering algorithm can be a hierarchical clustering algorithm.

[0050] Step S14: determining the to-be-migrated container identifier based on the container type, real-time running data, and target event.

[0051] In the embodiment, the to-be-migrated container identifier is determined based on the maximum resource utilization strategy, container type, real-time running data, and target event.

[0052] Step S16: obtaining resource requirement data corresponding to the to-be-migrated container and real-time resource availability data of the node cluster based on the to-be-migrated container identifier.

[0053] In the embodiment, the resource requirement data of the to-be-migrated container can represent the resource requirements (which can be real-time requirements) of CPU, memory, storage, network bandwidth, etc. required by the to-be-migrated container when running.

[0054] In the embodiment, the resource requirement data corresponding to the to-be-migrated container can be obtained through a monitoring system in the cluster.

[0055] In the embodiment, the real-time resource availability data of the node cluster refers to the remaining available resources of each node in the cluster. It can also be obtained through a monitoring system in the cluster.

[0056] Step S18: determining a second target node based on the resource requirement data, the real-time resource availability data, and a preset container migration decision model; wherein the second target node is a node for receiving the container to be migrated.

[0057] In the present embodiment, the preset container migration decision model can be a container migration decision model based on a deep reinforcement learning algorithm.

[0058] In the present embodiment, the container migration decision model based on the deep reinforcement learning algorithm can be a container migration decision model based on Dueling DQN.

[0059] This embodiment can accurately identify nodes with unbalanced loads and select the most suitable target node for container migration by combining historical running data, container type classification, and real-time resource utilization. Through this method, it can accurately decide which containers need to be migrated and reduce the number of migrations according to the resource requirements of the containers and the resource availability of the target nodes, optimizing the allocation of cluster resources and improving the resource utilization efficiency and load balancing effect of the cluster. In addition, this method can dynamically adapt to changes in the load of the cluster environment, avoiding the lag problem of traditional static rule systems, and significantly improving the overall flexibility and response speed.

[0060] In some embodiments, before the step of obtaining running data of a first target node in response to detecting a target event, the method further comprises: Step S02: traversing the current running data of each node in the node cluster.

[0061] Step S04: determining whether a node triggers a target event; wherein the target event includes at least a first target event and a second target event; wherein the first target event indicates that the CPU utilization in the node is greater than a CPU utilization threshold and / or the memory utilization is greater than the CPU utilization threshold; the second target event indicates that the absolute value of the difference between the CPU utilization and the memory utilization is greater than a difference threshold value. Step S06: determining the corresponding node as the first target node in the case where it is determined that a node triggers a target event.

[0062] In one specific embodiment, the embodiment can be applied to container deployment in Kubernetes. The embodiment can be a container migration method based on deep reinforcement learning in Kubernetes.

[0063] As shown in Figure 2 and Figure 3 There are two operations that trigger container migration: 1. The startup and shutdown of containers cause changes in the consumption of physical node resources, or the dynamic changes in the load of a large number of containers running on the node.

[0064] 2. Containers with the same resource usage preference are scheduled to the same node.

[0065] like Figure 4 As shown, regarding the selection of migration nodes, at a certain time point t, the cluster state is St. Because the load on the nodes in the cluster is unbalanced in the St state, the container migration operation is triggered. Based on the above analysis, there are two main situations that trigger the container migration operation: either a node in the cluster is overloaded or the nodes in the cluster are in a state of resource fragmentation.

[0066] Mathematical models can be used for these two cases: By periodically collecting cluster operating status data, including the load information of nodes and containers on those nodes, within a time period T, at time t, the node... The CPU and memory utilization rates are respectively .

[0067] In the first scenario, when the load on a certain node is too high, it indicates that the node... There is an overload situation within the node, requiring adjustments to the node. The container running on it is being migrated. This occurs when the node's CPU or memory utilization exceeds a threshold. or Its mathematical model is shown below:

[0068]

[0069] In the second scenario, when node resources are fragmented—that is, when the difference between the current CPU and memory utilization within the node is too large—this situation is defined as the absolute value of the difference between the node's CPU and memory utilization exceeding a certain threshold. This indicates that the node The uneven use of resources across different dimensions within the node requires adjustments to the node's performance. The container running on it performs migration operations. Its mathematical model is as follows:

[0070] In some threshold-based load balancing methods, a migration operation is triggered as soon as the load of a certain resource exceeds a set threshold. However, in actual production environments, a transient load value greater than the set threshold often occurs in a cluster, triggering a migration operation. Frequent triggering of migration operations can cause unnecessary waste of resources. To avoid this, in the present embodiment, the cluster is determined to need to trigger a migration operation only when the cluster has triggered a migration operation in continuous observation.

[0071] When the CPU or memory utilization of a node is less than a threshold and , or the absolute value of the difference between the CPU and memory utilization is less than , the migration operation is stopped. This ensures that the cluster load is balanced while maintaining a high level of resource utilization. The mathematical model is as follows:

[0072] The present embodiment dynamically identifies nodes that are overloaded or have uneven load by traversing the current running data of each node in the cluster before making a container migration decision to determine whether a target event has been triggered. This method effectively identifies nodes that may need to be migrated in advance, ensuring that the cluster can respond quickly when load fluctuations occur. By accurately determining whether the first target event or the second target event is triggered, the present embodiment can address the problem of node resource overload or excessive difference between memory and CPU load, thereby achieving more efficient load balancing and resource optimization, and improving the stability of the system and the accuracy of container migration.

[0073] In some embodiments, based on the historical running data of the first target node and a preset container type classification algorithm, the step of determining the container type of each container in the first target node comprises: Step S122: input the historical running data of the first target node into a preset K-Means clustering algorithm to obtain the container type of each container in the first target node; wherein the container type includes at least a first container type and a second container type; wherein the first container type represents a container with urgent CPU resource demand, and the second container type represents a container with urgent memory resource demand.

[0074] In the embodiment, the first container type is a container representing urgent demand for CPU resources. Specifically, the first container type can represent a container that mainly relies on central processing unit (CPU) resources at runtime. Containers of the first container type tend to require a large amount of computing power to handle complex computing tasks, data processing, compute-intensive applications, and the like. CPU is the bottleneck resource of the performance of the first container type, and therefore its resource demand is urgent, requiring the node to provide high CPU processing power. The containers of the first container type can be containers used to perform tasks such as data analysis, video encoding, AI inference, and the like.

[0075] In the embodiment, the second container type is a container representing urgent demand for memory resources. Specifically, the second container type can represent a container that has a relatively urgent demand for memory (RAM) resources at runtime. Containers corresponding to the second container type tend to require a large amount of memory space. Containers corresponding to the second container type tend to require a high memory utilization rate, especially when performing large-scale data caching, database queries, memory database operations, and the like, in which case the memory demand increases substantially.

[0076] In one specific implementation, historical data of container runtime on the node can be collected and used to cluster the containers using the K-Means algorithm.

[0077] When a container migration operation is triggered, a container running on the node needs to be selected for migration. Assume that at time t, the node needs to perform a migration operation, and the set of containers running on the node is . The container to be migrated is selected from the set of containers, and the selection strategy for migrating the container is to select a set of migration containers from the set of running containers to achieve a state of load balancing between the node and the cluster.

[0078] To quickly restore the load balancing state of the node and reduce the number of container migrations, the container selection strategy needs to quickly and efficiently select the containers to be migrated.

[0079] Consider the case where the containers on the node need to be migrated. When the CPU resource utilization on the node is too high, since the resource usage on the node is the cumulative result of resource consumption by the running containers, the running containers that consume a high amount of CPU resources on the node need to be selected for migration to a node with a lower CPU resource utilization. When the node has resource fragmentation, and assuming that the memory utilization is much higher than the CPU resource utilization, containers with a high memory resource utilization need to be selected for migration.

[0080] ​When the container cluster is in a load imbalance situation, such as a large difference in internal resource usage of the node, in order to more targetedly select the container to be migrated for this load imbalance situation, the embodiment combines the differences in resource usage of the containers carrying application services to select the migration container in the migration container selection stage. When the container is deployed to the node, it is difficult to predict in advance the resource consumption of the application service carried by the container due to the lack of information of the container runtime, and it is necessary to classify the containers carrying application services according to the resource usage of the application service runtime on the container. However, the load of the container has instantaneous fluctuation, and therefore it is not accurate to classify the container according to the usage of a single resource in a period of time.

[0081] To solve the above problem, the embodiment proposes a container clustering algorithm based on K-Means algorithm, which can cluster the containers running on the node according to the historical usage of the resources by the running containers on the node, and cluster the containers with similar resource usage into the same type. The resource demand of the containers of the same type in the future also has similarity, and therefore when migration is needed, the migration container can be more efficiently selected according to the current load imbalance situation and the container type.

[0082] According to the above analysis, an unsupervised method is needed to cluster the containers carrying application services when performing container migration; at the same time, the algorithm should have good convergence and be able to obtain results quickly in the scenario of container migration. K-Means algorithm has the advantages of high computational efficiency, simple algorithm implementation, and strong interpretability, and is suitable for clustering the containers carrying application services at a faster speed in the scenario of container migration, and therefore the embodiment selects the unsupervised clustering algorithm K-Means algorithm to classify the containers. K-Means algorithm is a clustering algorithm, and its goal is to divide data points into multiple different groups or clusters, so that the similarity between data points in the same group is high, and the similarity between different groups is low. The process of K-Means service container clustering algorithm is as follows: Input: data set D, number of clusters 2 (CPU-intensive, memory-intensive) Output: clustering result C 1. Randomly select 2 data points as initial cluster centers; 2. Initialize the clustering result C as 2 empty clusters; 3. repeat; 4. For each data point in the data set, calculate the distance to the 2 cluster centers, and assign it to the cluster with the closest distance; 5. For each cluster, calculate the mean of all data points in it, and update the cluster center of the cluster; 6. Until the clustering result no longer changes or the maximum number of iterations is reached; 7. Treat each cluster as a clustering result and return the result C.

[0083] To classify containers using the K-Means algorithm, we first obtain runtime data for the containers. For container ci, within a time period T, the state collected at every t time intervals is as follows:

[0084] In the formula, and and represent the CPU and memory resource utilization of the container at time t, respectively. For the set C of all containers running on the cluster, the resulting dataset D is:

[0085] The information collected from all containers is used as dataset D. K-Means clustering is then performed to obtain the clustering results. This implementation considers two types of containers: CPU-intensive and memory-intensive, meaning the number of clusters is 2. Based on the clustering results, containers are categorized into CPU-intensive and memory-intensive types. At the node... Above, the running CPU type and memory type containers constitute two sets. and .

[0086] This implementation method inputs the historical operational data of the first target node into a preset K-Means clustering algorithm, enabling efficient and accurate classification of containers within the first target node. The K-Means algorithm is a computationally efficient, simple-to-implement, and highly interpretable unsupervised clustering algorithm, making it well-suited for container migration scenarios. The core objective of this algorithm is to divide data points (containers) into two distinct clusters, where containers within each cluster exhibit high similarity, while the similarity between different clusters is low. In this implementation, the K-Means algorithm can quickly cluster containers, identifying containers with higher CPU resource requirements (first container type) and containers with higher memory resource requirements (second container type). This method not only has low computational complexity but also clearly explains the basis for container classification, providing accurate support for container migration decisions and ultimately optimizing cluster load balancing and resource allocation efficiency.

[0087] In some implementations, the step of determining the identifier of the container to be migrated based on container type, real-time runtime data, and target events includes: Step S142: determining a container identifier to be migrated using a preset maximum resource utilization strategy, and the container type, real-time running data, and target event; wherein the maximum resource utilization strategy indicates a strategy of selecting a container with the highest real-time resource utilization from a container set of the corresponding container type as the object to be migrated according to the resource type triggered by the target event.

[0088] By using the maximum resource utilization strategy, the container type, the real-time running data, and the target event, the embodiment can efficiently select the container to be migrated. According to the resource type triggered by the target event, the strategy automatically selects the container with the highest real-time resource utilization from the container set with the corresponding resource requirement as the object to be migrated. In this way, the execution subject can preferentially migrate the container with the most intensive resource consumption, thereby effectively reducing the pressure of the node with excessive load, optimizing the resource allocation and load balancing within the cluster, reducing unnecessary migration operations, improving the efficiency of container migration, and improving the overall performance of the cluster. This strategy not only improves the resource utilization, but also reduces the waste of system resources, and ensures the stable operation of the cluster.

[0089] In some embodiments, the step of determining the container identifier to be migrated using the preset maximum resource utilization strategy, and the container type, real-time running data, and target event, comprises: Step S1422: determining the type of the target event, and determining the container type to be selected based on the type of the target event; wherein if the target event is a first target event, determining the container type to be selected according to the resource type triggered in the first target event; if the target event is a second target event, determining the container type to be selected according to the difference between the CPU utilization and the memory utilization in the first target node; wherein if the first target event is triggered by the CPU utilization being greater than the CPU utilization threshold, determining the container type to be selected as the first container type; if the first target event is triggered by the memory utilization being greater than the memory utilization threshold, determining the container type to be selected as the second container type; if the difference is positive and the absolute value is greater than the difference threshold, determining the container type to be selected as the first container type; if the difference is negative and the absolute value is greater than the difference threshold, determining the container type to be selected as the second container type. Step S1424: selecting the container with the highest corresponding resource utilization in the real-time running data from the containers corresponding to the determined container type, and recording the container identifier as the container identifier to be migrated; wherein when the container type to be selected is the first container type, the corresponding resource is the CPU; and when the container type to be selected is the second container type, the corresponding resource is the memory.

[0090] In one specific embodiment, the node h𝑗 that needs to be migrated and the type R of the container that needs to be migrated are determined, and then the data of the container runtime on the node is collected, and the container is classified after obtaining the data. Then, the container to be migrated is selected from the container set corresponding to the type R according to the maximum resource selection strategy.

[0091] When triggering the migration operation, first, it is determined which type of container on the node needs to be selected for migration. For example, if the CPU resource usage rate is too high, a container is selected from the CPU-intensive container set for migration. Then, a specific container is selected from the type container set for migration. The migration strategies mainly include: random selection strategy, minimum resource utilization rate strategy, and maximum resource utilization rate strategy. 1. Random selection strategy: always randomly select a container on the node for migration. This method is difficult to quickly select a container that uses more resources, resulting in a long-term imbalance in the load of the node.

[0092] 2. Minimum resource utilization rate strategy: always select the container with the minimum resource utilization rate. This strategy can ensure that the current node always has a high resource utilization rate, but the resource utilization rate decreases slowly.

[0093] 3. Maximum resource utilization rate strategy: opposite to the minimum resource utilization rate strategy, this strategy always selects the container with the maximum resource utilization rate. This strategy can quickly reduce the resource utilization rate of the node, but cannot guarantee a high resource utilization rate of the node.

[0094] The maximum resource utilization rate strategy is combined with the K-Means clustering to select the container to be migrated from the container set according to the load imbalance of the node, because this can reduce the migration times and make the cluster quickly recover to the load balanced state. At the same time, because of the minimum resource utilization rate limit, the node can be kept in a state of fully utilized resources. That is, each time a container corresponding to and is selected from the set and . The entire container migration container selection process is shown in FIG. 8. Figure 5

[0095] ​The embodiment significantly improves the accuracy and efficiency of container migration through a double screening mechanism (event type -> container type -> resource utilization peak container). First, by accurately identifying the target event type, the execution subject can lock the container of the triggering resource type (for example, select CPU-intensive containers when CPU is overloaded) for node overload events (first target event), avoiding the migration of low correlation containers; for resource fragmentation events (second target event), the execution subject selects the corresponding container type according to the imbalance of resources within the node (for example, CPU is much higher than memory), thereby effectively eliminating resource fragmentation. Second, during the migration process, the system dynamically selects the container with the highest real-time resource utilization, ensuring that a single migration can maximize the release of node pressure and significantly shorten the load recovery time. Finally, migrating containers with high resource occupation priority releases resource hotspots, and combining subsequent deep reinforcement learning decision optimization target node selection, the load balance between nodes and the coordination of resources within the node are achieved, thereby comprehensively improving the resource utilization and load balancing efficiency of the cluster.

[0096] In some embodiments, the step of determining the second target node based on the resource requirement data, real-time resource availability data, and a preset container migration decision model includes: Step S182: Input the resource requirement data of the container to be migrated and the real-time resource availability data of the node cluster into the pre-trained Dueling DQN-based container migration decision model respectively to obtain the identification of the second target node.

[0097] By inputting the resource requirement data of the container to be migrated and the real-time resource availability data of the node cluster into the pre-trained Dueling DQN-based container migration decision model, the embodiment can intelligently select the most suitable second target node. This method uses deep reinforcement learning technology to consider the resource requirements of the container and the real-time resource situation of the target node, thereby achieving efficient target node selection. In this way, the execution subject can accurately identify the most suitable node to carry the container to be migrated among multiple nodes, optimizing the load balancing and resource allocation of the cluster, reducing unnecessary migration operations, and improving the overall performance and resource utilization efficiency of the cluster. This method not only enhances the intelligent decision-making ability of the system, but also greatly improves the efficiency and accuracy of container migration.

[0098] In some embodiments, the step of inputting the resource requirement data of the container to be migrated and the real-time resource availability data of the node cluster into the pre-trained Dueling DQN-based container migration decision model to obtain the identification of the second target node includes: Step S1822: Construct a state space based on resource demand data and real-time resource availability data; wherein, in the state space, the real-time resource availability data is the real-time resource availability data of nodes other than the first target node; Step S1824: Input the state space into the pre-trained container migration decision model based on Dueling DQN to obtain the action number; wherein, the container migration decision model based on Dueling DQN is trained by a preset reward function and an optimization objective function during the pre-training process; wherein, the reward function is a joint reward function used to characterize the load balancing reward within a node and the overall load balancing reward of the cluster, and the optimization objective function is used to characterize the minimum of the first, second, and third indicators under the condition of satisfying the first, second, third, fourth, and fifth constraints; wherein, the first indicator is used to represent the standard deviation of CPU utilization of cluster nodes, the second indicator is used to represent the standard deviation of memory utilization of cluster nodes, and the third indicator is used to represent the load imbalance degree within a node; the first constraint is used to represent the capacity constraint of CPU resources, the second constraint is used to represent the capacity constraint of memory resources, the third constraint is used to represent the constraint of consistency of migration quantity, the fourth constraint is used to represent the constraint of legality of migration operation, and the fifth constraint is used to represent the constraint of legality of migration path. Step S1826: Map the action number to a node identifier, which serves as the identifier for the second target node.

[0099] In one specific implementation, after obtaining the container to be migrated, a target node needs to be selected as the migration destination. Traditional reinforcement learning algorithms typically use hand-designed features to represent the state, and these features may not be optimal. Against this backdrop, deep learning technology offers a new approach to solving these problems.

[0100] The key components of a deep reinforcement learning algorithm are its state space, action space, and reward function. Below, we will model these three parts for the target node selection problem.

[0101] 1. State Space The state of the input deep reinforcement learning agent consists of two parts: the state of the cluster nodes and the state of the containers. The state of the containers is the container to be migrated selected by the overloaded nodes. Required CPU and memory resources:

[0102] The status information of cluster nodes is the current CPU and memory availability of the nodes, defined as:

[0103] Therefore, the state space is: ; It should be noted that the state space only contains the current state information of the nodes, and does not contain the state information of the load-unbalanced nodes themselves.

[0104] 2. Action space The action space refers to the set of actions output by the deep reinforcement learning agent. In the scenario of container migration target node selection, the action space is the set of all nodes except the node that needs to be migrated. In this embodiment, the node number is used to represent: ; 3. Reward function The design of the reward function determines the optimization goal of the deep reinforcement learning algorithm and whether it can converge. In the scenario of container migration, assume that the agent selects node as the destination of the container to be migrated. Before migration, the state of node is:

[0105] After migrating the container to node , the state after migration is: ; The internal load imbalance of a node is defined as the variance of the difference between CPU and memory resource utilization within the node: ; Therefore, before and after migration, the internal load imbalance of node is .

[0106] The goal of container migration is to achieve load balancing of the cluster, including load balancing of the entire cluster and load balancing within the node. The reward function should also be composed of two parts: the load balancing reward of the entire cluster and the load balancing reward within the node . The load balancing degree of the entire cluster before and after migration is .

[0107] The reward for the load balancing degree of the entire cluster when container migration is successful is:

[0108] In the formula, 0.4 represents the reward for the load balancing degree of the entire cluster when container migration is successful, and represent the load imbalance degrees of the entire cluster before and after migration, represents the overall shift in average load state before and after container migration. ​​

[0109] Intra-node load balancing reward is defined as:

[0110] In the formula, respectively represent the degree of load imbalance in the node before and after the container migration, -1 and -0.8 represent the reward.

[0111] The intra-node load balancing reward is combined with the cluster overall load balancing reward value , and the reward function of the migration algorithm is:

[0112] The container deployment problem can be formally expressed as: a set of containers to be deployed , , and the container has a CPU and memory resource request amount represented by . For a set of physical machine node clusters , the CPU and memory resource capacity of node is represented by , . In order to represent whether the container is deployed on node , a binary variable is defined, which represents that the container is deployed on node , otherwise: :

[0113] The goal of container migration is to adjust the load of nodes in the cluster to make the cluster return to a load balanced state. The load balanced state includes two aspects, the overall load balanced state of the cluster and the intra-node load balanced state of the cluster, and two indicators are used to measure them in the embodiment.

[0114] The standard deviation optimization objective of the CPU and memory utilization rate of the nodes in the cluster is designed for the overall load balanced state of the cluster:

[0115] In the formula, respectively represent the standard deviation of the CPU and memory utilization rate of the nodes in the cluster, respectively represent the CPU utilization rate and the memory utilization rate of node j at the tth moment, respectively represent the CPU average utilization rate and the memory average utilization rate of node j at the tth moment. ​

[0116] In addition, the embodiment uses another indicator As a load balancing judgment indicator to measure the load situation between different resource dimensions within a node. The indicator is defined as the variance of the utilization rate of various resources within the cluster.

[0117] First, the node The average utilization rate of the CPU and memory resources within the node is: ; The utilization rate variance of different resource dimensions within the node is: ; The overall internal load imbalance of the cluster is: ; In this embodiment, the standard deviation of the CPU and memory utilization rate of the nodes within the cluster and the overall internal load imbalance are the optimization objectives. Combined with the system model definition, the optimization problem of the container migration algorithm can be represented as:

[0118] In the formula, can be represented as the optimization objective function, can be represented as the first indicator, can be represented as the second indicator, can be represented as the third indicator. Specifically, is represented as the standard deviation of the CPU utilization rate of the cluster nodes, is represented as the standard deviation of the memory utilization rate of the cluster nodes, is the sum of the internal imbalance of each node in the node cluster.

[0119] can be represented as the first constraint; can be represented as the second constraint; Specifically, can be represented as whether the container is deployed on the node , if yes, it is 1, otherwise it is 0, can be represented as whether the container needs to be migrated from node to node hK, if yes, it is 1, otherwise it is 0, is represented as whether the container needs to be migrated from node hK to node , if yes, it is 1, otherwise it is 0, is the CPU resource request amount of the container , is the node Total CPU resource capacity; For containers RAM resource requests For nodes Total RAM resource capacity; This can be represented as a third constraint; Specifically, This represents a node. Container on Whether migration is required is set to 1 if migration is required and 0 if migration is not required. Therefore, this third constraint can be a node. Containers The total number of containers migrated into node hK must equal the number of nodes. Container on The number of containers that need to be migrated, that is, the third constraint, must ensure that the number of containers that need to be migrated is consistent with the number of migration operations actually performed.

[0120] This can be represented as the fourth constraint; Specifically, when Time, that is, container Not deployed to nodes Above, so nodes Container on The probability of being migrated is 0 (at this time, the node) There is no container (cm) on it; on the contrary, when Time, that is, node Container It may be migrated or it may not be migrated, so it can be 1 or 0.

[0121] This can be represented as the fifth constraint; Specifically, the same applies to this constraint; it is a constraint on the legality of the migration path. When the container cm is not on node hj, A value of 0 means that nodes that have never been deployed are prohibited from migrating out of the container. , It can be either 1 or 0, allowing migration out of this node.

[0122] The embodiment significantly improves the safety, balance and long-term benefits of container migration decisions by precisely constructing a state space and a multi-objective optimization training mechanism. First, the design of the state space actively excludes the first target node (overloaded or fragmented node), ensuring that the target node selection always meets the CPU and memory resource capacity constraints, thereby avoiding invalid migration operations from the source. Second, the joint reward function optimizes both the overall load balancing of the cluster (reducing the standard deviation of CPU / memory utilization) and the internal resource coordination of the node (minimizing resource fragmentation), addressing the shortcomings of traditional methods that only optimize a single objective. During training, the five constraints mechanism (such as migration path legality, operation consistency, etc.) strengthens the legality of the migration operation, ensuring the feasibility and stability of the solution. Finally, the optimization objective function minimizes key indicators such as cluster standard deviation and node imbalance, achieving long-term stable migration decisions, reducing resource overhead caused by repeated migration, and maximizing the long-term benefits of the cluster.

[0123] In some embodiments, the step of inputting the state space into a pre-trained container migration decision model based on Dueling DQN to obtain an action number includes: The input layer of the Dueling DQN-based container migration decision model receives a state space vector in the state space; The shared feature extraction layer of the Dueling DQN-based container migration decision model performs feature extraction on the state space vector, and then outputs a state value function through a pre-set first branch and an advantage function through a second branch; wherein the state value function is used to evaluate the overall state value of the cluster, and the advantage function is used to evaluate the relative advantage of each migration action; The merging layer of the Dueling DQN-based container migration decision model merges the state value function and the advantage function to obtain an action value function; The action number that maximizes the output value of the action value function is selected as the output.

[0124] In one specific embodiment, the Dueling DQN agent can be updated.

[0125] Traditional Q-learning algorithms cannot be applied to problems with excessively large state or action spaces due to the dimensionality of the Q-table. The core idea of the DQN algorithm is to use a deep neural network to approximate the Q-value function, which to some extent solves the problem of space explosion. Suppose the parameters of the neural network used to approximate the function Q are , i.e. the Q value of all possible actions a under each state S can be represented as . The neural network used to fit the function Q is called the Q network. According to the update rule of the Q-learning algorithm Q value function: ; S represents the selection of the container to be migrated, a represents the action of selecting the container, and the above formula uses the time difference learning target to incrementally update , for a set of data , the loss function of the Q network can be constructed in the form of mean square error: In addition to the Q network, the DQN algorithm has two important modules: experience replay and target network, which can help DQN achieve more stable and excellent performance. The DQN algorithm uses the experience replay method, which maintains a replay buffer to store the four-tuple data (state, action, reward, next state) sampled from the environment each time, and randomly samples some data from the replay buffer for training when training the Q network. This can make the samples meet the independent assumption and improve sample efficiency.

[0126] To solve the instability of neural network training, the DQN uses the idea of target network. Two sets of Q networks are used: 1. The original training network is used to calculate the term in the loss function and is updated using normal gradient descent method.

[0127] 2. The target network is used to calculate the term in the loss function, where represents the parameters in the target network. In order to make the update target more stable, the target network will not be updated every step. Specifically, the target network uses a relatively old set of parameters of the training network, and the training network is updated at every step of training, while the parameters of the target network are updated only once every C steps, i.e. . This makes the target network more stable than the training network.

[0128] Dueling DQN is an improved algorithm of DQN, and the network structure of Dueling DQN is as shown in Figure 6 : It only makes minor changes to the traditional DQN algorithm, but can greatly improve the performance of DQN. In reinforcement learning, the result of subtracting the state value function V from the action value function Q is defined as the advantage function A, i.e.: . Under the unified state, the sum of the advantage values of all actions is 0, because the expected value of the action value of all actions is the state value of this state. According to this, in Dueling DQN, the Q network is modeled as: ; wherein, is a state value function, and is the advantage function of taking different actions in this state, representing the difference of taking different actions; is a network parameter shared by the state value function and the advantage function, usually used in the first few layers of a neural network to extract features; and and are parameters of the state value function and the advantage function, respectively. In such a model, the neural network no longer directly outputs the Q value, but trains the last few layers of the neural network to output the state value function and the advantage function, respectively, and then sums them to obtain the Q value.

[0129] The present embodiment realizes significant advantages in container migration decision-making through a double-branch decision separation mechanism. First, the state value function independently evaluates the overall load balancing value of the cluster, avoiding the interference of action noise, thereby improving the accuracy of decision-making. The advantage function focuses on quantifying the relative benefits of each migration action, finely evaluating the improvement in load balancing due to target node selection, and achieving more accurate action-level evaluation. Second, the dynamic adaptability of the model is enhanced, and the shared feature extraction layer integrates real-time resource availability data, enabling adaptive learning and representation of the dynamic state of the cluster. The merging layer dynamically generates the optimal action value through function synthesis (Q = V + A), effectively matching complex and variable load scenarios. By maximizing the action value function output, it ensures that the migration decision optimizes the immediate load release while also considering the stability of long-term resources, thereby reducing the overhead caused by repeated migration. In addition, the double-branch structure significantly reduces the difficulty of model training, accelerates the convergence process (compared to single-branch DQN), and ensures the real-time nature of online decision-making, greatly improving the reliability of the project.

[0130] In one specific embodiment, a deep reinforcement learning-based container migration method in Kubernetes was experimented, and the experimental results were obtained.

[0131] It can be understood that container technology is an emerging virtualization solution that can package an application and its required resources (such as an operating system, libraries, configuration files, etc.) together, so that it can run on different operating systems without the need to reconfigure the environment. After the container is deployed on the container cluster, different application services in the container technology-based cloud environment have different preferences for resource usage, and these preferences change dynamically over time. This can cause dynamic changes in the load of running containers, which can lead to uneven node load in the cluster. Container migration refers to migrating a deployed container to run on other nodes, adjusting the mapping relationship between containers and cloud servers to achieve load balancing.

[0132] In the research of container migration using heuristic algorithms, the related technology proposes a container management strategy based on the Best-Fit algorithm to improve the resource utilization of the host and reduce server overload or underload. However, this method does not consider the migration cost when selecting the best host. A soft threshold container migration strategy based on the correlation of the dominant resource ratio is proposed for migration out, and a container placement algorithm based on the fairness of the dominant resource is proposed for migration in. The migration time strategy based on the resource fragmentation evaluation algorithm is proposed for the evaluation of the migration time. However, this algorithm does not make special optimization for the load balancing target. To solve the problem of large migration cost caused by the single target of container migration selection, the related technology proposes a container migration mechanism for edge network load balancing, which evaluates the edge network state by establishing a load differentiation matrix model and determines the container migration time. The improved ant colony system algorithm is used to obtain the container migration decision result. The migration algorithm using meta-heuristic algorithm is sensitive to parameters and cannot well face the complex and variable container cluster environment. However, the above migration algorithms still use rule-based or heuristic algorithms for migration, which cannot adapt to the dynamic changes of the cluster environment. In addition, the characteristics of the resource demand and resource usage of the service type carried by the container are not considered when selecting the migration container, which reduces the migration efficiency. These algorithms often only consider the overall load balancing level of the cluster, ignoring the load balancing between different resource dimensions in the node.

[0133] Reinforcement learning is a branch of machine learning that can be traced back to the artificial intelligence research in the 1950s and 1960s. At that time, researchers hoped to enable machines to learn and make decisions like humans, but traditional machine learning methods could not solve problems involving unknown environments. To solve this problem, researchers began to explore a new learning method, namely reinforcement learning.

[0134] Deep reinforcement learning is a machine learning method that organically combines the capabilities of deep learning and reinforcement learning. Deep reinforcement learning can effectively approximate complex functions using the capabilities of deep learning, and the learning capabilities of deep reinforcement learning have been significantly improved through the supervision of the fusion of deep learning and reinforcement learning processes. In recent years, reinforcement learning (RL) combined with deep neural networks has been successfully applied in various fields, such as dynamic decision-making, such as autonomous vehicles, autonomous robots, and games.

[0135] Load balancing methods based on deep reinforcement learning have been applied in computer networks, Internet of Things, and cloud task scheduling: deep reinforcement learning algorithms can make reasonable decisions in the face of dynamic changes in load in large-scale, complex cloud environments, achieving the goal of load balancing.

[0136] All experiments are carried out in the Windows 10 operating system, Python 3.6 environment, and the Pytorch version is 1.9.0. The experiment is carried out in a simulation environment. The simulation environment used or referred to in related research uses the widely used cloud computing environment simulation tool CloudSim. CloudSim can provide effective simulation of real cloud computing environments. The specific implementation of CloudSim is referred to in the present embodiment, and a simulation platform suitable for the Python environment is developed on this basis, which can simulate the running process of the container cluster in the real environment.

[0137] The present embodiment uses the data set published in AlibabaClusterTraceProgram as the experimental data set. The selected attributes and parameter values in the migration experiment are shown in Table 1: Three types of heterogeneous nodes are set in the present experiment, and the number of resources possessed by the nodes is different. The configurations of the three types of nodes are shown in Table 1.

[0138] Table 1: Container configuration table

[0139] The main parameters of the DRLCM container migration algorithm are shown in the following table. The neural network part of the Dueling DQN uses a three-layer network structure and uses the Relu function as the activation function. In terms of reinforcement learning parameters, the size of the replay unit is set to 10000, and the deep neural network is updated only when there are at least 500 data in the replay unit. The specific parameter settings of the experiment are shown in Table 2.

[0140] Table 2: Experimental parameter settings

[0141] In order to fully compare the algorithm, the present embodiment selects four comparison methods, covering rule-based container migration methods, heuristic scheduling methods and deep reinforcement learning-based container migration methods: 1. Random scheduling algorithm (Random): Select a rule-based random scheduling algorithm as the benchmark method of the experiment. This method randomly selects containers on the node that needs to be migrated, and randomly selects the migration node.

[0142] 2. LLR scheduling algorithm (LLR): LLR container migration algorithm is a heuristic container migration algorithm. The LLR algorithm is improved on the basis of the Sandpiper algorithm and applied to the container migration scene. The basic idea of this algorithm is to migrate the containers on the node with the heaviest load to the node with the lightest load, and the LLR algorithm considers the load influence caused by multiple computing resources.

[0143] 3. DQLCM algorithm (DQLCM): DRLCM is a container migration algorithm based on the deep reinforcement learning DQN algorithm.

[0144] 4. ATCM algorithm: heuristic container migration method aiming to restore the load balance of the container cluster.

[0145] The comparison experiment is the number of migrations of different migration selection strategies. The following strategies are selected for comparison of the number of migrations, and the experiment is carried out under the condition of 2144 containers and 10 nodes: 1. DRLCM migration container selection strategy: combining K-Means container clustering and maximum utilization (MU) strategy to select the container to be migrated, which is the migration container selection strategy proposed in this embodiment.

[0146] 2. Maximum utilization selection strategy (MaxUsage): selecting the container with the maximum current resource utilization as the container to be migrated; 3. ATCM container migration selection strategy: selecting the container that can minimize the absolute value of the node load after migration and the average load of the cluster as the container to be migrated.

[0147] 4. LLR selection strategy: selecting the container with the maximum VSR value as the container to be migrated; 5. Random strategy: randomly selecting a container as the migration container.

[0148] Table 3 shows the number of container migrations and the impact on the load balance of the container cluster when different migration strategies are used. The number of migrations of the Random strategy is the largest, reaching 1607 times. The container migration selection strategy of the algorithm DRLCM proposed in this embodiment has the least number of migrations, which is 1033 times, compared with other methods. In particular, compared with the MU algorithm alone and the container selection strategy combining the K-Means container clustering algorithm proposed in this embodiment and the MU algorithm, the number of migrations of the MU algorithm is 1153 times, and the selection strategy proposed in this embodiment reduces the number of migrations by 10.4%. This is because, compared with the MU algorithm, the selection strategy of the K-Means container clustering can more specifically select the container to be migrated according to the load of the current cluster and the difference in resource utilization of the containers, thereby reducing the number of migrations.

[0149] Table 3: Migration strategy experiment results

[0150] ​​​​In terms of the influence of different migration strategies on the load balancing of the container cluster, the different migration strategies are combined with the DRLCD agent to verify the performance of the migration strategy based on the DRLCM algorithm proposed in the embodiment. As can be seen from Table 3, the use of the DRLCM migration strategy of the embodiment makes the cluster have the minimum value of the CPU and memory utilization standard deviation and the node internal load imbalance degree before and after migration compared with other migration strategies, which shows that the migration container selection strategy proposed in the embodiment can more effectively restore the load balancing state of the cluster. This is because the migration strategy proposed in the embodiment considers the type of the container bearing application service and its consumption of resources when in use, and can more reasonably and more targetedly select the container to be migrated according to the load state of the cluster; at the same time, the clustering result can also show that the containers bearing the same type of application service have similar consumption of resources in the future period of time, and the reasonable migration of the containers can make different types of containers run on one node, so that the container cluster will maintain a load balanced state after migration.

[0151] Figure 7 The cumulative reward curves of different algorithms using the same reward function value are shown. The higher the cumulative reward is, the better the performance in the optimization target of load balancing is. Among them, the cumulative reward curves of the DRLCM and DQLCM algorithms quickly converge, verifying the effectiveness of the reward function design proposed in the embodiment.

[0152] When the reward function converges, the cumulative reward value of the DRLCM based on the DuelingDQN method proposed in the embodiment is 34% higher than that of the DQLCM based on the DQN method, because the DuelingDQN algorithm can better learn the difference in the load balancing state caused by different actions than the DQN algorithm, and can obtain a higher reward value in learning.

[0153] The standard deviation of the CPU and memory utilization of the cluster nodes is used as an index to measure the overall load balancing degree of the cluster in the embodiment, and the smaller the utilization standard deviation is, the more balanced the load of the cluster is. The standard deviation of the CPU utilization before and after migration is shown in FIG. 6. Figure 8 As can be seen from the figure, the algorithms can all reduce the standard deviation of the CPU resource utilization of the cluster through the operation of container migration, and improve the load balancing level of the container cluster. The DRLCM container migration algorithm proposed in the embodiment performs best in improving the overall load balancing level of the cluster, which is at most 40% higher than the DQLCM algorithm based on the DQN. This is because the DuelingDQN agent can make more action choices that are beneficial to improving the load balancing degree than the DQN agent when migrating the containers.

[0154] The results of the standard deviation of memory utilization before and after migration are shown in Figure 9 Similar to the results of the standard deviation of CPU utilization, these algorithms can all reduce the standard deviation of memory utilization through container migration operations, improving the load balancing level of the container cluster. The DRLCM container migration algorithm proposed in this embodiment performs best in improving the overall load balancing level of the cluster, improving by at most 25% compared to the DQLCM algorithm based on DQN, and as the number of nodes increases, the load balancing effect of the DRLCM algorithm compared to the DQLCM algorithm is more obvious. This is because the network structure of DuelingDQN also outputs the action advantage value between different actions, and as the action space increases, the advantage of this approach compared to the DQN algorithm will be more obvious. From Figure 9 It can be seen from Figure 10 that the DRLCM container migration algorithm can achieve load balancing of the container cluster in multiple resource dimensions.

[0155] The experimental results on the index of node internal load imbalance are shown in Figure 10 It can be seen from the accompanying Figure 10 that the five algorithms can all reduce the node internal load imbalance. The DRLCM algorithm optimizes the node internal load imbalance when designing the reward function, so it achieves the best effect in different node numbers. Similarly, as the number of nodes increases, the DRLCM algorithm performs better in reducing the node internal load imbalance, and compared to the DQLCM algorithm which also uses deep reinforcement learning algorithm, the DRLCM algorithm improves by at most 25% compared to the DQLCM algorithm when the number of nodes is 10. The random algorithm has a large randomness, so the results under different experimental conditions also differ greatly, and it cannot achieve a stable result on the index of node load imbalance. The LLR algorithm does not consider the node internal load imbalance when designing, so the effect is not ideal.

[0156] The embodiment provides a container migration method, comprising: determining a first target node and a second target node; wherein the first target node and the second target node are determined by using the method for determining a target node provided in the above embodiment; migrating a to-be-migrated container in the first target node to the second target node.

[0157] According to the embodiment of the present application, an electronic device is provided. The electronic device in the present embodiment can include one or more of the following components: a processor, a network interface, a memory, a non-volatile memory, and one or more application programs, wherein the one or more application programs can be stored in the non-volatile memory and configured to be executed by the one or more processors, and the one or more programs are configured to perform the method described in the foregoing method embodiment.

[0158] According to an embodiment of the present application, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is executed by a computer, and the computer program causes the computer to execute the method described in any of the above embodiments.

[0159] According to an embodiment of the present application, a computer program product is provided, and the computer program product contains instructions. The instructions are executed by a computer, and the instructions cause the computer to execute a method described in any of the above embodiments.

[0160] Optionally, the specific examples in the embodiments can refer to the examples described in the above embodiments, and the embodiments will not be described here.

[0161] The serial numbers of the above embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0162] In the above embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0163] The above is only the preferred embodiment of the present application, and it should be pointed out that for ordinary skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present application.

Claims

1. A method of determining a target node, characterized by, The method is applied to a controller in a node cluster, the node cluster comprising at least two nodes, the nodes being deployed with at least two containers; the method comprising: in response to detecting a target event, obtaining running data of a first target node; wherein the target event is an event for characterizing triggering container migration performed on the first target node, and the running data comprises historical running data and real-time running data in a preset time period; based on the historical running data of the first target node and a preset container type classification algorithm, determining a container type of each container in the first target node; based on the container type, the real-time running data and the target event, determining a to-be-migrated container identifier; based on the to-be-migrated container identifier, obtaining corresponding resource requirement data and real-time resource availability data of the node cluster; based on the resource requirement data, the real-time resource availability data and a preset container migration decision model, determining a second target node; wherein the second target node is a node for receiving the to-be-migrated container.

2. The determination method according to claim 1, characterized in that, Before the step of obtaining the running data of the first target node in response to detecting the target event, the method further comprises: traversing current running data of each node in the node cluster; determining whether a node triggers a target event; wherein the target event comprises at least a first target event and a second target event; wherein the first target event indicates that CPU utilization in the node is greater than a CPU utilization threshold and / or memory utilization is greater than a CPU utilization threshold; the second target event indicates that an absolute value of a difference between CPU utilization and memory utilization is greater than a difference threshold value; in a case where it is determined that a node triggers a target event, the corresponding node is determined as the first target node.

3. The determination method according to claim 2, characterized in that, The step of determining the container type of each container in the first target node based on the historical running data of the first target node and a preset container type classification algorithm comprises: inputting the historical running data of the first target node into a preset K-Means clustering algorithm to obtain the container type of each container in the first target node; wherein the container type comprises at least a first container type and a second container type; wherein the first container type indicates a container with urgent demand for CPU resources, and the second container type indicates a container with urgent demand for memory resources.

4. The determination method according to claim 3, characterized in that, The step of determining the to-be-migrated container identifier based on the container type, the real-time running data and the target event comprises: using a preset maximum resource utilization strategy, and the container type, the real-time running data and the target event to determine the to-be-migrated container identifier; wherein the maximum resource utilization strategy indicates a strategy of selecting a container with the highest real-time resource utilization from a container set of a corresponding container type as a to-be-migrated object according to a resource type triggered by the target event.

5. The determination method according to claim 4, characterized in that, The step of using the preset maximum resource utilization strategy, and the container type, the real-time running data and the target event to determine the to-be-migrated container identifier comprises: determining the type of the target event, and determining the container type to be selected based on the type of the target event; wherein, if the target event is a first target event, determining the container type to be selected according to the resource type triggered in the first target event; if the target event is a second target event, determining the container type to be selected according to the difference between the CPU utilization and the memory utilization in the first target node; wherein, if the first target event is triggered by the CPU utilization being greater than a CPU utilization threshold, determining the container type to be selected as the first container type; if the first target event is triggered by the memory utilization being greater than a memory utilization threshold, determining the container type to be selected as the second container type; if the difference is positive and the absolute value is greater than a difference threshold, determining the container type to be selected as the first container type; if the difference is negative and the absolute value is greater than a difference threshold, determining the container type to be selected as the second container type; from the container corresponding to the determined container type, selecting the container with the highest corresponding resource utilization in the real-time running data, and recording the container identifier as the container identifier to be migrated; wherein, when the container type to be selected is the first container type, the corresponding resource is CPU; when the container type to be selected is the second container type, the corresponding resource is memory.

6. The determination method of claim 1, wherein, The step of determining the second target node based on the resource requirement data, the real-time resource availability data, and the preset container migration decision model comprises: inputting the resource requirement data corresponding to the container identifier to be migrated and the real-time resource availability data of the node cluster into the pre-trained container migration decision model based on Dueling DQN respectively, to obtain the identifier of the second target node.

7. The determination method according to claim 6, characterized in that, The step of inputting the resource requirement data corresponding to the container identifier to be migrated and the real-time resource availability data of the node cluster into the pre-trained container migration decision model based on Dueling DQN respectively, to obtain the identifier of the second target node, comprises: constructing a state space based on the resource requirement data and the real-time resource availability data; wherein, in the state space, the real-time resource availability data is the real-time resource availability data of other nodes except the first target node; input the state space into a pre-trained container migration decision model based on a Dueling DQN to obtain an action number; wherein the container migration decision model based on the Dueling DQN is trained by a preset reward function and an optimization objective function in a pre-training process; wherein the reward function is a joint reward function for representing a node internal load balancing reward and a cluster overall load balancing reward, and the optimization objective function is used for jointly minimizing a first index, a second index and a third index under the conditions of satisfying a first constraint, a second constraint, a third constraint, a fourth constraint and a fifth constraint; wherein the first index is an index for representing a cluster node CPU utilization rate standard deviation, the second index is an index for representing a cluster node memory utilization rate standard deviation, and the third index is an index for representing a node internal load imbalance degree; the first constraint is used for representing a CPU resource capacity constraint, the second constraint is used for representing a memory resource capacity constraint, the third constraint is used for representing a migration number consistency constraint, the fourth constraint is used for representing a migration operation legality constraint, and the fifth constraint is used for representing a migration path legality constraint; map the action number to a node identifier as an identifier of a second target node.

8. The determination method according to claim 7, characterized in that, The step of inputting the state space into the pre-trained container migration decision model based on the Dueling DQN to obtain the action number comprises: an input layer of the container migration decision model based on the Dueling DQN receives a state space vector in the state space; a shared feature extraction layer of the container migration decision model based on the Dueling DQN performs feature extraction on the state space vector, and then outputs a state value function through a preset first branch and outputs an advantage function through a second branch; wherein the state value function is used for evaluating a cluster overall state value, and the advantage function is used for evaluating a relative advantage of each migration action; a merging layer of the container migration decision model based on the Dueling DQN merges the state value function and the advantage function to obtain an action value function; select an action number that makes an output value of the action value function maximum as an output.

9. A method of migration of a container, characterized in that, comprise: determine a first target node and a second target node; wherein the first target node and the second target node are determined by using a target node determination method according to any one of claims 1-8; migrate a to-be-migrated container in the first target node to the second target node.

10. An electronic device, comprising: comprise: a memory and one or more processors in communication connection with the memory; the memory stores instructions executable by the one or more processors, and the instructions are executed by the one or more processors to enable the one or more processors to implement the method according to any one of claims 1-8.

Images