Distributed node switching method and device, electronic equipment and storage medium

Abstract

The invention relates to the technical field of cloud computing, and discloses a distributed node switching method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining historical operation data of a distributed system; based on the historical operation data, training an initial node switching state transition model of the distributed system to obtain a target node switching state transition model; based on the historical operation data and the target environment model, training an initial node switching model of the distributed system to obtain a target node switching model; inputting the current operation data of the distributed system into the target node switching model; and based on the target node switching model, determining a current node switching strategy of the distributed system according to the current operation data. According to the embodiment of the invention, the target node switching model is trained by using the target environment model obtained by training, so that the node switching strategy output by the target node switching model can enable the system environment state to be optimal, and the performance of the distributed system is improved.

Description

Technical Field

[0001] The present application relates to the field of cloud computing technology, and in particular to a distributed node switching method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of cloud computing technology, the scale of distributed systems continues to grow. While integrating more software and hardware resources brings higher computing performance and storage capacity, it also brings problems such as load imbalance.

[0003] In related technologies, in order to solve the problem of load imbalance, a load threshold for each node is usually set based on the resource configuration of the node. When the actual load of any node exceeds the load threshold, the computing task on the node is switched to an idle node or a node with a lower load.

[0004] However, the configuration of each node in a distributed system is different, and the existing computing task switching strategy cannot guarantee that the computing task will be switched to the best target switching node. Therefore, there is an urgent need for a distributed node switching method that can determine the best target node, which is of great significance to improving the performance of distributed systems. Summary of the invention

[0005] The present application provides a distributed node switching method, device, electronic device and storage medium to solve the defects of related technologies such as the inability to ensure that computing tasks are switched to the best target switching node.

[0006] The first aspect of the present application provides a distributed node switching method, comprising:

[0007] Obtain historical operation data of distributed systems;

[0008] Based on the historical operation data, the initial node switching state transfer model of the distributed system is trained to obtain a target node switching state transfer model; wherein the target node switching state transfer model includes a target environment model, and the environment model is used to describe the state change of each node in the distributed system;

[0009] Based on the historical operation data and the target environment model, an initial node switching model of the distributed system is trained to obtain a target node switching model;

[0010] Inputting the current operation data of the distributed system into the target node switching model;

[0011] Based on the target node switching model and according to the current operating data, a current node switching strategy of the distributed system is determined; wherein the current node switching strategy at least includes a target switching node.

[0012] In an optional implementation, the training of the initial node switching state transition model of the distributed system based on the historical operation data to obtain the target node switching state transition model includes:

[0013] Dividing the historical operation data into time periods to obtain the operation status information of each node at each time period;

[0014] Taking the running status data of each node at any said moment as input, inputting it into the initial strategy model, and obtaining the first node switching training strategy of any node at that moment output by the initial strategy model;

[0015] Based on the initial environment model, according to the first node switching training strategy and the current environment state information, determine the first environment state prediction information after the node is switched according to the first node switching training strategy;

[0016] Determining a first predicted reward value corresponding to the first node switching training strategy according to the first environmental state prediction information and the current environmental state information;

[0017] Determine, based on the historical operation data, actual information of the environment state and an actual reward value after node switching according to the first node switching training strategy;

[0018] Iteratively optimizing the model parameters of the initial strategy model according to the first node switching training strategy, the actual reward value, the current environment state information and the actual environment state information to obtain a target strategy model;

[0019] Determine, based on the historical operation data, actual information of the environment state and an actual reward value after node switching according to the first node switching training strategy;

[0020] Iteratively optimizing the model parameters of the initial environment model according to the first predicted reward value, the first environment state prediction information, the actual environment state information and the actual reward value to obtain a target environment model;

[0021] The target node switching state transfer model includes the target strategy model and the target environment model.

[0022] In an optional implementation, determining the first predicted reward value corresponding to the first node switching training strategy according to the first environmental state prediction information and the current environmental state information includes:

[0023] Based on the following formula, the first predicted reward value corresponding to the node switching training strategy is determined:

[0024] r i′ =α1K1+α2K2-α3M

[0025] Among them, r i ′ represents the first predicted reward value corresponding to the first node switching training strategy of node i in the distributed system, K1 represents the system power consumption optimization value represented by the first environmental state prediction information and the current environmental state information, K2 represents the system average resource utilization optimization value represented by the first environmental state prediction information and the current environmental state information, M represents the number of computing tasks switched by node i, and α1, α2 and α3 represent preset coefficients.

[0026] In an optional implementation, the training of the initial node switching model of the distributed system based on the historical operation data and the target environment model to obtain the target node switching model includes:

[0027] Dividing the historical operation data into time periods to obtain the operation status information of each node at each time period;

[0028] Taking the running status data of each node at any said moment as input, inputting it into the initial node switching model, and obtaining the second node switching training strategy of any node of the initial node switching model at that moment;

[0029] Based on the target environment model, according to the second node switching training strategy and the current environment state information, determine the second environment state prediction information after the node is switched according to the second node switching training strategy;

[0030] Determining a second predicted reward value corresponding to the second node switching training strategy according to the second environmental state prediction information and the current environmental state information;

[0031] According to the second node switching training strategy, the second predicted reward value, the current environment state information and the second environment state prediction information, the model parameters of the initial node switching model are iteratively optimized to obtain the target node switching model.

[0032] In an optional implementation, the iterative optimization of the model parameters of the initial node switching model according to the second node switching training strategy, the second predicted reward value, the current environment state information and the second environment state prediction information to obtain the target node switching model includes:

[0033] Determine a performance optimization reward value of the second node switching training strategy according to the current environment state information and the second environment state prediction information;

[0034] Determining a comprehensive reward value of the second node switching training strategy according to the second prediction reward value and the performance optimization reward value;

[0035] According to the second node switching training strategy and the comprehensive reward value of the second node switching training strategy, the model parameters of the initial node switching model are iteratively optimized to obtain a target node switching model.

[0036] In an optional implementation, the determining, based on the target node switching model and according to the current operating data, a current node switching strategy of the distributed system includes:

[0037] Extracting features from the current operation data to obtain current operation features of the distributed system;

[0038] Based on a recurrent neural network, determining a current running hidden feature according to the current running feature;

[0039] Inputting the currently running hidden features into the policy network and the graph neural network respectively;

[0040] Based on the policy network, and according to the currently running hidden features, determining whether each node in the distributed system needs to be switched;

[0041] When any node in the distributed system needs to be switched, based on the graph neural network and according to the currently running hidden features, a current node switching strategy of the node is determined;

[0042] Among them, the target node switching model includes the recurrent neural network, the policy network and the graph neural network.

[0043] A second aspect of the present application provides a distributed node switching device, comprising:

[0044] The acquisition module is used to obtain the historical operation data of the distributed system;

[0045] A first training module is used to train the initial node switching state transfer model of the distributed system based on the historical operation data to obtain a target node switching state transfer model; wherein the target node switching state transfer model includes a target environment model, and the environment model is used to describe the state change of each node in the distributed system;

[0046] A second training module is used to train the initial node switching model of the distributed system based on the historical operation data and the target environment model to obtain a target node switching model;

[0047] An input module, used for inputting the current operation data of the distributed system into the target node switching model;

[0048] A switching module is used to determine a current node switching strategy of the distributed system based on the target node switching model and the current operating data; wherein the current node switching strategy at least includes a target switching node.

[0049] A third aspect of the present application provides an electronic device, comprising: at least one processor and a memory;

[0050] The memory stores computer-executable instructions;

[0051] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method described in the first aspect and various possible designs of the first aspect.

[0052] A fourth aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the method described in the first aspect and various possible designs of the first aspect are implemented.

[0053] A fifth aspect of the present application provides a computer program product, including computer instructions, which are used to enable a computer to execute the method described in the first aspect and various possible designs of the first aspect.

[0054] The technical solution of this application has the following advantages:

[0055] The present application provides a distributed node switching method, device, electronic device and storage medium, the method comprising: obtaining historical operation data of a distributed system; based on the historical operation data, training the initial node switching state transfer model of the distributed system to obtain a target node switching state transfer model; wherein the target node switching state transfer model includes a target environment model, and the environment model is used to describe the state changes of each node in the distributed system; based on the historical operation data and the target environment model, training the initial node switching model of the distributed system to obtain a target node switching model; inputting the current operation data of the distributed system into the target node switching model; based on the target node switching model, determining the current node switching strategy of the distributed system according to the current operation data; wherein the current node switching strategy at least includes a target switching node. The method provided by the above scheme trains the target node switching model by using the target environment model obtained by training, so as to ensure that the node switching strategy output by the target node switching model can make the system environment state optimal, that is, according to the node switching strategy, the computing task of the node to be switched can be switched to the optimal target switching node, thereby improving the performance of the distributed system. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the following is a brief introduction to the drawings required for use in the embodiments or the related technical descriptions. Obviously, the drawings described below are some embodiments of the present application, and a person skilled in the art can also obtain other drawings based on these drawings.

[0057] Figure 1 This is a schematic diagram of the structure of the distributed node switching system based on the embodiment of the present application;

[0058] Figure 2 A schematic diagram of a flow chart of a distributed node switching method provided in an embodiment of the present application;

[0059] Figure 3 A schematic diagram of the application architecture of the distributed node switching method provided in an embodiment of the present application;

[0060] Figure 4 A schematic diagram of the structure of a node switching state transfer model provided in an embodiment of the present application;

[0061] Figure 5 A schematic diagram of the structure of a node switching model provided in an embodiment of the present application;

[0062] Figure 6 A schematic diagram of the overall process of the distributed node switching method provided in an embodiment of the present application;

[0063] Figure 7 A schematic diagram of the training process of the node switching model provided in an embodiment of the present application;

[0064] Figure 8 A schematic diagram of the structure of a distributed system provided in an embodiment of the present application;

[0065] Fig. 9 A schematic diagram of the structure of an exemplary attention mechanism network provided in an embodiment of the present application;

[0066] Fig.10 A schematic diagram of the structure of a distributed node switching device provided in an embodiment of the present application;

[0067] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0068] The above drawings have shown clear embodiments of the present application, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present disclosure in any way, but to illustrate the concepts of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0070] First, the terms involved in this application are explained:

[0071] Reinforcement Learning: Reinforcement Learning (RL) is a machine learning technique that allows an agent to interact with the environment and learns a sequential task decision-making strategy that enables the agent to obtain the maximum target return based on the rewards provided by the environment.

[0072] Graph Attention Networks: Graph Attention Networks (GAT) is a graph neural network that uses the attention mechanism to calculate the importance weights of associated nodes, identify the relevance of associated nodes under resource-constrained conditions, and accurately aggregate node information based on attention weights.

[0073] In addition, the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. In the description of the following embodiments, the meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.

[0074] In recent years, cloud computing has attracted widespread attention as an emerging machine learning technology, setting off a huge information technology revolution and has been widely used in various fields such as medical care, finance, education, and government affairs. With the vigorous development of cloud computing technology, the scale of distributed systems continues to grow. While integrating more software and hardware resources brings higher computing performance and storage capacity, it also brings problems such as load imbalance. In a distributed system, the performance of different service nodes varies, and the resources used by different service requests vary, resulting in load imbalance between nodes. At the same time, as the number of service nodes in the system increases, the number of node failures also increases. This not only seriously reduces the user experience, but also affects the stability of the distributed system and reduces the resource utilization of the system.

[0075] Most of the existing node switching strategies are to manually set node switching rules, or monitor node status statistical thresholds to trigger node switching. For example, save the correspondence between each service and the node where the service is located, as well as the network port corresponding to each service; comprehensively consider whether the node's network port traffic and current load exceed the preset threshold, determine the busyness of the node, and then switch the service running on the busy node to the idle node, which alleviates the load pressure of the busy node to a certain extent. Another example is to register the PCB information of different service processes through the maintenance service information table, and distribute the load based on the pre-configured load balancing algorithm; at the same time, regularly monitor the operation of the distributed system, determine whether the node is overloaded through the load threshold, and switch the service of the overloaded node to the non-overloaded node; according to the recorded service operation information, adaptively change the overload threshold of each node, and achieve load balancing to a certain extent.

[0076] The existing node switching solutions have the following three main defects:

[0077] (1) Node switching is not flexible enough

[0078] In existing methods, the node switching threshold is basically manually set according to the resource configuration of the node, and whether it is overloaded is determined by the rule of whether the load threshold is reached. However, the manually set thresholds and node switching rules are often only for specific systems. When the system environment changes, they need to be reconfigured, lacking flexible adaptive capabilities; when a node is close to the load threshold and other nodes are idle, the rules cannot be triggered to autonomously select idle nodes for load balancing, reducing resource utilization; in addition, manually setting node switching rules may result in omissions, especially for large-scale distributed systems.

[0079] (2) Low node switching efficiency and high resource consumption cost

[0080] Adjusting the load based on system resource status statistics requires storing a large amount of distributed system historical data and processing and analyzing this data, which requires high-performance storage devices and complex machine learning algorithms, resulting in high costs. For dynamically changing large-scale networks and nodes, the system detects node overload and responds with node switching, which is difficult to meet the real-time requirements of the system and has low efficiency.

[0081] (3) It is difficult to select the optimal node switching target

[0082] For overloaded nodes, the current node switching strategy usually selects nodes with lower load for switching, or comprehensively considers factors such as the node's network performance, security, switching cost, node architecture, etc., and traverses all nodes to select the node switching target. It is difficult to comprehensively consider all indicator factors to select the optimal node switching target.

[0083] In view of the above problems, the embodiments of the present application provide a distributed node switching method, device, electronic device and storage medium, the method comprising: obtaining historical operation data of a distributed system; based on the historical operation data, training the initial node switching state transfer model of the distributed system to obtain a target node switching state transfer model; wherein the target node switching state transfer model includes a target environment model, and the environment model is used to describe the state changes of each node in the distributed system; based on the historical operation data and the target environment model, training the initial node switching model of the distributed system to obtain a target node switching model; inputting the current operation data of the distributed system into the target node switching model; based on the target node switching model, determining the current node switching strategy of the distributed system according to the current operation data; wherein the current node switching strategy at least includes a target switching node. The method provided by the above scheme trains the target node switching model by using the target environment model obtained by training, so as to ensure that the node switching strategy output by the target node switching model can make the system environment state optimal, that is, according to the node switching strategy, the computing task of the node to be switched can be switched to the optimal target switching node, thereby improving the performance of the distributed system.

[0084] The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0085] First, the structure of the distributed node switching system on which this application is based is described:

[0086] The distributed node switching method, device, electronic device and storage medium provided in the embodiments of the present application are suitable for formulating a node switching strategy for each node in a distributed system to ensure that the computing task of the node to be switched is switched to the best target switching node. Figure 1 As shown, it is a structural diagram of the distributed node switching system based on the embodiment of the present application, which mainly includes a distributed system, a data acquisition device and a distributed node switching device. Among them, the distributed system includes multiple nodes, the data acquisition device is used to collect the current operation data of the distributed system, and send the collected current operation data to the distributed node switching device, and the distributed node switching device determines the current node switching strategy of the distributed system based on the received current operation data to optimize the performance of the distributed system.

[0087] The embodiment of the present application provides a distributed node switching method for formulating a node switching strategy for each node in a distributed system to ensure that the computing task of the node to be switched is switched to the best target switching node. The execution subject of the embodiment of the present application is an electronic device, such as a server, a desktop computer, a laptop computer, a tablet computer, and other electronic devices that can be used to formulate a node switching strategy for a distributed system.

[0088] like Figure 2 FIG. 1 is a flow chart of a distributed node switching method provided in an embodiment of the present application, the method comprising:

[0089] Step 201, obtaining historical operation data of the distributed system.

[0090] Among them, the node switching history data based on manual rules and statistical data in the distributed system is collected to obtain the historical operation data of the distributed system. In the actual distributed system, monitoring tools or log analysis tools are used to collect the historical operation data of all nodes in the system, including but not limited to the number of system nodes, node CPU utilization, memory utilization, storage resource usage, network load, power consumption, node switching time, node switching target, system total power consumption, system average resource utilization, and node failure conditions. Specifically, the collected data should be correct, comprehensive, and continuous to ensure accurate model training.

[0091] Furthermore, in order to further ensure the data quality of historical operation data used in subsequent training, the collected data can be cleaned to remove errors, anomalies or missing values ​​to ensure the integrity and accuracy of node operation data. If the training data comes from different distributed systems or different node switching strategies, the training data should be converted to unify the data format and content.

[0092] Step 202: Based on historical operation data, the initial node switching state transfer model of the distributed system is trained to obtain a target node switching state transfer model.

[0093] The target node switching state transfer model includes a target environment model, and the environment model is used to describe the state changes of each node in the distributed system.

[0094] Specifically, a reinforcement learning algorithm can be used to train the initial node switching state transfer model of the distributed system based on historical operation data, so as to continuously optimize the node switching state transfer model and obtain the target node switching state transfer model.

[0095] Step 203: Based on the historical operation data and the target environment model, the initial node switching model of the distributed system is trained to obtain the target node switching model.

[0096] It should be noted that the target environment model in the target node switching state transfer model obtained through the above training is close to the characteristics of the real distributed system. By training the initial node switching model of the distributed system based on historical operation data and the target environment model, the reliability of the node switching model training process is guaranteed and the accuracy of the final target node switching model is improved.

[0097] Step 204: input the current operation data of the distributed system into the target node switching model.

[0098] Among them, the current operating data of the distributed system includes the number of system nodes, the current CPU utilization of each node, the current memory utilization, the current storage resource usage, the current network load, the current power consumption, the current system total power consumption, the current system average resource utilization, and the current node failure status.

[0099] Step 205 : Based on the target node switching model and the current operation data, determine the current node switching strategy of the distributed system.

[0100] The current node switching strategy at least includes a target switching node.

[0101] Specifically, the target node switching model performs model calculation based on the current operating data obtained to determine whether each node in the system needs to perform node switching. If node switching is required, a node switching strategy for each node is formulated to obtain the current node switching strategy of the distributed system.

[0102] Furthermore, according to the current node switching strategy of the distributed system, the computing task on the node to be switched can be switched to the target switching node, so as to respond to the computing task based on the resources of the target switching node.

[0103] Among them, Figure 3 As shown, it is a schematic diagram of the application architecture of the distributed node switching method provided in an embodiment of the present application. The architecture includes four parts: training data collection and processing, training node switching state transition model, training node switching model and node switching model application.

[0104] Based on the above embodiment, as an implementable manner, in one embodiment, based on historical operation data, the initial node switching state transition model of the distributed system is trained to obtain the target node switching state transition model, including:

[0105] Step 2021, dividing the historical operation data into time periods to obtain the operation status information of each node at each time period;

[0106] Step 2022, taking the running status data of each node at any time as input, inputting it into the initial strategy model, and obtaining the first node switching training strategy of any node at that time output by the initial strategy model;

[0107] Step 2023, based on the initial environment model, according to the first node switching training strategy and the current environment state information, determine the first environment state prediction information after the node is switched according to the first node switching training strategy;

[0108] Step 2024, determining a first predicted reward value corresponding to the first node switching training strategy according to the first environmental state prediction information and the current environmental state information;

[0109] Step 2025, determining actual information of the environment state and actual reward value after node switching according to the first node switching training strategy based on historical operation data;

[0110] Step 2026, iteratively optimizing the model parameters of the initial strategy model according to the first node switching training strategy, the actual reward value, the current environment state information and the actual environment state information to obtain the target strategy model;

[0111] Step 2027, iteratively optimize the model parameters of the initial environment model according to the first predicted reward value, the first environment state prediction information, the environment state actual information and the actual reward value to obtain the target environment model.

[0112] The state transfer model for switching to the target node includes a target strategy model and a target environment model.

[0113] Specifically, an experience pool D for training the node switching state transition model can be constructed based on the data that has been cleaned and converted. The historical operation status information of each node is divided according to time, and a data sequence of each node that is continuous in time is obtained, that is, the operation status information of each node at each time is obtained. Among them, each piece of operation status information includes the node's CPU utilization, memory utilization, storage load, network load, power consumption, whether node switching occurs, node switching target, total power consumption of the system, power consumption reduction percentage, and system average resource utilization.

[0114] Specifically, Figure 4 As shown, it is a schematic diagram of the structure of the node switching state transfer model provided in the embodiment of the present application. The strategy model in the node switching state transfer model adopts a model-based reinforcement learning algorithm, interacts with the environment through trial and error, and continuously adjusts the strategy according to the rewards or penalties of the environment feedback. Strategy Model The state of the agent (node) is input, and the output is the probability distribution of each action taken by the agent. The action selector uses ∈-greedy to control the selection of actions. After executing the selected action, the reward is calculated, and then the policy model is updated. For this model, the parameters that need to be initialized are the policy model The network parameters w, the parameters ∈ of the action selector, the learning rate α and the discount factor γ of reinforcement learning.

[0115] The environment model is a model used to describe the dynamics of the distributed system environment, which can help the intelligent agent better understand the environment and make decisions, thereby improving learning efficiency and performance. The input is the state of the environment and the currently executed action, and the output is the predicted next state of the environment and the reward. The model is initialized by randomly sampling from a uniform distribution or a normal distribution.

[0116] Before the strategy model has learned a good strategy, it cannot be trained in a real distributed system, otherwise it will affect the operation of the system. Therefore, the environment model can be used as a simulated distributed system environment for subsequent model training, and the training of subsequent node switching models can be completed without affecting the operation of the real system.

[0117] Specifically, the distributed system environment state and the state of the intelligent agent, that is, the historical operation data, are collected and input into the strategy model after feature extraction to obtain the probabilities of all possible actions. The optimal action under the current strategy is output and executed through ∈-greedy greed.

[0118] Specifically, in one embodiment, taking into account the performance difference of the system before and after the node switching, to ensure that the performance of the distributed system can be improved after the node is completed, the first predicted reward value corresponding to the node switching training strategy can be determined based on the following formula:

[0119] r i ′ =α1K1+α2K2-α3M

[0120] Among them, r i ′ represents the first predicted reward value corresponding to the first node switching training strategy of node i in the distributed system, K1 represents the system power consumption optimization value represented by the first environmental state prediction information and the current environmental state information, K2 represents the system average resource utilization optimization value represented by the first environmental state prediction information and the current environmental state information, M represents the number of computing tasks switched by node i, and α1, α2 and α3 represent preset coefficients.

[0121] Specifically, the influence of system performance factors K1, K2 and M on system performance can be controlled by adjusting the preset coefficients α1, α2 and α3. K1 is actually the percentage of system power consumption reduction at the current moment compared with the previous moment, K2 is actually the percentage of average system resource utilization improvement at the current moment compared with the previous moment, and M is the number of node switching at the current moment. Therefore, the model expects to learn the strategy of adaptive node switching to reduce system power consumption and improve system resource utilization under the encouragement of reward r, while minimizing the number of node switching as much as possible. The number of node switching is the number of computing tasks that the node needs to migrate.

[0122] Furthermore, based on the above rewards, we can construct the experience sample (o i ,a i ,r i ,o′) and stored in the experience pool D, where a i , o i and r i They represent the action space (the first node switching training strategy), local state space (current environment state information) and timely reward (actual reward value) of the ith agent (node), respectively. i ′ Represents the actual information of the environment state. It is randomly selected from the experience pool, and the loss is calculated by the following formula and the strategy model is updated in reverse

[0123]

[0124] Among them, a i ′ For the state o i ′ When , it is estimated that the agent will take the action that obtains the maximum reward, and the terminal state indicates that the agent reaches the terminal state, at which time the performance of the distributed system reaches the optimal state. In state o′ i The value obtained by the strategy model is estimated, y i Indicates that in state o i The actual total reward obtained when

[0125] Further, the environment model is trained. After adding the current state, action, reward, and next state to the experience pool, the environment model parameters are updated using the historical data in the experience pool, and the improved reward is used as the label to train the environment model parameters. After sampling, the current state and action are input into the environment model. Output the predicted next state o″ i and the predicted reward r′ i , the error is calculated using the following formula:

[0126]

[0127] Among them, r′ i represents the first prediction reward value, |r i -r′ i | is the real reward (actual reward value) and the environment model The difference in predicted rewards, o″ i represents the first environmental state prediction information, |o′ i -o″ i | is the real next state and environment model The gap between the predicted next state and the predicted state. The smaller the gap between the predicted state and the predicted state, the smaller the error. The environment model is trained by backpropagating the error. Finally, we get an environmental model that is close to the dynamics of a real distributed environmental system.

[0128] Based on the above embodiment, as an implementable manner, in one embodiment, based on historical operation data and a target environment model, an initial node switching model of a distributed system is trained to obtain a target node switching model, including:

[0129] Step 2031, dividing the historical operation data into time periods to obtain the operation status information of each node at each time period;

[0130] Step 2032, taking the running status data of each node at any time as input, inputting it into the initial node switching model, and obtaining the second node switching training strategy of any node of the initial node switching model at that time;

[0131] Step 2033, based on the target environment model, according to the second node switching training strategy and the current environment state information, determine the second environment state prediction information after the node switching is performed according to the second node switching training strategy;

[0132] Step 2034, determining a second predicted reward value corresponding to the second node switching training strategy according to the second environmental state prediction information and the current environmental state information;

[0133] Step 2035, iteratively optimize the model parameters of the initial node switching model according to the second node switching training strategy, the second predicted reward value, the current environment state information and the second environment state prediction information to obtain the target node switching model.

[0134] Specifically, Figure 5 As shown, it is a schematic diagram of the structure of the node switching model provided in the embodiment of the present application, the original state and Figure 4The states in the graph are the same, and they are all historical operation data. First, the initial node switching model is obtained after model initialization. For the training of the node switching model, the graph attention mechanism is embedded on the basis of the reinforcement learning model to realize the selection of the node switching target, thereby constructing a node switching model. The agents (nodes) in the distributed system are defined as points in the graph, and the attention relationship between agents is defined as an edge. The observed state of each agent is used as the feature of its point, and the point feature is input into the graph attention network, and the importance weight between agents (whose value is between 0 and 1) is output, that is, the feature of the edge. The agent with the largest attention weight is selected as the node switching target, and this process is embedded in the reinforcement learning process of the agent, and the reward of reinforcement learning is used as the loss function.

[0135] To reduce the training cost, the same reinforcement learning model structure as in the above policy model is used to build the policy network, and the parameters are copied to initialize the policy network. The perceptron and long short-term memory network are used to build and randomly initialize the feature extraction module and recurrent neural network, respectively. The feature extraction module is used to extract the key features in the original state, and the recurrent neural network is used to extract the historical information in the input state sequence. The key-value query mode is used to build the graph neural network and initialize the query vector W. q and the key-value vector W k , whose input is the node hidden feature h processed by the recurrent neural network, and the output is the attention weight of the current node relative to other nodes.

[0136] Specifically, the node switching model interacts with the environment model. The distributed system environment state and the state of the agent are collected, and the hidden feature h is extracted through the feature extraction module and the recurrent neural network. The hidden feature h is input into the policy network and the graph neural network respectively to obtain the action a to be executed and the node switching target t, and the current state and action are further input into the environment model. Get the next state o ′ and reward r.

[0137] Among them, action a is selected from the available actions using the ∈-greedy greedy method. i With the characteristics of other agents h j The graph attention network is input in sequence to calculate the edge weights, and the node switching target t is the node with the largest attention weight.

[0138] The calculation formula is as follows:

[0139]

[0140] in, h j The transpose of W kt is the transpose of . t represents the index of agent j that agent i pays the most attention to, and agent i believes that switching the node to agent j is the optimal strategy.

[0141] Specifically, in one embodiment, the performance optimization reward value of the second node switching training strategy can be determined based on the current environment state information and the second environment state prediction information; the comprehensive reward value of the second node switching training strategy can be determined based on the second prediction reward value and the performance optimization reward value; based on the second node switching training strategy and the comprehensive reward value of the second node switching training strategy, the model parameters of the initial node switching model are iteratively optimized to obtain the target node switching model.

[0142] Specifically, for the node switching task, the agent's reward r is designed by considering the performance difference of the system before and after the node switching, the switching cost, the node switching failure time and other real-time indicators:

[0143] r=r ′ +μ(α1K1+α2K2-α3M-α4T)

[0144] Among them, α1, α2, α3, α4, and μ are coefficients, which can be adjusted according to system needs to affect the main factors of node switching. ′ For environmental model The output estimated reward, i.e. the second predicted reward value, μ(α1K1+α2K2-α3M-α4T) represents the performance optimization reward value, K1 is the percentage of system power consumption reduction compared with the previous moment, K2 is the percentage of average system resource utilization improvement compared with the previous moment, M is the number of node switching at the current moment, and T is the time failure cost of this node switching. Therefore, the model expects to learn an adaptive node switching strategy under the encouragement of reward r, so as to reduce system power consumption, improve system resource utilization, reduce the number of node switching, and reduce the time cost of node switching.

[0145] Training method of node switching model: Based on the previously calculated rewards, construct experience samples for each agent (o i ,a i ,r i ,o′ i ) and stored in the experience pool D, where a i , o i and r i Represent the action space, local state space and timely reward of the ith agent respectively. Since the environment model currently applied is the target environment model obtained through the above training, the r in the experience sample i That is the second predicted reward value output by the model, o i ′This is the second environment state prediction information. It is randomly selected from the experience pool, and the loss is calculated by the following formula to reversely update the policy network and graph neural network parameters W q and W k .

[0146]

[0147] Among them, a i i For the state o i ′ When , it is estimated that the agent will take the action that obtains the maximum reward, and the terminal state indicates that the agent reaches the terminal state. For the state o i ′ The value obtained by the strategy model is estimated, y i Indicates that in state o i The actual total reward obtained when

[0148] Based on the above embodiment, as an implementable manner, in one embodiment, based on the target node switching model and according to the current operation data, determining the current node switching strategy of the distributed system includes:

[0149] Step 2051, extracting features from the current operation data to obtain current operation features of the distributed system;

[0150] Step 2052, based on the recurrent neural network, according to the current running characteristics, determine the current running hidden characteristics;

[0151] Step 2053, input the currently running hidden features into the policy network and the graph neural network respectively;

[0152] Step 2054, based on the policy network and according to the currently running hidden features, determine whether each node in the distributed system needs to be switched;

[0153] Step 2055, when any node in the distributed system needs to be switched, the current node switching strategy of the node is determined based on the graph neural network and the currently running hidden features.

[0154] Among them, the target node switching model includes recurrent neural network, policy network and graph neural network.

[0155] Specifically, after the node switching model training is completed, the model is run in a real distributed system, and the state of the current node, that is, the current running data, is input. The model will output whether the node needs to take a node switching action in the current state. If node switching is required, the node switching target will be output, and the action will be automatically executed to complete the node switching strategy and realize the adaptive switching of nodes. When a node failure occurs, the node fault flag is activated, its node attention is set to 0, and the node is in a non-switchable state. That is, the node switching will not select the faulty node (attention is 0) for switching, and automatic filtering of faulty nodes is realized. At the same time, when the system detects that the node fault flag is activated, the node recovery plan will be triggered. After the node is repaired, it will rejoin the system, thereby realizing self-repair of the node failure.

[0156] The method provided in the embodiment of the present application can be used to dynamically and adaptively make intelligent decisions on node switching in a distributed system, thereby improving the resource utilization of the system and reducing costs. Figure 6 As shown in FIG. 1 , it is a schematic diagram of the overall process of the distributed node switching method provided in the embodiment of the present application, and a DDQN (deep reinforcement learning) network is used to construct a node agent model. Figure 7 As shown, it is a schematic diagram of the training process of the node switching model provided in the embodiment of the present application. Figure 6 and Figure 7 The process is as follows Figure 2 The process shown is an exemplary implementation method, and the implementation principle is the same and will not be repeated here.

[0157] Specifically, Figure 8 As shown, it is a schematic diagram of the structure of the distributed system provided in the embodiment of the present application. During the operation of the distributed system, each node continuously processes and receives tasks, and there are node failures, so it is necessary to adaptively switch nodes in the distributed system. In the method provided in the embodiment of the present application, the service node can be abstracted as an agent, and the agent (server node) obtains the required status information through the interface. The "rectangle" represents the agent (server node), the "solid line" represents the attention relationship between the agents, and the thicker the "solid line", the greater the attention weight between the agents. Taking a distributed system with 6 agents as an example, Figure 8 It shows the process of agent 1 using graph attention mechanism to select node switching target and switch to agent 3.

[0158] In order to facilitate those skilled in the art to better understand the distributed node switching method provided in the embodiment of the present application, the implementation of the present invention is described in detail, which specifically includes the following steps:

[0159] (1) Historical data collection: Collect data such as CPU utilization, memory utilization, node power consumption, node switching time, node switching target, system total power consumption, and system average resource utilization of nodes in the distributed system.

[0160] (2) Data cleaning and format conversion: Clean the collected data to remove errors, anomalies or missing values, convert the training data, and unify the data format and content.

[0161] (3) Constructing a training experience pool: Divide the collected data by time and construct the state features of the nodes at each time. A set of feasible state attribute features is shown in Table 1.

[0162] Table 1 Node status characteristics

[0163]

[0164] (4) Initialization process: Initialize the agent strategy model DDQN network parameters and environment model Parameters.

[0165] (5) Node switching state conversion model training cycle process:

[0166] (5.1) Interaction sampling: The agent interacts with the environment to store samples and randomly selects a batch of training samples from the experience pool.

[0167] (5.2) Calculate rewards: Calculate the rewards of the agent in the node switching state transition model.

[0168] (5.3) Calculate the error: Calculate the error between the network output and the training sample.

[0169] (5.4) Training model: Back propagate the error to train the DDQN model and the environment model

[0170] (5.5) Model output: Output environment model As a training environment for node switching models.

[0171] (6) Node switching model online training cycle process:

[0172] (6.1) Obtain the current system status through the agent’s API interface.

[0173] (6.2) Selecting the node switching target: A node switching target is selected for each agent through the graph attention mechanism, where the graph attention mechanism is implemented using the GRU memory unit, as shown in Fig. 9 As shown, it is a schematic diagram of the structure of an exemplary attention mechanism network provided in an embodiment of the present application.

[0174] (6.3) Estimated action value: Input the acquired system state into the DDQN network to obtain the estimated value of the action that the agent may take.

[0175] (6.4) Select action and execute: Based on the value estimate and the node switching target, select the agent to perform the node switching action or not to perform the action.

[0176] (6.5) Obtain reward and next state: Input the node's current state and action into the environment model Get feedback from the environment and move to the next state.

[0177] (6.6) Storing experience: storing the agent’s state, action, reward, and next state information into the experience pool.

[0178] (6.7) DDQN model update: randomly extract samples from the experience pool, calculate the loss, and use the gradient descent method to reversely update the DDQN network parameters.

[0179] (6.8) Graph attention model update: When updating the DDQN model parameters, update the graph attention model parameters.

[0180] When the loop condition is not met, that is, when the loop reaches a preset number of training times or a preset indicator, a node adaptive switching strategy solution that performs well in the distributed system can be obtained, that is, a target node switching model is obtained, and can be further applied to actual distributed systems.

[0181] The distributed node switching method provided by the embodiment of the present application obtains the historical operation data of the distributed system; based on the historical operation data, the initial node switching state transfer model of the distributed system is trained to obtain the target node switching state transfer model; wherein, the target node switching state transfer model includes a target environment model, and the environment model is used to describe the state changes of each node in the distributed system; based on the historical operation data and the target environment model, the initial node switching model of the distributed system is trained to obtain the target node switching model; the current operation data of the distributed system is input into the target node switching model; based on the target node switching model, according to the current operation data, the current node switching strategy of the distributed system is determined; wherein, the current node switching strategy at least includes the target switching node. The method provided by the above scheme trains the target node switching model by using the target environment model obtained by training to ensure that the node switching strategy output by the target node switching model can make the system environment state optimal, that is, according to the node switching strategy, the computing task of the node to be switched can be switched to the optimal target switching node, thereby improving the performance of the distributed system. In addition, a simulated distributed system environment is constructed using a model-based reinforcement learning algorithm, and the node switching model is trained without affecting the operation of the existing distributed system, reducing the impact on the existing services of the system. At the same time, the node switching state transfer model parameters are reused to reduce the resource consumption and time cost of model training. A reward function based on indicators such as node switching power consumption, switching times, and switching time is designed. Real-time indicators such as time are added based on statistical data to guide the node switching model to improve efficiency while reducing costs. Using the graph attention mechanism, the attention weights between nodes are calculated by extracting node hidden features, and the node with the largest attention weight is selected as the switching target, which solves the problem of difficulty in selecting appropriate switching targets and resulting in poor node switching effects. The reinforcement learning algorithm is used to monitor the distributed system resource information in real time, realize the real-time dynamic adaptive switching of system nodes, and improve the utilization of system resources. At the same time, the node switching model has a certain reasoning ability, can be used in a variety of distributed service scenarios, and quickly generate node switching solutions in real time.

[0182] An embodiment of the present application provides a distributed node switching device, which is used to execute the distributed node switching method provided in the above embodiment.

[0183] like Fig.10 , which is a schematic diagram of the structure of a distributed node switching device provided in an embodiment of the present application. The distributed node switching device 100 comprises: an acquisition module 1001 , a first training module 1002 , a second training module 1003 , an input module 1004 and a switching module 1005 .

[0184] Among them, the acquisition module is used to obtain the historical operation data of the distributed system; the first training module is used to train the initial node switching state transfer model of the distributed system based on the historical operation data to obtain the target node switching state transfer model; wherein the target node switching state transfer model includes a target environment model, and the environment model is used to describe the state changes of each node in the distributed system; the second training module is used to train the initial node switching model of the distributed system based on the historical operation data and the target environment model to obtain the target node switching model; the input module is used to input the current operation data of the distributed system into the target node switching model; the switching module is used to determine the current node switching strategy of the distributed system based on the target node switching model and the current operation data; wherein the current node switching strategy at least includes the target switching node.

[0185] Regarding the distributed node switching device in this embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0186] The distributed node switching device provided in the embodiment of the present application is used to execute the distributed node switching method provided in the above embodiment. Its implementation method and principle are the same and will not be repeated here.

[0187] An embodiment of the present application provides an electronic device for executing the distributed node switching method provided in the above embodiment.

[0188] like Fig.11 , which is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device 110 includes: at least one processor 1101 and a memory 1102.

[0189] The memory stores computer-executable instructions; at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the distributed node switching method provided in the above embodiment.

[0190] The electronic device provided in the embodiment of the present application is used to execute the distributed node switching method provided in the above embodiment. Its implementation method and principle are the same and will not be repeated here.

[0191] An embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, a distributed node switching method provided in any of the above embodiments is implemented.

[0192] The storage medium containing computer executable instructions provided in the embodiment of the present application can be used to store computer executable instructions of the distributed node switching method provided in the aforementioned embodiment. Its implementation method and principle are the same and will not be repeated here.

[0193] An embodiment of the present application provides a computer program product, including computer instructions, which are used to enable a computer to execute the distributed node switching method provided in the aforementioned embodiment.

[0194] The embodiment of the present application provides a computer program product that can be used to execute computer instructions of the distributed node switching method provided in the aforementioned embodiment. The implementation method and principle are the same and will not be repeated here.

[0195] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0196] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0197] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0198] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0199] A part of the present application may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of computer program instructions in computer-readable media includes, but is not limited to, source files, executable files, installation package files, etc., and accordingly, the way in which computer program instructions are executed by a computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to the computer.

[0200] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0201] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A distributed node switching method, characterized in that: include: Obtain historical operation data of distributed systems; Based on the historical operation data, the initial node switching state transfer model of the distributed system is trained to obtain a target node switching state transfer model; wherein the target node switching state transfer model includes a target environment model, and the environment model is used to describe the state change of each node in the distributed system; Based on the historical operation data and the target environment model, an initial node switching model of the distributed system is trained to obtain a target node switching model; Inputting the current operation data of the distributed system into the target node switching model; Based on the target node switching model and according to the current operating data, a current node switching strategy of the distributed system is determined; wherein the current node switching strategy at least includes a target switching node.

2. The method according to claim 1, characterized in that The training of the initial node switching state transfer model of the distributed system based on the historical operation data to obtain the target node switching state transfer model includes: Dividing the historical operation data into time periods to obtain the operation status information of each node at each time period; Taking the running status data of each node at any said moment as input, inputting it into the initial strategy model, and obtaining the first node switching training strategy of any node at that moment output by the initial strategy model; Based on the initial environment model, according to the first node switching training strategy and the current environment state information, determine the first environment state prediction information after the node is switched according to the first node switching training strategy; Determining a first predicted reward value corresponding to the first node switching training strategy according to the first environmental state prediction information and the current environmental state information; Determine, based on the historical operation data, actual information of the environment state and an actual reward value after node switching according to the first node switching training strategy; Iteratively optimizing the model parameters of the initial strategy model according to the first node switching training strategy, the actual reward value, the current environment state information and the actual environment state information to obtain a target strategy model; Determine, based on the historical operation data, actual information of the environment state and an actual reward value after node switching according to the first node switching training strategy; Iteratively optimizing the model parameters of the initial environment model according to the first predicted reward value, the first environment state prediction information, the actual environment state information and the actual reward value to obtain a target environment model; The target node switching state transfer model includes the target strategy model and the target environment model.

3. The method according to claim 2, characterized in that The determining, according to the first environmental state prediction information and the current environmental state information, a first predicted reward value corresponding to the first node switching training strategy includes: Based on the following formula, the first predicted reward value corresponding to the node switching training strategy is determined: r i ′ =α1K1+α2K2-α3M Among them, r i ′ represents the first predicted reward value corresponding to the first node switching training strategy of node i in the distributed system, K1 represents the system power consumption optimization value represented by the first environmental state prediction information and the current environmental state information, K2 represents the system average resource utilization optimization value represented by the first environmental state prediction information and the current environmental state information, M represents the number of computing tasks switched by node i, and α1, α2 and α3 represent preset coefficients.

4. The method according to claim 1, characterized in that: The training of the initial node switching model of the distributed system based on the historical operation data and the target environment model to obtain the target node switching model includes: Dividing the historical operation data into time periods to obtain the operation status information of each node at each time period; Taking the running status data of each node at any said moment as input, inputting it into the initial node switching model, and obtaining the second node switching training strategy of any node of the initial node switching model at that moment; Based on the target environment model, according to the second node switching training strategy and the current environment state information, determine the second environment state prediction information after the node is switched according to the second node switching training strategy; Determining a second predicted reward value corresponding to the second node switching training strategy according to the second environmental state prediction information and the current environmental state information; According to the second node switching training strategy, the second predicted reward value, the current environment state information and the second environment state prediction information, the model parameters of the initial node switching model are iteratively optimized to obtain the target node switching model.

5. The method according to claim 4, characterized in that The iterative optimization of the model parameters of the initial node switching model according to the second node switching training strategy, the second predicted reward value, the current environment state information and the second environment state prediction information to obtain the target node switching model includes: Determine a performance optimization reward value of the second node switching training strategy according to the current environment state information and the second environment state prediction information; Determining a comprehensive reward value of the second node switching training strategy according to the second prediction reward value and the performance optimization reward value; According to the second node switching training strategy and the comprehensive reward value of the second node switching training strategy, the model parameters of the initial node switching model are iteratively optimized to obtain a target node switching model.

6. The method according to claim 1, characterized in that The determining, based on the target node switching model and according to the current operating data, a current node switching strategy of the distributed system includes: Extracting features from the current operation data to obtain current operation features of the distributed system; Based on a recurrent neural network, determining a current running hidden feature according to the current running feature; Inputting the currently running hidden features into the policy network and the graph neural network respectively; Based on the policy network, and according to the currently running hidden features, determining whether each node in the distributed system needs to be switched; When any node in the distributed system needs to be switched, based on the graph neural network and according to the currently running hidden features, a current node switching strategy of the node is determined; Among them, the target node switching model includes the recurrent neural network, the policy network and the graph neural network.

7. A distributed node switching device, characterized in that: include: The acquisition module is used to obtain the historical operation data of the distributed system; A first training module is used to train the initial node switching state transfer model of the distributed system based on the historical operation data to obtain a target node switching state transfer model; wherein the target node switching state transfer model includes a target environment model, and the environment model is used to describe the state change of each node in the distributed system; A second training module is used to train the initial node switching model of the distributed system based on the historical operation data and the target environment model to obtain a target node switching model; An input module, used for inputting the current operation data of the distributed system into the target node switching model; A switching module is used to determine a current node switching strategy of the distributed system based on the target node switching model and the current operating data; wherein the current node switching strategy at least includes a target switching node.

8. An electronic device, characterized in that: include: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the method according to any one of claims 1 to 6.

Images