A DAG micro-service migration method and device oriented to cooperation and latency sensitivity

Abstract

The application discloses a kind of DAG microservice migration methods and devices for cooperation and latency sensitive, the method includes: using service pre-migration algorithm to carry out the training of service pre-migration model, obtain the trained service pre-migration model;The trained service pre-migration model is deployed to user end, and when mobile occurs in user end, the trained service pre-migration model carries out service pre-migration decision according to the local information obtained, and obtains target edge cloud;Microservice migration algorithm is used to carry out the migration decision of microservice to the edge server in target edge cloud, and obtains the edge server to be migrated, and carries out the migration of microservice.The application can effectively reduce the delay of microservice migration, and still can obtain good effect under a variety of changing conditions.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of mobile edge computing and artificial intelligence, and particularly relates to a DAG micro-service migration method and device for cooperation and latency sensitivity. BACKGROUND

[0002] MEC has significant application advantages compared to traditional cloud computing architecture due to its "near-user computing" feature. However, the implementation of mobile edge computing also faces many challenges, especially in mobile edge computing, how to efficiently perform service migration (Service Migration) is a key problem. Since the coverage of a single edge server is limited, when a user moves in space, the original edge server that provides services for it may not be able to continue to maintain low delay to meet its needs due to distance, so it is usually necessary to dynamically select a reasonable communication path or dynamically migrate services to a new edge node to ensure service continuity and efficiency. By reasonably planning the communication path between the original edge server and the user, the service delay is minimized as much as possible, which is called path selection. By migrating services on the original edge server to a new edge server to provide low-delay services for users, this is called service migration. However, in the case where the user is very far from the edge server, dynamically finding the best communication path to reduce service delay is ultimately a limited palliative measure. In most cases, service migration is still needed to alleviate the high service delay caused by user location changes. In the MEC environment, the decision-making of whether to migrate services and where to migrate is a huge challenge. This decision-making process needs to consider many factors, such as delay, energy consumption, computing power, and network conditions. In particular, in a mobile and dynamic environment, how to dynamically adjust and allocate resources according to the user's location, needs and environment, and as much as possible to reduce the user's waiting time and improve the user's service experience, all of which need to be considered.

[0003] Although there have been a large number of research works exploring the service migration problem in MEC, there are still obvious deficiencies in the prior art. First, the current service migration strategy research mostly focuses on the migration of independent tasks, which cannot effectively handle DAG jobs with complex dependency relationships. Second, existing service migration strategies usually rely on complex optimization algorithms, which limits their running efficiency in large-scale MEC environments and makes it difficult to meet the real-time needs in practical applications. In addition, due to the uncertainty of user movement patterns and the dynamic changes of edge server resources, many theoretically superior migration strategies are difficult to maintain ideal performance in practical applications.

[0004] Therefore, the prior art still needs to be improved and improved. SUMMARY

[0005] The technical problem to be solved by the present application is to provide a collaborative and latency-sensitive DAG micro-service migration method and device to solve the above-mentioned defects of the prior art.

[0006] In a first aspect, the present application provides a collaborative and latency-sensitive DAG micro-service migration method, wherein the method comprises:

[0007] The service pre-migration algorithm is used to train the service pre-migration model to obtain a trained service pre-migration model.

[0008] The trained service pre-migration model is deployed to the user end, and when the user end moves, the trained service pre-migration model makes a service pre-migration decision based on the acquired local information to obtain a target edge cloud.

[0009] The micro-service migration algorithm is used to make a micro-service migration decision for the edge server in the target edge cloud to obtain an edge server to be migrated, and the micro-service migration is performed based on the migration decision.

[0010] In an implementation manner, the service pre-migration algorithm is used to train the service pre-migration model to obtain a trained service pre-migration model, comprising:

[0011] The PPO-based reinforcement learning method is used to develop a PPO-based service pre-migration algorithm, and the service pre-migration model is trained based on the service pre-migration algorithm to obtain a trained service pre-migration model; wherein the service pre-migration algorithm comprises an initialization stage, a data collection stage, a calculation advantage and discount return stage, an update strategy and value function stage, and a strategy evaluation and adjustment stage.

[0012] In an implementation manner, in the initialization stage, the long short-term memory network is used to learn the hidden state from the historical observation sequence to capture the long-term dependence and infer the environment hidden state.

[0013] In an implementation manner, the micro-service migration algorithm is used to make a micro-service migration decision for the edge server in the target edge cloud to obtain an edge server to be migrated, comprising:

[0014] According to the topological sorting, the subtasks in the task are divided into a group of micro-service layers, and a network flow graph is initialized for each micro-service layer.

[0015] After the network flow graph is initialized for each micro-service layer, the continuous shortest path algorithm is applied to solve the minimum cost maximum flow problem to obtain the optimal matching between the micro-service in each layer and the edge server, and an optimal matching set is obtained.

[0016] Based on the optimal matching set, obtain the edge server to be migrated.

[0017] In an implementation manner, a network flow graph is initialized for each micro-service layer, including:

[0018] A source node and a sink node are added in each network flow graph; for each sub-task in each micro-service layer, an edge is added from the source node to each sub-task node, and the capacity of the edge is set to 1.

[0019] In an implementation manner, the network flow graph is initialized for each micro-service layer, and further including:

[0020] If the computing requirement of the sub-task is less than or equal to the resource capacity of the edge server, an edge is added from the sub-task node to the edge server, and the capacity of the edge is set to 1.

[0021] In an implementation manner, the network flow graph is initialized for each micro-service layer, and further including:

[0022] All edge servers in the target edge cloud are connected to the sink node, and the capacity of the edge between each edge server and the sink node is the number of available CPU cores on the corresponding edge server.

[0023] In a second aspect, the embodiments of the present application further provide a DAG micro-service migration device for cooperation and latency sensitivity, wherein the device includes:

[0024] A model training module is configured to train a service pre-migration model by using a service pre-migration algorithm, and obtain a trained service pre-migration model.

[0025] A service pre-migration module is configured to deploy the trained service pre-migration model to a user terminal, and when the user terminal moves, the trained service pre-migration model makes a service pre-migration decision according to acquired local information, and obtains a target edge cloud.

[0026] A micro-service migration module is configured to make a micro-service migration decision for edge servers in the target edge cloud by using a micro-service migration algorithm, obtain edge servers to be migrated, and perform micro-service migration based on the migration decision.

[0027] In a third aspect, the embodiments of the present application further provide a terminal, wherein the terminal includes a memory, a processor, and a DAG micro-service migration program for cooperation and latency sensitivity stored in the memory and executable on the processor, and when the processor executes the DAG micro-service migration program for cooperation and latency sensitivity, the steps of the DAG micro-service migration method for cooperation and latency sensitivity in any of the above solutions are implemented.

[0028] In a fourth aspect, the embodiments of the present application further provide a computer readable storage medium, wherein the computer readable storage medium stores a cooperative and latency sensitive DAG micro-service migration program, and the cooperative and latency sensitive DAG micro-service migration program, when executed by a processor, implements the steps of the cooperative and latency sensitive DAG micro-service migration method in any of the above solutions.

[0029] Beneficial effects: Compared with the prior art, the present application provides a cooperative and latency sensitive DAG micro-service migration method. The present application first trains a service pre-migration model by using a service pre-migration algorithm to obtain a trained service pre-migration model. Then, the trained service pre-migration model is deployed to a user end, and when the user end moves, the trained service pre-migration model makes a service pre-migration decision according to the obtained local information to obtain a target edge cloud. Next, a micro-service migration algorithm is used to make a micro-service migration decision for the edge servers in the target edge cloud to obtain an edge server to be migrated, and the migration of the micro-service is performed based on the migration decision. The present application can effectively reduce the delay of micro-service migration, and still achieve good results under various changing conditions. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 The flow chart of the preferred embodiment of the cooperative and latency sensitive DAG micro-service migration method provided by the embodiments of the present application.

[0031] Figure 2 The structure diagram of the cooperative micro-service migration framework provided by the embodiments of the present application.

[0032] Figure 3 The system model diagram of the cooperative micro-service migration provided by the embodiments of the present application.

[0033] Figure 4 The training and deployment process diagram of the service pre-migration model provided by the embodiments of the present application.

[0034] Figure 5 The network flow construction diagram of the DAG task corresponding service provided by the embodiments of the present application.

[0035] Figure 6 The architecture diagram of the cooperative and latency sensitive DAG micro-service migration device provided by the embodiments of the present application.

[0036] Figure 7 The principle block diagram of the terminal provided by the embodiments of the present application. DETAILED DESCRIPTION

[0037] For the purposes of the present application, the technical solutions and effects, the following will be further described in detail with reference to the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application, and are not intended to limit the present application.

[0038] The flowchart shown in the drawings is only an example and does not necessarily include all contents and operations or steps, nor does it necessarily execute in the order described. For example, some operations or steps can be further divided, combined or partially merged, so the actual execution order can be changed according to the actual situation.

[0039] It should be understood that the terms used in the present application specification herein are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the present application specification and the appended claims, unless otherwise clear from the context, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0040] It should be understood that in order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, the terms "first", "second" and the like are used to distinguish the same or similar items with basically the same function and effect. For example, the first control information and the second control information are only used to distinguish different control information, and do not limit the order.

[0041] Those skilled in the art can understand that the terms "first", "second" and the like do not limit the quantity and execution order, and the terms "first", "second" and the like do not necessarily mean different.

[0042] It should also be understood that the term "and / or" used in the present application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0043] The existing service migration research work hardly considers the dependency that may exist between microservices, in order to make up for the lack of related methods, the embodiment provides a DAG microservice migration method for cooperation and latency sensitivity, studies the microservice migration problem between microservices in mobile edge computing, and converts the microservice migration problem with dependency into a two-stage problem for solving, wherein the first stage is to determine whether the DAG service needs to be migrated and which edge cloud is the most suitable for migration based on the DAG service pre-migration algorithm. In the second stage, the microservice migration algorithm based on topological sorting and network flow is used to solve the problem, so that the best edge server to be migrated is found for each microservice in the DAG. In a specific application, the embodiment first trains a service pre-migration model using a service pre-migration algorithm, and obtains a trained service pre-migration model. Then, the trained service pre-migration model is deployed to the user end, and when the user end moves, the trained service pre-migration model makes a service pre-migration decision based on the acquired local information to obtain a target edge cloud. Next, the microservice migration algorithm is used to make a microservice migration decision for the edge servers in the target edge cloud to obtain the edge servers to be migrated, and the migration of the microservices is performed based on the migration decision. The embodiment can effectively reduce the delay of microservice migration, and still achieve good results under various changing conditions.

[0044] The DAG microservice migration method for cooperation and latency sensitivity of the embodiment can be applied to a terminal, which can be a computer or a smart terminal product such as a mobile phone. Specifically, as shown in Figure 1 The DAG microservice migration method for cooperation and latency sensitivity of the embodiment includes the following steps:

[0045] Step S100, training a service pre-migration model using a service pre-migration algorithm to obtain a trained service pre-migration model;

[0046] Step S200, deploying the trained service pre-migration model to the user end, and when the user end moves, the trained service pre-migration model makes a service pre-migration decision based on the acquired local information to obtain a target edge cloud;

[0047] Step S300, using a microservice migration algorithm to make a microservice migration decision for the edge servers in the target edge cloud to obtain the edge servers to be migrated, and performing the migration of the microservices based on the migration decision.

[0048] Due to the huge action space of DAG (Directed Acyclic Graph)-based service migration decision, it poses great challenges to the performance of reinforcement learning model. Unlike independent tasks, whose action space is only related to the number of edge servers, the action space of DAG task migration is jointly determined by the number of subtasks in DAG task and the number of edge servers, especially in the case of allowing collaboration between edge servers. When DAG task contains a large number of subtasks or edge servers are densely deployed, the action space of deep reinforcement learning will become very large, thus greatly increasing the difficulty and training overhead of model training. The embodiment proposes a collaborative microservice migration framework, as shown in Figure 2 , Figure 2 is a structural diagram of the collaborative microservice migration framework. The system structure includes a user end 10, an edge cloud 20 and a remote cloud 30. The user end 10 stores and executes the model, the edge cloud 20 executes the microservice migration algorithm, and the remote cloud 30 trains the model. As shown in Figure 3 , Figure 3 is a system model diagram of collaborative microservice migration. The embodiment converts the above microservice migration problem into a two-stage optimization problem for solving. In the collaborative microservice migration framework, edge servers are divided into edge clouds according to their location information. Let E be the set of edge clouds, where the remote cloud is also regarded as a special edge cloud in E. Given a DAG task t u unloaded by a user u, the task in the first stage (referred to as the service pre-migration stage) is to make a pre-migration decision for the DAG-based service, that is, to determine the target edge cloud suitable for containing the DAG-based service corresponding to t u . After determining the target edge cloud of time slot t , the task in the second stage (referred to as the microservice migration decision stage) is to find the edge server to be migrated for each microservice in and perform microservice migration operation.

[0049] Specifically, in the first stage, the embodiment models the service pre-migration problem as a partially observable Markov decision process, and designs a service pre-migration model based on reinforcement learning to solve the problem. The service pre-migration model is used to determine whether the DAG service in the original edge cloud should be migrated with the movement of the user, and when pre-migration is needed, which edge cloud is the most suitable target edge cloud. Figure 4 The training and deployment process of the service pre-migration model is shown. By training the model centrally in the cloud and deploying it on the user end, the trained model makes pre-migration decisions for the DAG service based on the information available to the user end, effectively adapting to the dynamics and uncertainties of the MEC environment and improving the user experience.

[0050] When training the service pre-migration model, a PPO-based reinforcement learning method is used to develop a PPO-based service pre-migration algorithm, and the service pre-migration model is trained based on the service pre-migration algorithm to obtain a trained service pre-migration model, aiming to solve the pre-migration problem in the first stage. In the scenario of the present application, since the global information of the cluster cannot be obtained, the trained service pre-migration model can only observe part of the information, so the present application models the service pre-migration process as a partially observable Markov decision process. In this process, the service pre-migration model cannot directly access the real state of the environment, but can only infer the state of the environment through observable information. In order to solve this problem, the concept of belief state is usually introduced. The belief state b u,t is the probability distribution of the agent on the current system state, which represents the agent's speculation on the current possible state of the system according to the historical observations. Specifically, the belief state is derived from the historical observation sequence and the past decisions.

[0051] The updating process of the belief state is usually recursive, which needs to be updated by new observations at each time. Although the traditional belief updating method can represent the belief of the agent, it has computational difficulties in the case of a huge state space or long-term dependence. In order to overcome the limitations of the traditional belief state method, the present embodiment uses a long short-term memory network (LSTM) to replace the belief state. LSTM is a special recurrent neural network that can effectively learn the hidden state from the historical observation sequence, capture long-term dependencies, and replace the traditional belief state. In the present embodiment, the service pre-migration model uses the hidden state h u,t generated by the LSTM to replace the traditional belief state b u,t to make decisions, so that the solution of the POMDP problem (partially observable Markov decision process) becomes more efficient and has better adaptability.

[0052] The service pre-migration algorithm in the present embodiment includes an initialization phase, a data collection phase, a calculation of advantage and discounted return phase, an updating of policy and value function phase, and a policy evaluation and adjustment phase.

[0053] In the initialization phase: the system initializes the user set U and the edge cloud set E and their initial states. The observation space O includes 4 parameters: P t (u) represents the local edge cloud index where the user u∈U is located at time slot t; D t (t u ) represents the data size of the task t u corresponding to the service that needs to be migrated at time slot t; r t represents the wireless transmission rate at time slot t; and c t (t u) represents the time slot t processing task t u The number of CPU cycles required. The state vector of user u at time slot t is o u,t

[0054] [P t (u),D t (t u ),r t ,c t (t u )]. Let denote the action space. When observing state o u,t , the service pre-migration model determines the action a u for time slot t task t u,t = ec e E. Let denote the reward function. Given action a u,t , the service pre-migration model obtains reward r u,t from the environment, which is represented as negative t u service delay, which is defined as:

[0055] r u,t = -L′ t (t u , ec p , ec q ).

[0056] In POMDP, the hidden state of the environment is invisible to the agent, and the service pre-migration model can only infer the hidden state of the environment through observation o u,t . LSTM generates hidden state h u,t through processing of historical observations, which can be regarded as the service pre-migration model's speculation of the current system state, replacing the traditional belief state b u,t . LSTM controls the forgetting, retaining and outputting of information through memory cells and gate mechanisms, and its update process is as follows:

[0057] h u,t = LSTM(O u,t , h u,t-1 ),

[0058] where O u,t = {o u,1 , o u,2 , …, o u,t} is the historical observation sequence, h u,t-1 is the hidden state of the last time step, and h u,t is the hidden state of the current time step, representing the service pre-migration model's belief of the current system state.

[0059] In defining and ​After that, the policy network π(a u,t | h u,t ; θ) and the value network V(h u,t ; φ) can be initialized, where θ and φ are network parameters. Hyperparameters that can be set include the total number of training rounds N, the PPO update frequency K, the mini-batch size M, the discount factor γ, the truncation parameter ∈, and the learning rate α.

[0060] In the data collection phase, which involves three steps, described as follows:

[0061] Step i. Environment interaction: the initial state h u,0 is observed, and an action a u,0 is chosen according to the policy network.

[0062] Step ii. Action execution and reward observation: the action a u,0 is executed, and a new state h u,1 and a reward r u,0 are obtained.

[0063] Step iii. Experience storage: the tuple (h u,0 , a u,0 , r u,0 , h u,1 ) is stored.

[0064] In the computation of advantage and discounted return phase: A t is the advantage function estimate, representing the expected return of choosing a particular action at time slot t relative to the average case. A t is computed according to the generalized advantage estimation:

[0065] A t = δ t + (γλ)δ t+1 + … + (γλ) T-t+1 δ T-1 ,

[0066] where γ is the discount factor, and λ is the generalized advantage estimation hyperparameter that balances bias and variance. δ t = r u,t + γV(h u,t+1 ; φ) - V(h u,t ; φ) is the temporal difference error at time slot t, where r u,t is the immediate reward, and V(h u,t ; φ) and V(h u,t+1 ; φ) are the value function estimates for the current state and the next state, respectively. Since the trajectory is finite, the advantage estimate A t can be computed at the end of the T time slots of the trajectory sequence:

[0067]

[0068] In reinforcement learning, the discounted return G t is defined as the weighted sum of all future rewards starting from time slot t to the end of the trajectory sequence. The weighted sum is adjusted by a discount factor γ, where 0≤γ≤1. The role of the discount factor γ is to control the relative importance of future rewards in the current decision. Specifically, a lower value of γ will make the service pre-migration model focus more on the immediate reward, thus exhibiting short-sighted behavior; while a higher value of γ allows the service pre-migration model to consider long-term benefits more in decision-making, enabling it to weigh current and future rewards. The discounted return G t is defined as:

[0069]

[0070] In practical applications, since the trajectory sequence is finite, the discounted return is usually calculated from time slot t to the end of the trajectory sequence. Therefore, G t can be defined as:

[0071]

[0072] This is a finite sum over T time slots.

[0073] In the update policy and value function phase, this phase involves two steps, which are described as follows.

[0074] Step i. Policy optimization: optimize the policy using the following objective function:

[0075]

[0076] where is the probability ratio. ∈ is a hyperparameter that controls the clipping range. clip(x, a, b) is a function that limits the value to the interval [a, b]. The policy network parameters θ are updated using gradient ascent:

[0077] Step ii. Value network update: calculate the target value using the discounted return G t to minimize the following loss function:

[0078]

[0079] The value network parameters φ are updated by gradient descent:

[0080] In the policy evaluation and adjustment phase, the current policy is evaluated at fixed intervals (or when the loss function reaches a certain performance indicator), and the policy is adjusted according to the environment feedback, ultimately obtaining a trained model.

[0081] Based on the above five stages, the service pre-migration model based on PPO deep reinforcement learning is trained. The service pre-migration model can effectively determine the target edge cloud of DAG service migration.

[0082] It should be noted that the present embodiment mainly adopts a reinforcement learning model based on PPO (Proximal Policy Optimization), which is suitable for partially observable Markov decision processes. In other implementation manners, the reinforcement learning method based on deep Q network (DQN) or actor-critic structure can also be used in this scenario. Although PPO performs well in stability and optimization, DQN or actor-critic model may have an advantage in action selection when dealing with large-scale and complex environments, especially when the action space is very large in DAG tasks.

[0083] Further, the microservice migration decision problem in the second stage can be regarded as a matching problem between edge servers in the target edge cloud and the tasks t u In the design of the present application, the present embodiment converts this problem into a minimum cost maximum flow (MCMF) problem and proposes a microservice migration decision algorithm based on the continuous shortest path algorithm to solve it. Since the microservice migration decision algorithm needs to obtain the global information of edge servers in the edge cloud in advance, it needs to be deployed in the central node or control node of the edge cloud.

[0084] After determining the target edge cloud for migration, the microservice migration decision problem in the second stage is to find the edge server to be migrated in the target edge cloud u for each microservice (corresponding to a subtask) in the task t to minimize the total service delay for processing these tasks. This problem can be regarded as a matching problem between microservices and edge servers. Unlike typical matching problems, this problem needs to consider the dependency relationship between subtasks, because the execution of a subtask depends on the results of its predecessor subtasks. For example, Figure 5 shows a DAG task t u containing 5 subtasks, where subtask can only start execution after subtasks and are completed.

[0085] Considering the dependency relationship between subtasks, the present embodiment performs batch processing of microservice migration according to the structure of the DAG task. Specifically, given a DAG task t u , the present embodiment divides it into a set of microservice layers Each microservice layer contains a set of microservices, denoted as These microservices can be processed in parallel, i.e., the dependency between microservices does not need to be considered when designing the matching strategy for each microservice layer. For the dependency problem between different microservice layers, the present embodiment solves it by processing the microservice migration sequentially from the first layer to the last layer.

[0086] For each microservice layer, the matching problem between the microservices in this layer and the edge servers in the target edge cloud can be regarded as a minimum cost maximum flow (MCMF) problem. Given a flow network with capacity and cost on each edge, the goal of the MCMF problem is to find the maximum flow from the source node to the sink node while minimizing the total cost of the flow. Similarly, the microservice migration problem of the present embodiment aims to establish a matching relationship between all microservices and edge servers in the target edge cloud while minimizing the total service delay. By reasonably constructing the network flow, the present embodiment can find the optimal matching result for each microservice layer.

[0087] For each microservice layer The modeling process of the network flow is described as follows. The present embodiment introduces a source node s and a sink node t. All microservices in are connected to the source node s, and the capacity of the edge between them is set to 1. p All edge servers in are connected to the sink node t, and the capacity of these edges is set to the number of available CPU cores n(e p ) on the edge server, and the weight of these edges is set to 0. For each pair of microservice in and edge server in , if the edge server has enough resources to process the microservice, an edge is established. The weight of this edge is the total service delay of the microservice corresponding to the processing of the corresponding subtask. If the edge server is insufficient in resources, no edge is formed. The capacity of such edges is set to 1. After initializing the network flow graph for each microservice layer, the present embodiment applies the successive shortest path algorithm to solve the minimum cost maximum flow problem, and the optimal matching between the microservices in each layer and the edge servers is obtained, resulting in an optimal matching set. Based on the optimal matching set, the present embodiment obtains the edge servers to be migrated.

[0088] The microservice migration decision phase algorithm of the present embodiment: through the trained service pre-migration model, the target edge cloud ec * to be migrated can be determined. Then, according to the microservice migration algorithm, the microservices are migrated to the appropriate edge servers in ec * . First, according to the topological sorting, t uThe subtasks in are divided into a set of microservice layers. Then, a network flow graph is initialized for each microservice layer. A source node s and a sink node t are added to each graph. Each subtask in Add an edge from the source node s to each subtask node with a capacity of 1 and a weight of 0. The edge between the microservice and the edge server is determined by the capacity of the edge server and the task computing requirements. The computing requirements are less than or equal to the server capacity Then add an edge from the subtask node to the edge server with a capacity of 1 and a weight of Otherwise, no edge is added. The capacity represents the server’s ability to process the microservice corresponding to the subtask, and the weight reflects the cost of migrating the microservice to the server. Afterwards, an edge is added from each edge server to the sink node t, with a capacity equal to the number of CPU cores available on the edge server n(e q ), with a weight of 0. This ensures that the edge server can handle the microservices according to its resource constraints (number of CPU cores). After constructing the network flow, the continuous shortest path algorithm is applied to solve the minimum cost maximum flow problem, thereby obtaining the optimal match between the microservice and the edge server, which is recorded as the matching set Once all matching sets for each microservice layer are found, the final microservice migration decision can be made. The edge server to be migrated is obtained.

[0089] like Figure 4 As shown, Figure 4 The process of constructing a network flow for a DAG task with five subtasks is demonstrated in Figure 2. The target edge cloud includes three edge servers, {e1, e2, e3}, each with sufficient resources to handle the microservice corresponding to any subtask. A topological sorting algorithm is used to partition the subtasks into three microservice layers. For each microservice layer, a source node s and a sink node t are introduced to construct the network. Source node s connects to all subtasks within that layer, while sink node t connects to the three edge servers. Each edge in the network is associated with two variables, C and W, where C represents the capacity and W represents the weight. By determining the maximum possible flow from s to t without exceeding the edge capacity, the optimal match between the microservice and edge server corresponding to each subtask layer can be found. By constructing a network flow model from layers 1 to 3, the final matching result is obtained.

[0090] It should be noted that in other implementations, the application can also be based on a DAG task topology-aware scheduling strategy to replace the topological sorting and network flow modeling in the above embodiments. For example, a graph neural network (GNN) can directly extract migration features from the DAG topology structure to predict the optimal migration strategy. This deep learning-based graph model avoids the strong dependence of the network flow model on global information.

[0091] The workflow of the DAG service migration algorithm proposed in this embodiment includes two stages: a service pre-migration stage based on reinforcement learning and a microservice migration decision stage. For each time slot, if a user location update is detected, a pre-migration decision model trained based on the service pre-migration algorithm is called to find the target edge cloud, and then a microservice migration algorithm is called to make migration decisions for each microservice. After migration is completed, the environment is updated to facilitate future migration decisions.

[0092] To improve system performance, in actual applications, the application can also use distributed reinforcement learning, hybrid optimization strategy, stateless decision model, and meta-learning-based optimization strategy to implement the scheme of the application. If a distributed reinforcement learning framework is used, multiple agents (agents) can train in parallel on multiple edge servers and make migration decisions based on local information. This distributed architecture can reduce the computational and communication burden of a single agent and enhance the fault tolerance and scalability of the system. In addition to purely reinforcement learning-based optimization methods, a hybrid optimization framework is proposed that combines traditional optimization algorithms with machine learning methods. For example, heuristic algorithms such as genetic algorithms, simulated annealing, and particle swarm optimization can be combined to handle microservice migration decision problems, especially in cases where the DAG task complexity is high. Such methods may focus more on computational efficiency, especially when there is not enough data, and heuristic algorithms may perform more efficiently. In addition, to avoid the complexity of partially observable Markov decision processes, the application can also try to simplify the problem by using stateless migration models. This approach can eliminate the belief state calculation part of reinforcement learning, simplifying the calculation process, but may lose sensitivity to changes in system state. Although the current scheme uses the PPO algorithm for service migration strategy optimization, meta-learning (Meta-Learning) is a new learning paradigm that can quickly adapt to new task scenarios. For example, a migration strategy based on MAML (Model-Agnostic Meta-Learning) can adapt to multiple DAG tasks in dynamic environments without extensive retraining. This method can significantly reduce model training time and improve adaptability.

[0093] To increase the application fields and application prospects of the application, the technical method can be optimized and expanded in the following aspects:

[0094] 1) Multi-dimensional evaluation and optimization: In addition to the basic factors such as latency and migration cost considered in the present invention, more dimensions of optimization objectives can be further introduced in the future. For example, considering the impact on energy consumption during migration, especially in mobile devices with limited battery life, optimizing the balance between energy consumption and delay will become an important research direction. To this end, energy consumption can be added as an additional optimization objective in the reward function of reinforcement learning.

[0095] (2) Cross-domain service migration: Mobile edge computing not only involves service migration between traditional network edge nodes and user devices, but also may involve service migration between different domains and technologies in the future. For example, services can be migrated between cloud computing, edge computing, and even Internet of Things devices. In such cross-domain service migration, resources in different domains may have different limitations and advantages, so further research is needed to explore how to design cross-domain migration strategies to maximize the utilization and optimization of resources.

[0096] (3) Data privacy and security: Although the microservice migration framework of the present invention effectively improves system performance, in practical applications, how to ensure the privacy and security of user data remains a pressing problem. Future research can explore how to conduct microservice migration while ensuring data privacy and security, such as through encryption communication, privacy computing, and other technologies to ensure the security of data during migration.

[0097] Based on the above embodiments, the present invention also provides a DAG microservice migration device oriented to collaboration and latency sensitivity, as shown in Figure 6 The device includes a model training module 100, a service pre-migration module 200, and a microservice migration module 300. Specifically, the model training module 100 is used to train a service pre-migration model using a service pre-migration algorithm to obtain a trained service pre-migration model. The service pre-migration module 200 is used to deploy the trained service pre-migration model to a user terminal and, when the user terminal moves, the trained service pre-migration model makes a service pre-migration decision based on the acquired local information to obtain a target edge cloud. The microservice migration module 300 is used to make a microservice migration decision for the edge servers in the target edge cloud using a microservice migration algorithm to obtain an edge server to be migrated and execute the migration of the microservice based on the migration decision.

[0098] The working principles of the modules in the DAG microservice migration device oriented to collaboration and latency sensitivity of the present embodiment are the same as those of the steps in the above method embodiments, which will not be repeated here.

[0099] The modules in the above DAG micro-service migration device oriented to collaboration and latency sensitivity can be implemented by software, hardware and combinations thereof, in whole or in part. The modules can be embedded in or independent of a processor in the terminal in hardware, or stored in a memory in the terminal in software, so as to be called and executed by the processor to perform the operations corresponding to the modules.

[0100] Based on the above embodiments, the application further provides a terminal, a principle block diagram of which can be shown as Figure 7 The terminal can include one or more processors 100 (only one is shown in the figure), a memory 101, and a computer program 102 stored in the memory 101 and executable on the one or more processors 100. Figure 7

[0101] In an embodiment, the processor 100 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0102] In an embodiment, the memory 101 can be an internal storage unit of the electronic device, such as a hard disk or a memory of the electronic device. The memory 101 can also be an external storage device of the electronic device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 101 can include both the internal storage unit and the external storage device of the electronic device. The memory 101 is used to store computer programs and other programs and data required by the terminal. The memory 101 can also be used to temporarily store data that has been output or will be output.

[0103] Those skilled in the art can understand that, Figure 7 The principle block diagram shown in the above embodiments is only a block diagram of part of the structure related to the application scheme, and does not constitute a limitation on the terminal to which the application scheme is applied. The specific terminal can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components. ​

[0104] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, operating database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0105] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for collaborative and latency-sensitive DAG microservice migration, characterized in that, The method comprises: Training a service pre-migration model using a service pre-migration algorithm to obtain a trained service pre-migration model; Deploying the trained service pre-migration model to the user end, and when the user end moves, the trained service pre-migration model makes a service pre-migration decision based on the obtained local information to obtain a target edge cloud; Using a micro-service migration algorithm to make a migration decision for micro-services of the edge servers in the target edge cloud to obtain the edge servers to be migrated, and performing migration of the micro-services based on the migration decision, wherein performing migration of the micro-services based on the migration decision is to find the edge servers to be migrated in the target edge cloud for each micro-service in the task to perform migration; Training a service pre-migration model using a service pre-migration algorithm to obtain a trained service pre-migration model, comprising: Developing a PPO-based service pre-migration algorithm using a PPO-based reinforcement learning method, and training a service pre-migration model based on the service pre-migration algorithm to obtain a trained service pre-migration model; wherein the service pre-migration algorithm comprises: an initialization phase, a data collection phase, a calculation of advantage and discount return phase, an update of policy and value function phase, and a policy evaluation and adjustment phase; In computing the advantage and discounted return phases, is the advantage function estimate, representing the expected return at time slot selecting a particular action relative to the average case, computed from the generalized advantage estimates: where, is a discount factor, is a generalized advantage estimation hyperparameter balancing bias and variance, is a time slot of the time-difference error, is an immediate reward, and are value function estimates for the current state and next state, respectively, since the trajectory is finite, the advantage estimate can be computed at the end of the trajectory sequence: in, is the update frequency of the reinforcement learning model. In reinforcement learning, the discounted return Defined as a time slot The weighted sum of all future rewards from the beginning to the end of the trajectory sequence, which is discounted by a factor Adjust, where , discount factor The role of is to control the relative importance of future rewards in current decisions; Discount return Defined as: Since the sequence of trajectories is finite, the discounted return can be computed from time slot to the end of the sequence of trajectories, and can be defined as: In the update of policy and value function phase, this phase involves two steps, comprising: Step i, policy optimization: optimize the policy using the following objective function: where, is the probability ratio, is the initialization policy network, is the hyperparameter controlling the clipping range, clip is the function that limits the value to the interval , the policy network parameters is updated using gradient ascent: , is the learning rate; Step ii, Value network update: Using discounted returns Compute target values to minimize the following loss function: Value network parameters Update by gradient descent: .

2. The collaborative and latency-sensitive DAG microservice migration method of claim 1, wherein, In the initialization phase, use a long short-term memory network to learn the hidden state from the historical observation sequence, capture long-term dependencies, and infer the environment hidden state. 3.The DAG microservice migration method for collaboration and latency sensitivity of claim 1, wherein, Using a micro-service migration algorithm to make a migration decision for micro-services of the edge servers in the target edge cloud to obtain the edge servers to be migrated, comprising: Dividing the sub-tasks in the task into a group of micro-service layers according to topological sorting, and initializing a network flow graph for each micro-service layer; Initializing a network flow graph for each micro-service layer, applying a successive shortest path algorithm to solve the minimum cost maximum flow problem to obtain the optimal matching between the micro-services in each layer and the edge servers, and obtaining an optimal matching set; Based on the optimal matching set, the edge servers to be migrated are obtained.

4. The collaborative and latency-sensitive DAG microservice migration method of claim 3, wherein, Initializing a network flow graph for each micro-service layer, comprising: Adding a source node and a sink node in each network flow graph; for each sub-task in each micro-service layer, add an edge from the source node to each sub-task node, and set the capacity of the edge to 1.

5. The collaborative and latency-sensitive DAG microservice migration method of claim 4, wherein, Initializing a network flow graph for each micro-service layer, further comprising: If the computing demand of the sub-task is less than or equal to the resource capacity of the edge server, add an edge from the sub-task node to the edge server, and set the capacity of the edge to 1.

6. The collaborative and latency-sensitive DAG microservice migration method of claim 4, wherein, Initializing a network flow graph for each micro-service layer, further comprising: All edge servers in the target edge cloud are connected to the sink node, and the capacity of the edge between each edge server and the sink node is the number of available CPU cores on the corresponding edge server.

7. A device for collaborative and latency-sensitive DAG microservice migration, characterized in that, The device is used to implement the steps of the DAG micro-service migration method for collaboration and latency sensitivity according to any one of claims 1-6, and the device comprises: The model training module is configured to train a service pre-migration model by using a service pre-migration algorithm, and obtain a trained service pre-migration model. The service pre-migration module is configured to deploy the trained service pre-migration model to a user terminal, and when the user terminal moves, the trained service pre-migration model makes a service pre-migration decision according to acquired local information, and obtains a target edge cloud. The micro-service migration module is configured to make a migration decision of micro-services of an edge server in the target edge cloud by using a micro-service migration algorithm, obtain an edge server to be migrated, and perform migration of the micro-services based on the migration decision.

8. A terminal, characterized by comprising: The terminal comprises a memory, a processor, and a collaboration and latency sensitive DAG micro-service migration program stored in the memory and executable on the processor. When the processor executes the collaboration and latency sensitive DAG micro-service migration program, the steps of the collaboration and latency sensitive DAG micro-service migration method according to any one of claims 1-6 are implemented.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a collaboration and latency sensitive DAG micro-service migration program. When the processor executes the collaboration and latency sensitive DAG micro-service migration program, the steps of the collaboration and latency sensitive DAG micro-service migration method according to any one of claims 1-6 are implemented.

Images