Active and passive migration method and device of container in meta-calculation, terminal and medium

Abstract

The invention discloses an active and passive migration method and device of a container in meta-computing, a terminal and a medium, the method comprises the following steps: determining total task duration information and constraint conditions of container migration, the container migration comprising active migration and passive migration; based on the total task duration information and the constraint condition, constructing a container migration function model; the container migration function model is explored by adopting a PPCM algorithm with the purpose of outputting the minimum total task duration, a target container migration strategy is generated, and the target container migration strategy is used for reflecting a container migration mode corresponding to the minimum total task duration output in the container migration function model. According to the method, the PPCM algorithm based on the PPO is adopted to explore the container migration function model, the sharing characteristics between layers in the container migration process can be captured, the learning speed and the decision-making efficiency during model exploration are improved, and the container migration cost is optimized.

Description

Technical Field

[0001] The present invention relates to the field of mobile communication technology, and in particular to a method, device, terminal and medium for active and passive migration of containers in meta-computing. Background Art

[0002] Mobile edge computing (MEC) provides a large amount of computing and storage resources at the edge of the network, allowing mobile devices to execute complex applications that meet quality of service (QoS) standards. However, the MEC model mainly relies on the computing power of edge nodes, and the lack of trust between them hinders the integration of all computing resources, resulting in the emergence of computing power islands. Metacomputing integrates cloud, edge and other resources and virtualizes the network into a metacomputer, making the underlying resource management transparent to end users, allowing services to be deployed through containers on the metacomputer and dynamically performing task scheduling to optimize resource utilization.

[0003] However, existing research on meta-computing focuses on using containers for service migration, ignoring the fine-grained storage of images on layer units, neglecting the layer granularity in service migration, not considering layer characteristics, and failing to solve the active and passive migration decision problems of containers in meta-computing.

[0004] Therefore, the prior art still needs to be improved and enhanced. Summary of the invention

[0005] The technical problem to be solved by the present invention is that, in view of the above-mentioned defects of the prior art, a method, device, terminal and medium for active and passive migration of containers in meta-computing are provided, aiming to solve the problem that meta-computing in the prior art does not consider layer characteristics and cannot solve the active and passive migration decisions of containers in meta-computing.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0007] In a first aspect, the present invention provides a method for active and passive migration of containers in meta-computing, wherein the method comprises:

[0008] Determine total task duration information and constraint conditions of container migration, wherein the container migration includes active migration and passive migration;

[0009] Based on the total task duration information and the constraint conditions, a container migration function model is constructed;

[0010] The PPCM algorithm is adopted to explore the container migration function model with the goal of outputting the minimum total task duration, and a target container migration strategy is generated. The target container migration strategy is used to reflect the container migration mode corresponding to the minimum total task duration output in the container migration function model.

[0011] In one implementation, the determining of total task duration information and constraint conditions for container migration, wherein the container migration includes active migration and passive migration, includes:

[0012] Obtaining first delay information generated by container migration, second delay information generated by migration queuing, and third delay information generated by task downloading, and determining a migration delay based on the first delay information, the second delay information, and the third delay information;

[0013] Acquire fourth delay information generated by executing the task, and determine a calculation delay based on the fourth delay information;

[0014] Acquire fifth delay information generated by container access, and determine deployment delay based on the fifth delay information;

[0015] Acquire sixth delay information between the container and the user, and determine a return delay based on the sixth delay information;

[0016] A total task duration of container migration is determined based on the migration delay, the calculation delay, the deployment delay, and the return delay.

[0017] In one implementation, the determining of total task duration information and constraint conditions of container migration, wherein the container migration includes active migration and passive migration, further includes:

[0018] Obtain the maximum number of containers and computing power threshold of each node to determine the first constraint condition;

[0019] Obtain the deployment scope of each task and determine the second constraint condition;

[0020] Based on the first constraint condition and the second constraint condition, a constraint condition for container migration is determined.

[0021] In one implementation, the PPCM algorithm is used to explore the container migration function model with the goal of outputting the minimum total task duration, and a target container migration strategy is generated. The target container migration strategy is used to reflect the container migration mode corresponding to the minimum total task duration output in the container migration function model, including:

[0022] Using the PPCM algorithm, the exploration process of the container migration function model is transformed into a process of finding the maximum reward;

[0023] According to the maximized reward, the target container migration strategy corresponding to the output of the minimum total task duration is determined, where:

[0024] The reward is expressed as: r t =-T k ;

[0025] The exploration process of the container migration function model is expressed as: minT = ∑ k∈K T k ;

[0026] The process of finding the maximum reward is expressed as: Among them, t is the time point when the user's location changes, k is the task created by the user at time point t, and r t For reward, T k is the total task duration, γ t is the discount factor, γ t ∈(0,1).

[0027] In one implementation, before the PPCM algorithm is used to transform the exploration process of the container migration function model into a process of finding a maximum reward, the following steps are included:

[0028] Obtaining node status and task status in the container migration function model;

[0029] Generate a target container migration strategy according to the node status and the task status, wherein:

[0030] The node status is:

[0031] Where n is a node, N is a set of nodes, |N| is the number of nodes, u is a mobile user, is the position coordinate of the edge node, is the distance between the mobile user and the node for the next task, is the download size of the task on the node, is the total size of all running tasks on the node, is the state of each layer where the node is located, f n is the CPU resource, b n is bandwidth;

[0032] The task status is:

[0033] Where k is the task, |L| is the number of layers in the container, and L u (t) is the location coordinate of the mobile user, f u is the resources required for the task, b u is the bandwidth required for the task, D u (t) is the download size required for the task.

[0034] In one implementation, generating a target container migration strategy according to the node status and the task status includes:

[0035] The node state and the task state are input into the policy network, and the DCN network is used to extract the feature relationship within and between layers to obtain the layer shared features;

[0036] Inputting the layer shared features into the value network, using a multi-head attention mechanism to perform state evaluation on the layer shared features, and obtaining a state evaluation result;

[0037] A target container migration strategy is generated according to the status assessment result.

[0038] In one implementation, the policy network and the value network are updated using a policy gradient-based algorithm, wherein the loss function of the policy gradient is:

[0039] Among them, E is the expected function, s is the state, a is the execution action, θ is the given parameter, θ k is the parameter of the old strategy, π θ is the strategy with given parameters θ, π θ (a|s) is the probability that the new strategy executes action a in state s, is the probability of the old strategy executing action a in state s, is the advantage function, which is used to measure the performance of executing action a in state s relative to the average level. The function clip(x,y,z)=max(min(x,z),y) limits x to the interval [y,z], where ∈ represents the range of clip.

[0040] In a second aspect, an embodiment of the present invention further provides an active and passive migration device for containers in meta-computing, wherein the device comprises:

[0041] A migration duration determination module, used to determine the total task duration information and constraint conditions of container migration, wherein the container migration includes active migration and passive migration;

[0042] A migration model building module, used to build a container migration function model based on the total task duration information and the constraint conditions;

[0043] The migration strategy generation module is used to adopt the PPCM algorithm to explore the container migration function model with the goal of outputting the minimum total task duration, and generate a target container migration strategy. The target container migration strategy is used to reflect the container migration mode corresponding to the minimum total task duration output in the container migration function model.

[0044] In a third aspect, an embodiment of the present invention further provides a terminal, wherein the terminal includes a memory, a processor, and an active and passive migration program for containers in meta-computing stored in the memory and executable on the processor, and when the processor executes the active and passive migration program for containers in meta-computing, the steps of the active and passive migration method for containers in meta-computing of any one of the above-mentioned schemes are implemented.

[0045] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores active and passive migration programs for containers in meta-computing, and when the active and passive migration programs for containers in meta-computing are executed by a processor, the steps of the active and passive migration method for containers in meta-computing described in any one of the above-mentioned schemes are implemented.

[0046] Beneficial effects: Compared with the prior art, the present invention provides an active and passive migration method for containers in meta-computing, firstly determining the total task duration information and constraints of container migration, wherein the container migration includes active migration and passive migration, and then constructing a container migration function model based on the total task duration information and the constraints, and finally, using the PPCM algorithm to explore the container migration function model with the goal of outputting the minimum total task duration, generating a target container migration strategy, wherein the target container migration strategy is used to reflect the container migration mode corresponding to the output of the minimum total task duration in the container migration function model. The present invention uses a PPCM algorithm based on PPO to explore the container migration function model, which can capture the shared features between layers in the container migration process, improve the learning speed and decision-making efficiency during model exploration, and optimize the long-term migration cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A flowchart of a specific implementation of the active and passive migration method of containers in meta-computing provided by an embodiment of the present invention.

[0048] Figure 2 A schematic diagram of container migration of a metacomputing system provided in an embodiment of the present invention.

[0049] Figure 3 This is a PPCM algorithm framework diagram provided by an embodiment of the present invention.

[0050] Figure 4 This is an analysis chart of experimental results of rewards, policy network loss, and value function loss of the PPCM algorithm provided in an embodiment of the present invention.

[0051] Figure 5 A functional schematic diagram of an active and passive migration device for containers in meta-computing provided in an embodiment of the present invention.

[0052] Figure 6A functional block diagram of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples.

[0054] Those skilled in the art will appreciate that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Unless specifically stated, the singular forms "one", "said", and "the" used herein may also include plural forms, and the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or their groups. It should be understood that when an element is "connected" or "coupled" to another element, it may be directly connected or coupled to other elements, or there may also be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0055] This embodiment provides an active and passive migration method for containers in meta-computing. In specific implementation, this embodiment first determines the total task duration information and constraints of container migration, and the container migration includes active migration and passive migration. Then, based on the total task duration information and the constraints, a container migration function model is constructed. Finally, the PPCM algorithm is used to explore the container migration function model with the goal of outputting the minimum total task duration, and a target container migration strategy is generated. The target container migration strategy is used to reflect the container migration mode corresponding to the output of the minimum total task duration in the container migration function model. It can be seen that this embodiment uses the PPCM algorithm based on PPO to explore the container migration function model, which can capture the shared features between layers in the container migration process, improve the learning speed and decision-making efficiency during model exploration, and optimize the long-term migration cost.

[0056] The active and passive migration method of containers in meta-computing of this embodiment can be applied to a terminal, which can be an intelligent device such as a mobile phone or a computer. In this embodiment, the terminal can be an external device connected to the active and passive migration device of containers in meta-computing, or it can be a device built into the active and passive migration device of containers in meta-computing.

[0057] like Figure 1 As shown in , the active and passive migration method of containers in meta-computing of this embodiment includes the following steps:

[0058] Step S100: Determine total task duration information and constraint conditions of container migration, where the container migration includes active migration and passive migration.

[0059] like Figure 2 As shown, this embodiment first constructs a multi-user meta-computing system model, where a group of mobile users are distributed in a specific area U = {u1, u2, ..., u |U|}, where |U| represents the size of the user set. In this meta-computing system model, mobile users can offload their computing tasks to edge node services. Since the user's location only changes at the beginning of each time slice, this embodiment can regard this operation process as different time points T=

[0060] {t1,t2,…,t T|}, where the set of computational tasks generated by the user is represented by K = {k1, k2, …, k K|}, |K| represents the size of the generated computing task set, and the edge node set is represented by N = {n1,n2,…,n |N|}, located at the edge of the core network, |N| represents the size of the edge node set, and the cloud is regarded as an edge node with unlimited computing resources n |N|+1 In this embodiment, each node has its CPU frequency f n , bandwidth bn and storage space b n For cloud nodes, the CPU frequency is f cloud , bandwidth is b cloud In order to process computing tasks, this embodiment represents various containers established and deployed on edge nodes as C = {c1, c2, ..., c |C|}, |C| represents the size of the container set. In addition, the execution of the container also requires an image file, which is represented as M = {m1,m2,…,m |M|}, |M| represents the size of the image file set. The image contains related layers, represented by L = {l1,l2,…,l |L|}, |L| represents the set size of the layer, and the distance between each layer l∈L is d l The set of layers in a container c∈C is denoted as L c = in Indicates that container c contains layer l, otherwise For each task k∈K created by a user at time t, the requested CPU resource is p k , the specified container is c k After scheduling, the nodes assigned to the task are represented as in indicates that task k is assigned to node n, otherwise

[0061] In one implementation, step S100 of this embodiment includes the following steps:

[0062] Step S101, obtaining first delay information generated by container migration, second delay information generated by migration queuing, and third delay information generated by task downloading, and determining migration delay based on the first delay information, the second delay information, and the third delay information;

[0063] Step S102, obtaining fourth delay information generated by executing the task, and determining a calculation delay based on the fourth delay information;

[0064] Step S103: Acquire fifth delay information generated by container access, and determine deployment delay based on the fifth delay information;

[0065] Step S104: Acquire sixth delay information between the container and the user, and determine a return delay based on the sixth delay information;

[0066] Step S105: Determine the total task duration of container migration based on the migration delay, the calculation delay, the deployment delay and the return delay.

[0067] Specifically, in the metacomputing system, mobile user u requests service from the edge node at time slice t, which is expressed as Typically, servers are interconnected through stable backhaul links, so that mobile users can access services through multi-hop communications between edge nodes, even if they are not directly connected to the edge node providing the service. For best results, services should be dynamically migrated between nodes. Migration is divided into two parts: active migration and passive migration. Active migration is triggered when the current node lacks CPU and memory resources, resulting in task recycling and queuing. The resources occupied by the task on the node are then released. Passive migration occurs when the mobile user exceeds the communication range of the node. Since the service range of each edge node is limited, exceeding the range forces the user to switch to another node. If no node meets the conditions during the migration process, the task will be migrated to the cloud. Tasks move dynamically between nodes, generating delays, and the final task time includes migration delay, computation delay, deployment delay, and backhaul delay.

[0068] Furthermore, this embodiment obtains the first delay information generated by container migration, the second delay information generated by migration queuing, and the third delay information generated by task downloading to determine the migration delay. Migration delay occurs when a task moves from one node to another, and includes three components, among which the first delay information is generated by container migration, which is expressed as:

[0069]

[0070] Among them, d t Indicates the current node and the previous node The distance between is a positive coefficient, Indicates the network bandwidth on the migration path. Indicates the size of the migration service. If d t =0, which means no migration will occur and the migration delay is 0; if migration occurs,

[0071] The second delay information is generated by the migration queue. If the task needs to be migrated, it will be put back into the task queue and wait for the selection of a new node, thus generating additional waiting time. Let the current time slice be $t$ and the time slice for entering the task queue be t u , the second delay information is expressed as: w(u)=tt u .

[0072] The third delay information is generated by task downloading and is expressed as: in, represents the layer size required by user u, b n Indicates the bandwidth of the node. For cloud nodes, b n =b cloud Each node has a layer download queue. When a new layer needs to be downloaded, it joins this queue and may have to wait for other layers to complete. Therefore, the download time of task k is equal to the maximum completion time of all necessary layers. Indicates the completion time of layer l on node n. If the layer already exists or has not yet started downloading, then Otherwise, it is expressed as:

[0073] Since the movement and download after migration on each node can occur simultaneously, we only need to consider the maximum of the two, so the migration delay of this embodiment is:

[0074] In addition, mobile users assign computing tasks to nodes for processing, which will generate computing time. In time slice t, the task size The processing density of the task offloaded by user u is denoted as κ. Therefore, the CPU cycles required to process the offloaded task can be calculated as At the same time, the workload of the node w t It is defined as the product of the size of all tasks running on a node and the processing density, that is, The computing power of a node is defined as f(n t ). For cloud f(n t )=f(c t). Assume that resources are distributed proportionally among nodes, meaning that tasks receive computing resources based on their CPU cycle requirements. Therefore, the computational latency of this embodiment is:

[0075] In this embodiment, the fifth delay information generated by container access includes the access delay between the mobile user and the node and the time to deploy to the node for downloading, wherein the access delay between the mobile user and the node is defined as: in, represents the offload task size ratio of user u in time slice t, ρ t = P = 1 / 20 / 2019 1:10 ...

[0076] max(A,D).

[0077] The backhaul delay in this embodiment is the delay between the edge node and the user, which is defined as:

[0078] Among them, y t Indicates the distance from the node to the user, is a positive coefficient, Indicates the network bandwidth of the return path.

[0079] In summary, the total task duration of container migration in this embodiment is the total duration caused by migration delay, calculation delay, deployment delay and return delay, that is: T k =S+C(n t )+P+B(u).

[0080] In one implementation, step S100 of this embodiment further includes the following steps:

[0081] Step S106: Obtain the maximum container capacity and computing capacity threshold of each node to determine the first constraint condition;

[0082] Step S107: Obtain the deployment range of each task and determine the second constraint condition;

[0083] Step S108: Determine a constraint condition for container migration based on the first constraint condition and the second constraint condition.

[0084] Specifically, the container migration process of this embodiment also needs to meet certain constraints. First, for each node, the maximum number of containers that can be executed simultaneously is limited, so the constraints that need to be met include: In addition, each node is limited by storage resources and computing power. If the resources of the current node are lower than the specified threshold at time slice t, active migration is required, that is, the constraints are met:

[0085]

[0086]

[0087] in, indicates whether layer l is active on node n at time t, σ mem and σ cpu They are all positive threshold control coefficients.

[0088] Secondly, for each task, since each task can only be deployed to one node or cloud at a time, the constraints that need to be met include:

[0089] Step S200: construct a container migration function model based on the total task duration information and the constraint conditions.

[0090] Since the goal of this embodiment is to minimize the total task duration of the dynamic meta-computing system while satisfying the constraints, this embodiment can construct a container migration function model based on the total task duration and the constraints as follows: minT = ∑ k∈K T k , when solving this model, the above constraints need to be met. Furthermore, the model problem solved in this embodiment can be regarded as a complex packing problem, which can only be solved by heuristic methods. Generally speaking, traditional algorithms can only make iterative decisions based on set strategies at each time step, lacking insight into the influencing factors between adjacent time steps and the ability to dynamically adjust strategies according to the meta-computing system. It is also very time-consuming to make decisions iteratively on time slices. Preferably, an active and passive container migration (PPCM) algorithm based on the perception layer of proximal policy optimization (PPO) is provided in this embodiment for solving the above-mentioned model problem.

[0091] Step S300: Using the PPCM algorithm, with the goal of outputting the minimum total task duration, the container migration function model is explored to generate a target container migration strategy, where the target container migration strategy is used to reflect the container migration mode corresponding to the minimum total task duration output in the container migration function model.

[0092] In one implementation, step S300 of this embodiment includes the following steps:

[0093] Step S301, using the PPCM algorithm to transform the exploration process of the container migration function model into a process of finding a maximum reward;

[0094] Step S302: Determine the target container migration strategy corresponding to the minimum total task duration when outputting the maximum reward, wherein the reward is expressed as: r t =-T k ;

[0095] The exploration process of the container migration function model is expressed as: minT = ∑ k∈K T k ;

[0096] The process of finding the maximum reward is expressed as: Among them, t is the time point when the user's location changes, k is the task created by the user at time point t, and r t For reward, T k is the total task duration, γ t is the discount factor, γ t ∈(0,1).

[0097] Specifically, before adopting the PPCM algorithm to transform the exploration process of the container migration function model into a process of finding the maximum reward, the present embodiment includes the following steps:

[0098] Step S3001, obtaining the node status and task status in the container migration function model;

[0099] Step S3002: Generate a target container migration strategy according to the node state and the task state, wherein the node state is:

[0100] Where n is a node, N is a set of nodes, |N| is the number of nodes, u is a mobile user, is the position coordinate of the edge node, is the distance between the mobile user and the node for the next task, is the download size of the task on the node, is the total size of all running tasks on the node, is the state of each layer where the node is located, f n is the CPU resource, b n is bandwidth;

[0101] The task status is: Among them, k is the task, |L| is the number of layers of the container, and L u (t) is the location coordinate of the mobile user, f u is the resources required for the task, b u is the bandwidth required for the task, D u (t) is the download size required for the task.

[0102] The states in the container migration function model of this embodiment provide a comprehensive overview of meta-computing, including nodes and tasks. Contains the location coordinates of edge nodes include and As well as the distance between the mobile user of the next task and the node, the total size of the next task to be downloaded on the node, and the resource status of the node including the CPU resource f n , bandwidth b n , the total size of all running tasks on the node, and the status of each layer owned by the node, so the final node status is: The task status Including the location coordinates L of the mobile user u (t), including x u (t) and y u (t), and the resources required for the task (such as CPU resources f u and bandwidth b u ), the total download size required by mobile users for the upcoming task D u (t), the layer state required for the next task Therefore, the final task status is: Therefore, the overall state of the algorithm in this embodiment is:

[0105] In addition, the action space in this embodiment includes all nodes and clouds, which is: a t ∈N∪{n |N|+1 At the same time, this embodiment transforms the exploration process of the container migration function model into the process of finding the maximum reward. The algorithm aims to maximize the reward, while in meta-computation, the goal is to minimize the total task time. Therefore, the reward calculation is expressed as: t =-T k , in the long run, the cumulative reward is: T k is the total task time, γ t is the discount factor, γ t ∈(0,1).

[0106] In one implementation, step S3002 of this embodiment specifically includes the following steps:

[0107] Step S3012: input the node state and the task state into the policy network, use the DCN network to extract the feature relationship within and between layers, and obtain the layer shared features;

[0108] Step S3022: input the layer shared features into the value network, and use a multi-head attention mechanism to perform state evaluation on the layer shared features to obtain a state evaluation result;

[0109] Step S3032: Generate a target container migration strategy according to the status assessment result.

[0110] In order to avoid the problem of inefficient learning caused by the exponential expansion of exploration space in meta-computation, especially in scenarios with limited rewards, traditional reinforcement learning often starts from random exploration. The PPCM algorithm proposed in this embodiment proposes to pre-train the agent through expert demonstrations collected by conventional means. The method includes first introducing a baseline algorithm based on expert demonstrations, promoting agent pre-training by storing demonstration results in an experience buffer, using imitation learning as a hot start, and then applying a traditional reinforcement learning algorithm to optimize the pre-training strategy. At the same time, in order to prevent the demonstration from having an excessive impact on the learning strategy, the number of demonstration samples is gradually reduced during the pre-training stage.

[0111] Figure 3 The PPCM algorithm framework diagram provided for this embodiment, the agent observes the node and task status, embeds and connects them, and then inputs them into the policy network for action selection, migration and reward generation. The policy network and value function are updated using a policy gradient-based algorithm. In the policy network, this embodiment uses DCN to extract the interdependencies within and between layers. The deep model efficiently learns low-dimensional feature intersections and high-dimensional nonlinear features while requiring minimal computing resources. Among them, the network input features are divided into dense features and sparse features. The sparse feature input includes the layer state of each edge node, while the dense feature input includes other states. The low-dimensional cross combination of the cross network layer learning features is expressed as: Among them, x l and x l+1 The column vectors representing the output of the lth layer and the (l+1)th cross layer, respectively, w l and b l In the deep network, this embodiment uses a fully connected neural network to learn the cross-combination of high-dimensional nonlinear features. The formula is as follows: l+1 =f(W l h l +b l ), where h l and h l+1 represents the lth and (l+1)th hidden layers, W l and change lis the parameter of the lth layer, and f(·) represents the ReLU function. Finally, the outputs of the cross network and the deep network are merged to create a combined layer. In the value network, this embodiment adopts a multi-head attention mechanism to evaluate the state value, which can merge the encoded representation information from different subspaces, use multiple self-attention sets to analyze the initial input sequence, and then integrate the results of each self-attention set for linear transformation to obtain the final output result, which is as follows: MultiHead(Q,K,V)=Concat(head1,…,head h )W O , where Q, K, and V are the query matrix, key matrix, and value matrix, respectively.

[0112] In one implementation, the policy network and the value network of this embodiment are updated using an algorithm based on policy gradient. Specifically, the loss function of the policy gradient is:

[0113] Among them, E is the expected function, s is the state, a is the execution action, θ is the given parameter, θ k is the parameter of the old strategy, π θ is the strategy with given parameters θ, π θ (a|s) is the probability that the new strategy executes action a in state s, is the probability of the old strategy executing action a in state s, is the advantage function, which is used to measure the performance of executing action a in state s relative to the average level. The function clip(x,y,z)=max(min(x,z),y) limits x to the interval [y,z], where ∈ represents the range of clip.

[0114] Since the PPCM algorithm in this embodiment is designed based on the PPO algorithm, PPO is built on the policy gradient algorithm to ensure efficiency and low computational complexity. θ , the optimization goal of PPO is: In addition, PPO uses the generalized advantage estimator for advantage calculation, which can be calculated as follows: λ represents the generalized advantage estimation (GAE) parameter, δ t represents the time difference error at time step t, and V represents the approximate function.

[0115] Table 1 below is an example of the PPCM algorithm of this embodiment (Algorithm 1). It can be seen that Algorithm 1 initializes the replay memory D for each time point. i (Line 2). Expert demonstrations occur during preprocessing, and at each time step t, the expert policy selects an action and records the state transition (st ,a t ,r t ,s t+1 ) in D i (Lines 4-8). Then, at each time step t, the algorithm observes the current state s t (Line 11) and determine the migration type. If a new task arrives and the task queue is not empty (Line 12). If the node resources are insufficient, perform active migration (Lines 14-15). If the user is out of communication range, perform passive migration (Lines 16-17). After migration, perform status update (Line 19). Based on the current strategy π θ Select and perform an action t , observe the reward and next state, and record the state transition in D i Finally, in the training phase, for each training step j in each episode i, the policy update is calculated (lines 26-29), where the update in this embodiment is completed by maximizing the objective function using the stochastic gradient ascent algorithm of the Adam algorithm.

[0116]

[0117] Table 1 PPCM algorithm

[0118] In order to further prove that the PPCM algorithm of this embodiment has significant performance improvement effect compared with traditional image-based algorithms and layer sharing-based heuristic algorithms, relevant experiments were carried out in this embodiment, and DEPSOFT, KUBE, MONKEY, DOWN, GA, DE, PSO, PPO and PPCM algorithms were used to explore migration strategies.

[0119] Among them, DEP SOFT: is a layer-based scheduling algorithm that uses the local layer size to calculate the score. At the same time, a threshold is established and a node is randomly selected from the nodes that exceed the threshold. KUBE: The algorithm is image-based. As one of the default scheduling algorithms of Kubernetes, it evaluates the distribution of requested images on nodes and assigns scores based on the available image sizes. The scheduling is determined based on this score. MONKEY: A random algorithm that randomly selects a node for scheduling. DOWN: When calculating the score, the size of the layer is changed to the estimated download time. GA: A metaheuristic algorithm inspired by the natural selection process that relies on biologically inspired operations. DE: Differential evolution optimizes problems by iteratively trying to improve candidate solutions relative to a given quality metric. PSO: Particle swarm optimization is a metaheuristic algorithm that solves the problem of a group of candidate solutions moving in the search space according to a simple mathematical formula. PPO: PPCM algorithm without expert demonstration.

[0120] In the experiment, the hyperparameters of the PPCM algorithm are shown in Table 2 below. In this embodiment, the actual movement trajectory in Rome, Italy is used to evaluate the algorithm. Nine edge nodes are deployed in each area, each node covering a 1×1 km grid, with a computing power of 128 GHz. The upload rate of each grid is gradually set to 60 Mbps, 48 Mbps, 36 Mbps, 24 Mbps, and 12 Mbps. The distance between two edge nodes is calculated using the Manhattan distance. The migration delay varies according to the service size and network conditions. It is assumed that the service size is uniformly distributed in the range of [0.5, 100] MB, and the migration delay factor in the training phase is uniformly distributed in the range of [1.0, 3.0] s / hop. At each time step, the tasks of mobile users and edge nodes are randomly drawn from a Poisson distribution. According to empirical data, the data size of each offloading task is uniformly distributed in the range of [0.05, 5] MB. The service size is randomly selected in the range of [0.5, 100] MB, and the processing density of the nodes is uniformly distributed in the range of [200, 10,000] cycles. Active migration depends on the resource threshold. The positive coefficient of the resource is 0.2 as the trigger threshold, which means that migration occurs when the resources are less than 20%. The node resource limit is in the range of [30, 100] GB, the wired transmission bandwidth is in the range of [800, 1024] Mbps, the backhaul bandwidth is in the range of [400, 600] Mbps, the backhaul delay factor is fixed at 0.02 s / hop, and the node supports a maximum of 30 containers.

[0121] TABLE I:Hyperparameter Settings of PPCM Algorithm

[0122]

[0123] Table 2 Hyperparameters of the PPCM Algorithm

[0124] It can be seen from Tables 3, 4, 5, 6, and 7 that as the number of tasks increases, the performance hierarchy based on the total delay is PPCM < PPO < DEP SOFT < DOWN < GA < DE < MONKEY < PSO < KUBE, and still the PPCM algorithm performs excellently. It can be seen from Tables 8, 9, 10, 11, and 12 that as the number of nodes increases, the average task delay also increases; as the number of nodes increases, the total task delay also increases. In terms of the total delay relationship, the hierarchical relationship is as follows: PPCM < PPO < DEP SOFT < DOWN < PSO < DE < GA < MONKEY < KUBE. There may be slight deviations in the intermediate algorithms. However, regardless of the number of nodes, PPCM is always superior to other algorithms. In addition, Figure 4The convergence of the PPCM algorithm is demonstrated in Figure 1, which shows that the policy network loss and the value function loss have high values ​​at the beginning of training. However, as the training steps progress, both decrease rapidly and stabilize near specific values, indicating that the PPCM algorithm in this embodiment has convergence.

[0125] Number of tasks DE DEP_SOFT DOWN GA KUBE MONKEY PSO PPO PPCM 1500 945.7 890.9 903.9 935.2 937.5 939.3 946.6 872.6 878 2000 903.9 865.7 892.8 895 905.3 907.7 904 858.16 844.4 2500 897.3 847 897.3 894.1 901.7 900.1 903.9 834.4 829.2 3000 885.6 851 893.8 893.4 877.2 894.8 895.1 843.2 836 3500 836.5 796.7 827.3 834.8 833.4 836.3 839.4 790.8 779.3

[0126] Table 3 The impact of the number of tasks on the deployment delay of various algorithms

[0127]

[0128] Table 4 Effect of the number of tasks on the return delay of various algorithms

[0129]

[0130] Table 5 The impact of the number of tasks on the migration delay of various algorithms

[0131]

[0132] Table 6 The impact of the number of tasks on the computational delay of various algorithms

[0133]

[0134] Table 7 The impact of the number of tasks on the total delay of various algorithms

[0135]

[0136] Table 8 Impact of the number of nodes on the deployment delay of various algorithms

[0137]

[0138] Table 9 The impact of the number of nodes on the return delay of various algorithms

[0139]

[0140] Table 10 The impact of the number of nodes on the migration delay of various algorithms

[0141]

[0142] Table 11 The impact of the number of nodes on the calculation delay of various algorithms

[0143]

[0144] Table 12 The impact of the number of nodes on the total delay of various algorithms

[0145] In summary, this embodiment provides an active and passive migration method for containers in meta-computing. In specific implementation, this embodiment first determines the total task duration information and constraints of container migration, and the container migration includes active migration and passive migration. Then, based on the total task duration information and the constraints, a container migration function model is constructed. Finally, the PPCM algorithm is adopted to explore the container migration function model with the goal of outputting the minimum total task duration, and a target container migration strategy is generated. The target container migration strategy is used to reflect the container migration mode corresponding to the output of the minimum total task duration in the container migration function model. This embodiment adopts the PPCM algorithm based on PPO to explore the container migration function model, which can capture the shared features between layers in the container migration process, improve the learning speed and decision-making efficiency during model exploration, and optimize the long-term migration cost.

[0146] Based on the above embodiments, the present invention also provides an active and passive migration device for containers in meta-computing, such as Figure 5 As shown in , the active and passive migration device of containers in meta-computing includes: a migration duration determination module 10, a migration model construction module 20, and a migration strategy generation module 30. Specifically, the migration duration determination module 10 is used to determine the total task duration information and constraints of container migration, and the container migration includes active migration and passive migration. The migration model construction module 20 is used to construct a container migration function model based on the total task duration information and the constraints. The migration strategy generation module 30 is used to adopt the PPCM algorithm, with the goal of outputting the minimum total task duration, to explore the container migration function model and generate a target container migration strategy, and the target container migration strategy is used to reflect the container migration mode corresponding to the output of the minimum total task duration in the container migration function model.

[0147] In one implementation, the migration duration determination module 10 includes:

[0148] a migration delay determination unit, configured to obtain first delay information generated by container migration, second delay information generated by migration queuing, and third delay information generated by task downloading, and determine the migration delay based on the first delay information, the second delay information, and the third delay information;

[0149] a computing delay determining unit, configured to obtain fourth delay information generated by executing the task, and determine the computing delay based on the fourth delay information;

[0150] a deployment delay determining unit, configured to obtain fifth delay information generated by container access, and determine a deployment delay based on the fifth delay information;

[0151] a return delay determining unit, configured to obtain sixth delay information between the container and the user, and determine a return delay based on the sixth delay information;

[0152] A total task duration determining unit is used to determine the total task duration of container migration based on the migration delay, the calculation delay, the deployment delay and the return delay.

[0153] In one implementation, the migration duration determination module 10 further includes:

[0154] A node constraint condition acquisition unit, used to acquire the maximum container capacity and computing capacity threshold of each node, and determine the first constraint condition;

[0155] A task constraint condition acquisition unit, used to acquire the deployment range of each task and determine the second constraint condition;

[0156] The constraint condition determination unit is used to determine the constraint condition of container migration based on the first constraint condition and the second constraint condition.

[0157] In one implementation, the migration strategy generation module 30 includes:

[0158] An exploration process conversion unit, used to convert the exploration process of the container migration function model into a process of finding a maximum reward by using a PPCM algorithm;

[0159] The minimum total task duration conversion unit is used to determine the target container migration strategy corresponding to the minimum total task duration when outputting the maximum reward, wherein the reward is expressed as: r t =-T k ; The exploration process of the container migration function model is expressed as: minT = ∑ k∈K T k ; The process of finding the maximum reward is expressed as: Among them, t is the time point when the user's location changes, k is the task created by the user at time point t, and r t For reward, T k is the total task duration, γ t is the discount factor, γ t ∈(0,1).

[0160] In one implementation, before the PPCM algorithm is used to transform the exploration process of the container migration function model into a process of finding a maximum reward, the following steps are included:

[0161] A node status and task status acquisition unit, used to acquire the node status and task status in the container migration function model;

[0162] The target container migration strategy generating unit is used to generate a target container migration strategy according to the node state and the task state, wherein the node state is:

[0163] Where n is a node, N is a set of nodes, |N| is the number of nodes, u is a mobile user, is the position coordinate of the edge node, is the distance between the mobile user and the node for the next task, is the download size of the task on the node, is the total size of all running tasks on the node, is the state of each layer where the node is located, f n is the CPU resource, b n is bandwidth;

[0164] The task status is: Among them, k is the task, |L| is the number of layers of the container, and L u (t) is the location coordinate of the mobile user, f u is the resources required for the task, b u is the bandwidth required for the task, D u (t) is the download size required for the task.

[0165] In one implementation, the target container migration strategy generation unit further includes:

[0166] A layer shared feature acquisition subunit, used to input the node state and the task state into the policy network, and use the DCN network to extract the feature relationship within and between layers to obtain the layer shared feature;

[0167] A state evaluation result determination subunit is used to input the layer shared features into the value network, use a multi-head attention mechanism to perform state evaluation on the layer shared features, and obtain a state evaluation result;

[0168] The target container migration strategy generation subunit is used to generate a target container migration strategy according to the status evaluation result.

[0169] In one implementation, the policy network and the value network are updated using a policy gradient-based algorithm, wherein the loss function of the policy gradient is:

[0170] Among them, E is the expected function, s is the state, a is the execution action, θ is the given parameter, θ k is the parameter of the old strategy, π θ is the strategy with given parameters θ, π θ (a|s) is the probability that the new strategy executes action a in state s, is the probability of the old strategy executing action a in state s, is the advantage function, which is used to measure the performance of executing action a in state s relative to the average level. The function clip(x,y,z)=max(min(x,z),y) limits x to the interval [y,z], where ∈ represents the range of clip.

[0171] The working principles of each module in the active and passive migration device of the container in the meta-computing of this embodiment are the same as the principles of each step in the above method embodiment, and will not be repeated here.

[0172] Based on the above embodiment, the present invention further provides a terminal, the principle block diagram of the terminal can be as follows: Figure 6 The terminal may include one or more processors 100 ( Figure 6 Only one is shown in the figure), a memory 101 and a computer program 102 stored in the memory 101 and executable on one or more processors 100, for example, an active and passive migration program for containers in meta-computing. When one or more processors 100 execute the computer program 102, the various steps in the embodiment of the method for active and passive migration of containers in meta-computing can be implemented. Alternatively, when one or more processors 100 execute the computer program 102, the functions of the various modules / units in the embodiment of the apparatus for active and passive migration of containers in meta-computing can be implemented, which is not limited here.

[0173] In one embodiment, the processor 100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0174] In one embodiment, the memory 101 may be an internal storage unit of an electronic device, such as a hard disk or memory of the electronic device. The memory 101 may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device. Further, the memory 101 may also include both an internal storage unit of the electronic device and an external storage device. The memory 101 is used to store computer programs and other programs and data required by the terminal. The memory 101 may also be used to temporarily store data that has been output or is to be output.

[0175] Those skilled in the art will understand that Figure 6 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the terminal to which the scheme of the present invention is applied. The specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0176] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, operating database or other media used in the embodiments provided by the present invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double operational data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0177] In summary, the present invention discloses a method, device, terminal and medium for active and passive migration of containers in meta-computing, wherein the method includes: determining the total task duration information and constraints of container migration, wherein the container migration includes active migration and passive migration; constructing a container migration function model based on the total task duration information and the constraints; using the PPCM algorithm, with the goal of outputting the minimum total task duration, exploring the container migration function model, and generating a target container migration strategy, wherein the target container migration strategy is used to reflect the container migration mode corresponding to the output of the minimum total task duration in the container migration function model. The present invention uses a PPCM algorithm based on PPO to explore the container migration function model, which can capture the shared features between layers in the container migration process, improve the learning speed and decision-making efficiency during model exploration, and optimize the long-term migration cost.

[0178] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for active and passive migration of containers in metacomputing, characterized in that: The method comprises: Determine total task duration information and constraint conditions of container migration, wherein the container migration includes active migration and passive migration; Based on the total task duration information and the constraint conditions, a container migration function model is constructed; The PPCM algorithm is adopted to explore the container migration function model with the goal of outputting the minimum total task duration, and a target container migration strategy is generated. The target container migration strategy is used to reflect the container migration mode corresponding to the minimum total task duration output in the container migration function model.

2. The method for active and passive migration of containers in metacomputing according to claim 1, characterized in that: The determining of the total task duration information and constraint conditions of the container migration, wherein the container migration includes active migration and passive migration, includes: Obtaining first delay information generated by container migration, second delay information generated by migration queuing, and third delay information generated by task downloading, and determining a migration delay based on the first delay information, the second delay information, and the third delay information; Acquire fourth delay information generated by executing the task, and determine a calculation delay based on the fourth delay information; Acquire fifth delay information generated by container access, and determine deployment delay based on the fifth delay information; Acquire sixth delay information between the container and the user, and determine a return delay based on the sixth delay information; A total task duration of container migration is determined based on the migration delay, the calculation delay, the deployment delay, and the return delay.

3. The method for active and passive migration of containers in metacomputing according to claim 2, characterized in that: The determining of total task duration information and constraint conditions of container migration, wherein the container migration includes active migration and passive migration, further includes: Obtain the maximum number of containers and computing power threshold of each node to determine the first constraint condition; Obtain the deployment scope of each task and determine the second constraint condition; Based on the first constraint condition and the second constraint condition, a constraint condition for container migration is determined.

4. The method for active and passive migration of containers in metacomputing according to claim 1, characterized in that: The PPCM algorithm is used to explore the container migration function model with the goal of outputting the minimum total task duration, and a target container migration strategy is generated. The target container migration strategy is used to reflect the container migration mode corresponding to the minimum total task duration output in the container migration function model, including: Using the PPCM algorithm, the exploration process of the container migration function model is transformed into a process of finding the maximum reward; According to the maximized reward, the target container migration strategy corresponding to the output of the minimum total task duration is determined, where: The reward is expressed as: r t =-T k ; The exploration process of the container migration function model is expressed as: minT = ∑ k∈K T k ; The process of finding the maximum reward is expressed as: Among them, t is the time point when the user's location changes, k is the task created by the user at time point t, and r t For reward, T k is the total task duration, γ t is the discount factor, γ t ∈(0,1).

5. The method for active and passive migration of containers in meta-computing according to claim 4, characterized in that: Before the PPCM algorithm is used to transform the exploration process of the container migration function model into a process of finding a maximum reward, the following steps are included: Obtaining node status and task status in the container migration function model; Generate a target container migration strategy according to the node status and the task status, wherein: The node status is: Where n is a node, N is a set of nodes, |N| is the number of nodes, u is a mobile user, is the position coordinate of the edge node, is the distance between the mobile user and the node for the next task, is the download size of the task on the node, is the total size of all running tasks on the node, is the state of each layer where the node is located, f n is the CPU resource, b n is bandwidth; The task status is: Where k is the task, |L| is the number of layers in the container, and L u (t) is the location coordinate of the mobile user, f u is the resources required for the task, b u is the bandwidth required for the task, D u (t) is the download size required for the task.

6. The method for active and passive migration of containers in meta-computing according to claim 5, characterized in that: The generating a target container migration strategy according to the node status and the task status includes: The node state and the task state are input into the policy network, and the DCN network is used to extract the feature relationship within and between layers to obtain the layer shared features; Inputting the layer shared features into the value network, using a multi-head attention mechanism to perform state evaluation on the layer shared features, and obtaining a state evaluation result; A target container migration strategy is generated according to the status assessment result.

7. The method for active and passive migration of containers in meta-computing according to claim 6, characterized in that: The policy network and the value network are updated using an algorithm based on policy gradient, wherein the loss function of the policy gradient is: Among them, E is the expected function, s is the state, a is the execution action, θ is the given parameter, θ k is the parameter of the old strategy, π θ is the strategy with given parameters θ, π θ (a|s) is the probability that the new strategy executes action a in state s, is the probability of the old strategy executing action a in state s, is the advantage function, which is used to measure the performance of executing action a in state s relative to the average level. The function clip(x,y,z)=max(min(x,z),y) limits x to the interval [y,z], where ∈ represents the range of clip.

8. An active and passive migration device for containers in metacomputing, characterized in that: The device comprises: A migration duration determination module, used to determine the total task duration information and constraint conditions of container migration, wherein the container migration includes active migration and passive migration; A migration model building module, used to build a container migration function model based on the total task duration information and the constraint conditions; The migration strategy generation module is used to adopt the PPCM algorithm to explore the container migration function model with the goal of outputting the minimum total task duration, and generate a target container migration strategy. The target container migration strategy is used to reflect the container migration mode corresponding to the minimum total task duration output in the container migration function model.

9. A terminal, characterized in that: The terminal includes a memory, a processor, and an active and passive migration program for containers in meta-computing stored in the memory and executable on the processor. When the processor executes the active and passive migration program for containers in meta-computing, the steps of the active and passive migration method for containers in meta-computing as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores active and passive migration programs for containers in meta-computing. When the active and passive migration programs for containers in meta-computing are executed by the processor, the steps of the active and passive migration method for containers in meta-computing as described in any one of claims 1-7 are implemented.

Images