Task unloading and resource allocation method based on mobile edge scene

Abstract

The invention discloses a task unloading and resource allocation method based on a mobile edge scene, which comprises the following steps of: a static decision-making stage: modeling a task unloading and resource allocation problem into a mixed integer nonlinear programming model, and aiming at minimizing the weighted sum of overall delay, overall energy consumption and individual delay; in the dynamic decision-making stage, a hierarchical learning framework is adopted, task unloading and resource allocation problems are decomposed, collaborative optimization is carried out through an upper-layer single agent and a lower-layer multi-agent, the upper-layer single agent is used for processing cloud data migration and resource allocation problems, and the lower-layer multi-agent is used for processing cloud data migration and resource allocation problems. And the lower-layer multi-agent is used for performing task unloading and base station selection on multiple users. The task unloading and resource allocation method based on hierarchical reinforcement learning is designed, the high-dimensional complex problem is more efficiently solved, and the task unloading and resource allocation optimization method is suitable for the task unloading and resource allocation optimization problem in a large-scale network.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of mobile edge computing, and in particular to a task offloading and resource allocation method based on a mobile edge scenario. BACKGROUND

[0002] In recent years, with the popularity of intelligent devices and the dramatic increase in data volume, traditional centralized cloud computing architecture faces great challenges in modern communication networks. In order to solve the problems of delay, bandwidth and computing resource bottleneck, mobile edge computing (MEC) has gradually become an important solution. MEC pushes computing and storage resources to the network edge, so that data can be processed closer to the user, thereby effectively reducing latency and reducing bandwidth occupancy. However, with the diversification of user demand and the continuous expansion of system size, how to efficiently schedule and allocate edge computing resources has become a key problem. Especially in a multi-user environment, how to dynamically allocate resources according to the user's location, computing resource demand, channel quality and other factors, and reasonably optimize system performance, is still a difficult problem.

[0003] At present, most researches focus on the migration and resource allocation of single computing tasks, but in actual mobile edge computing networks, a large number of computing tasks that need to be migrated are often deployed, and resource competition may occur between these tasks. Limited computing resources and bandwidth resources mean that the migration of a certain task may cause other tasks to be unable to obtain sufficient resources, thereby affecting the quality of service of other users. In addition, in the network environment, task migration not only involves the allocation of computing resources, but also needs to consider communication delay, energy consumption and load balancing. If the performance of a single task is taken as the optimization target, it may lead to the decline of the performance of other tasks, and even cause the non-uniform allocation of network resources. Under this background, how to reasonably coordinate the migration needs of multiple tasks through intelligent scheduling and dynamic resource management strategies, ensure the quality of user experience, and improve the overall efficiency of the system, has become a problem to be solved in mobile edge computing. SUMMARY

[0004] To solve the technical problems existing in the prior art, the present application provides a task offloading and resource allocation method based on a mobile edge scenario, which provides a task offloading and resource allocation strategy for multiple users, decides the offloading strategy, base station selection, migration strategy and resource allocation of the cloud for each user, so that the resource allocation result meets the overall resource demand, and the lowest overall delay, energy consumption and individual delay are achieved.

[0005] To achieve the above purpose, the present application provides a task offloading and resource allocation method based on a mobile edge scenario, comprising: Static decision-making phase: The task offloading and resource allocation problem is modeled as a mixed integer nonlinear programming model, with the goal of minimizing the weighted sum of overall delay, overall energy consumption and individual delay; Dynamic decision-making stage: A hierarchical learning framework is adopted to decompose the task offloading and resource allocation problems, and optimize them collaboratively by an upper-layer single agent and a lower-layer multi-agent. The upper-layer single agent is used to handle cloud data migration and resource allocation problems, while the lower-layer multi-agent is used to offload tasks for multiple users and select base stations.

[0006] Preferably, the mixed-integer nonlinear programming model includes an optimization objective function and several constraints. The optimization objective function is used to minimize overall latency, energy consumption, and individual latency. The constraints include computing resource allocation constraints, user equipment transmit power constraints, cloud base station migration power constraints, individual latency and energy consumption range constraints, task offloading selection constraints, and task migration selection constraints.

[0007] Preferably, the optimization objective function is: ; In the formula, This represents the number of users and the reciprocal of the total time. For the total time, For the number of users, Delayed from the current time, The energy consumption at the current moment. At a certain moment, Indexing for a specific user, , , , All are coefficients.

[0008] Preferably, the upper-layer single agent is a cloud-based Actor-Critic structure, responsible for base station migration decisions and resource allocation; The lower-level multi-agent structure is a multi-actor-critic structure used for multi-user task offloading decisions, transmit power control, and base station selection.

[0009] Preferably, the lower-layer multi-agent uses the MAPPO algorithm combined with LSTM and MAT neural networks to form an LMAT neural network architecture, which performs state feature extraction and action decision-making, and outputs an action vector containing discrete and continuous variables.

[0010] Preferably, the processing procedure of the LMAT neural network architecture includes: The temporal state information of multiple users is input into the LSTM network to extract temporal features and obtain the state temporal dependency vector. The state-time dependency vector is embedded and positional encoding is added to form an initial embedding matrix; The initial embedding matrix is ​​processed using a self-attention mechanism to model the competition and cooperation relationships between multi-user tasks; The resource and location features of edge servers and local devices are encoded into key-value pairs, and the features of user task agents are used as queries, interacting through a cross-attention mechanism; The output of the cross-attention mechanism is processed by a feedforward neural network, residual connections are made, and layer normalization is performed to finally generate the action probability distribution of each user agent.

[0011] Preferably, the state of the upper-layer single agent is: ,in, Represented by the total number of agents currently in operation. This represents the computing resources of all current edge servers. This represents the current location of all users. This represents all actions passed from the lower layer to the upper layer. This indicates the location of the relocated base station; The state of the lower-level multi-agent is: ,in, This represents the current amount of data for the agent. This represents the current local computing resources. This indicates the current user's location. This represents the channel quality transmitted to the base station. This represents the location of the base station within the user's range. Transmit power to a user.

[0012] Preferably, the action space of any agent in the lower-level multi-agent system is: ,in, This is represented as the uninstallation policy. This is expressed as the output RF power. This is represented as base station selection; the cloud action space of the upper-layer single agent is... , dimension L ,in , This is represented as a migration strategy. This is expressed as the resource allocation ratio.

[0013] Preferably, the total reward formula for the lower-level multi-agent is: ; In the formula, , All are weighted averages. For individual reward function values, For individual delay, Individual energy consumption.

[0014] Preferably, the total reward formula for the upper-layer single agent is: ; In the formula, R The value of the global reward function. , All are coefficients. For a user's migration distance, Set a value for the distance to the base station. For migration decisions, To migrate the maximum distance, To minimize the migration distance, , All are weighted averages. M Total number of users For internal reward function, i Index for a specific user.

[0015] Compared with the prior art, the present invention has the following advantages and technical effects: (1) In the static scenario, the present invention designs a mixed integer nonlinear programming model for the problem of mobile edge computing task offloading and resource allocation, and performs joint optimization considering individual latency, overall latency and overall energy consumption; (2) In dynamic scenarios, this invention designs a mobile edge computing task offloading and resource allocation method based on hierarchical reinforcement learning. The task is decomposed and the reliability and practicality of mobile edge computing task offloading and resource allocation decision-making in dynamic scenarios are enhanced by a two-layer intelligent agent sequential decision-making method. Attached Figure Description

[0016] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of a task unloading and resource allocation method based on a mobile edge scenario according to an embodiment of the present invention; Figure 2 This is a diagram of the MAT network structure according to an embodiment of the present invention; Figure 3 This is a schematic diagram of a hierarchical reinforcement learning structure according to an embodiment of the present invention; Figure 4 This is an example diagram of mobile edge computing task unloading and resource allocation in an embodiment of the present invention. Detailed Implementation

[0017] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0018] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0019] This embodiment proposes a task offloading and resource allocation method based on mobile edge scenarios, including: Static decision-making phase: The task offloading and resource allocation problem is modeled as a mixed integer nonlinear programming model, with the goal of minimizing the weighted sum of overall delay, overall energy consumption and individual delay; Dynamic decision-making stage: A hierarchical learning framework is adopted to decompose the task offloading and resource allocation problems, and optimize them collaboratively by an upper-layer single agent and a lower-layer multi-agent. The upper-layer single agent is used to handle cloud data migration and resource allocation problems, while the lower-layer multi-agent is used to offload tasks for multiple users and select base stations.

[0020] Furthermore, the mixed-integer nonlinear programming model includes an optimization objective function and several constraints. The optimization objective function is used to minimize the overall delay, energy consumption, and individual delay. The constraints include computing resource allocation constraints, user equipment transmit power constraints, cloud base station migration power constraints, individual delay and energy consumption range constraints, task offloading selection constraints, and task migration selection constraints.

[0021] Specifically, considering the constraints of minimum overall latency, energy consumption, and individual latency in mobile edge computing problems, this embodiment designs a method that reduces overall latency, energy consumption, and individual latency: The optimization objective function is: ; In the formula, This represents the number of users and the reciprocal of the total time. For the total time, For the number of users, Delayed from the current time, The energy consumption at the current moment. At a certain moment, Indexing for a specific user, , , , All are coefficients; The objective function consists of three parts. The first two parts represent the overall delay and energy consumption, respectively. This represents the overall average delay; in Part Two The coefficient represents the overall average energy consumption. , The first part quantifies the linear sum of the two; the second part represents the average individual latency for each user at each time step; the coefficients... , To quantify the linear sum of the two.

[0022] Computational resource allocation constraints: These constraints limit the allocated computing resources from exceeding the computing power of the allocated edge devices, and are expressed as follows: ; in, Indicates the current edge server c Maximum available computing resources Indicated as common m Individual users on the edge server c The sum of the above calculation resources, Indicates edge server, This refers to the set of all edge servers.

[0023] User equipment transmit power constraint: The power used by user equipment cannot exceed the defined power level range, expressed as: ; In the formula, For users m For base stations k The transmission power, Indicates user For signal base stations Maximum transmission power, k Indicates the current base station, m Indicates user, This is the set of all signal base stations.

[0024] Cloud-based base station migration power constraints: The horizontal range of power control for base station migration in the cloud is expressed as: ; In the formula, For edge servers c Migrate to edge server n The power consumed Represents edge server c Migrate to edge server n The maximum power consumed n This indicates the edge server to be migrated.

[0025] Individual latency and energy consumption range constraints: The final individual latency and energy consumption cannot exceed the defined range, expressed as: ; ; in, This represents the maximum range of custom individual latency. The energy consumption generated at time t must not exceed the defined range.

[0026] Task Unloading Selection Constraints: The selection of task unloading is represented as follows: ; In the formula, For users m Offload to base station k Decision-making; The meaning of this formula is that each user can arbitrarily choose the task to be unloaded within the range of zero to one. k Indicates the current base station, K This represents the set of all signal base stations.

[0027] The task migration selection constraint is expressed as: ; The formula is expressed as: edge server Migration task to another edge server n The resource ratio is within the range of 0-1. n This indicates the edge server to be migrated.

[0028] Furthermore, since integer linear programming problems are not suitable for highly dynamic network environments, this embodiment decomposes the complex task of mobile edge computing task offloading and resource allocation into two sub-tasks in dynamic scenarios, introduces a hierarchical structure, and designs a task offloading and resource allocation method based on multi-agent hierarchical reinforcement learning to solve high-dimensional complex problems more efficiently.

[0029] Furthermore, the upper-layer single agent is a cloud-based Actor-Critic structure, responsible for base station migration decisions and resource allocation; The lower-level multi-agent structure is a multi-actor-critic structure used for multi-user task offloading decisions, transmit power control, and base station selection.

[0030] Specifically, a two-layer framework is established based on hierarchical reinforcement learning. The lower layer is a multi-Actor-Critic framework for multiple users, and the upper layer is a cloud-based Actor-Critic framework. The complex problem of task offloading and resource allocation is decomposed into two sub-tasks. A multi-agent reinforcement learning layer and a single-agent reinforcement learning layer respectively decide the offloading strategy and migration path and resource allocation for multiple users, jointly optimizing the minimum overall latency, energy consumption and individual latency.

[0031] A two-layer Actor-Critic deep hierarchical reinforcement learning algorithm is established based on Markov decision-making. It consists of an upper-layer single agent and a lower-layer multi-agent. The upper-layer single agent handles cloud data migration and resource allocation problems, while the lower layer uses multi-agent to offload tasks for multiple users and select base stations. The hierarchical reinforcement learning utilizes the sequential decision-making characteristics in the Markov decision-making process to simulate stochastic policies and rewards in environments where the system state has Markov properties.

[0032] Furthermore, the lower-level multi-agent uses the MAPPO algorithm combined with LSTM and MAT neural networks to form an LMAT neural network architecture, which performs state feature extraction and action decision-making, and outputs an action vector containing discrete and continuous variables.

[0033] Specifically, the lower-level MAPPO algorithm combines LSTM and MAT (Multi-Agent Transformer) neural networks and optimizes them to form the LMAT (LSTM-MAT) architecture. This not only improves the model's ability to capture temporal information and model global dependencies, but also effectively supports multi-agent collaborative decision-making, while outputting actions that include both discrete and continuous variables. The LMAT neural network architecture is used to process the information features of multi-user task offloading. Multi-user task offloading and base station information features are input into the network, and a multi-head attention mechanism is used to process the multi-user task offloading and base station information features from different dimensional spaces, effectively capturing the contextual information of multi-user behavior.

[0034] After feature extraction via the LMAT neural network, the probability distribution matrix for candidate actions is obtained through two fully connected layers. The vectors output by the lower-level multi-agent are their own offloading strategy, output power, and base station selection, respectively. After the lower-layer multi-agent performs an action, the current state of the mobile edge scene changes. The upper layer learns the state information of the lower layer and inputs it into two fully connected layers to obtain the probability distribution matrix of the candidate actions. The output actions are the migration strategy and resource allocation ratio. Finally, the upper-layer single agent receives the upper-layer reward given by the environment. At the same time, the upper-layer action and the current network state are passed to the lower-layer agent. The lower-layer agent repeats the above steps, executes the lower-layer action, and changes the current state. Calculate the loss function for each layer of the Actor-Critic network and update the corresponding network parameters; Calculate the corresponding cumulative reward, and stop iterating until the model converges or the number of iterations is reached.

[0035] Furthermore, the processing procedure of the LMAT neural network architecture includes: The temporal state information of multiple users is input into the LSTM network to extract temporal features and obtain the state temporal dependency vector. The state-time dependency vector is embedded and positional encoding is added to form an initial embedding matrix; The initial embedding matrix is ​​processed using a self-attention mechanism to model the competition and cooperation relationships between multi-user tasks; The resource and location features of edge servers and local devices are encoded into key-value pairs, and the features of user task agents are used as queries, interacting through a cross-attention mechanism; The output of the cross-attention mechanism is processed by a feedforward neural network, residual connections are made, and layer normalization is performed to finally generate the action probability distribution of each user agent.

[0036] Specifically, in order to capture the global state information features of multiple users and ensure the efficiency of multi-agent reinforcement learning algorithms, this embodiment inputs all the states of multiple users into the encoder network and uses a distributed multi-head attention mechanism to process the feature information of multi-user task offloading from different time dimensions.

[0037] First, the state information of the lower-level multi-users is embedded as input and then positional encoding is added, resulting in: ; In the formula, Let this be the initial embedding vector for a user. Let the original state feature vector of a user be... For a user's location characteristics, For a user's location features, i Indexing for a specific user, PE ( ) represents the position encoding function; Subsequent interactions between tasks are implemented, and all task agents are embedded to form a matrix: ; In the formula, For a certain user i Initial embedding matrix, The initial embedding matrix for the Nth user; Entering the self-attention layer: ; In the formula, Q For querying the matrix, K The key matrix, V For value matrices, For the initial embedding matrix of all users, To query the weight matrix, The key weight matrix is... Value weight matrix; Attention calculation: ; In the formula, Let be the transpose of the key matrix. The dimension of the key vector; This means that tasks observe each other to identify competitive and cooperative relationships; Subsequently, all edge servers and local devices were encoded into embedding vectors. It is generated based on resource characteristics and location information, with the task agent acting as the query and the server acting as the key / value. Finally, each attention output channel is passed through a feedforward neural network, and then residual connections and layer normalization are performed to finally output the action probability distribution. ; In the formula, For probability, For the final feature vector, This is the output layer weight matrix. b This is the bias term for the output layer.

[0038] Furthermore, this embodiment transforms the task offloading and resource allocation problem of mobile edge computing into a two-layer agent sequential decision-making problem. The two agents decompose the task offloading and resource allocation problem into two different sub-tasks, and transform the sub-tasks into an MDP model, defining the state transition as... ,in, Indicates the current state. Indicates the current action. This represents the reward given by the environment after an action is performed. This indicates the next state after the current action is performed.

[0039] Since this invention relates to a hierarchical reinforcement learning framework, the states corresponding to the upper and lower layer agents are different, specifically: The state of the upper-level single agent is ,in, Represented by the total number of agents currently in operation. This represents the computing resources of all current edge servers. This represents the current location of all users. This represents all actions passed from the lower layer to the upper layer. This indicates the location of the relocated base station; The state of the lower-level multi-agent is ,in, This represents the current amount of data for the agent. This represents the current local computing resources. This indicates the current location of the user. This represents the channel quality transmitted to the base station. This indicates the location of the base station within the user's range. Output power for a certain user.

[0040] Furthermore, since this embodiment decomposes the mobile edge computing task offloading and resource allocation problem into two sub-tasks, the action definitions and action spaces of the upper-layer and lower-layer intelligent agents are different. The action space of any intelligent agent in the lower-layer multi-agent system is... ,in, This is represented as the uninstallation policy. This is expressed as the output RF power. This is represented as base station selection; the cloud action space of the upper-layer single agent is... , dimension L ,in, , This is represented as a migration strategy. This is expressed as the resource allocation ratio.

[0041] Furthermore, this embodiment designs the reward function from the perspectives of overall latency, energy consumption, and individual latency, specifically as follows: The formula for the total reward of the lower-level multi-agent is: ; In the formula, , All are weighted averages. For a user's reward function, For a user's delay function, Let this be the energy consumption function for a certain user. in, This represents latency for lower-level individual users. This represents the energy consumption sent by individual users.

[0042] The formula for the total reward of the upper-level single agent is: ; In the formula, R The value of the global reward function. , All are coefficients. For a user's migration distance, Set a value for the distance to the base station. For migration decisions, To migrate the maximum distance, To minimize the migration distance, , All are weighted averages. M Total number of users For internal reward function, i Index for a specific user; in, Indicated as when the first i The migration distance of a single user is greater than the threshold. Furthermore, when the migration decision (m) is 1 (indicating migration), the indicator function value is 1, indicating that this is an "unnecessary migration," and a penalty is applied. This refers to a situation where the load is unbalanced when migrating to other base stations, and the goal is to avoid migrating to a single base station. This represents the average latency and average energy consumption obtained after the migration.

[0043] In this embodiment, the lower layer uses an LMAT neural network plus MAPPO for action selection, while the upper layer uses a Hybrid SAC reinforcement learning algorithm with an Actor-Critic fully connected layer to ensure that the upper layer can output offline and continuous actions.

[0044] To more clearly illustrate the technical solution of the present invention, specific embodiments are provided below for description: like Figure 1 As shown, the neural network parameters and the states of the upper and lower layer agents are initialized. The lower layer agent state is input into the LMAT neural network to obtain actions. It is then checked whether multi-user actions are offloaded to the edge server. If they are offloaded, the offload latency is calculated; otherwise, the latency is calculated locally to obtain the internal reward. The internal reward and action state are input into the upper layer agent to obtain the upper layer environment state, internal reward function, and lower layer actions. These are then input into the neural network to obtain the actions and rewards, resulting in the final latency, energy consumption, and individual latency. The environment states of the upper and lower layers are updated, and the network parameters are updated. This process is repeated until convergence.

[0045] like Figure 2 , Figure 3 As shown, a task offloading and resource allocation method based on a hierarchical reinforcement learning model framework is presented, including a recurrent neural network for feature extraction and a MAT multi-head attention mechanism. The interaction between the two-layer agent and the environment in the hierarchical reinforcement learning model is illustrated, and the lower layer uses the MAPPO algorithm and the upper layer uses the Hybrid SAC algorithm to train their respective neural networks.

[0046] At time t, the lower-layer agent inputs its state into the LAMT network structure, samples a lower-layer action from the action probability matrix, and interacts with the environment to obtain the next state. and rewards Then, the upper-level agent selects an action based on the lower-level actions and the current state, and interacts with the internal system to obtain the next state. and rewards Internally driven and responding to the environment, the two-layered agent updates the corresponding policy network and value network using the MAPPO algorithm and Hybrid SAC algorithm, respectively. When an empirical... First, the time-series difference error is calculated using a value network. : ; in, As a discount factor, For the observed reward, For the estimation of the state value at time t+1 in a value network, This is an estimate of the state value at time t for a value network.

[0047] Then the advantage function is calculated recursively, which represents the state... Next, action The advantage function, relative to the mean, is calculated using the GAE: ; in, It is a hyperparameter of GAE used to control the trade-off between bias and variance in the estimation; For the advantage estimation function, For the first t Timing differential error, For the first t +1 timing difference error, This is the decay weight.

[0048] MAPPO uses PPO-Clip to directly clip the ratio of the old and new strategies to a certain extent, thus constraining the magnitude of change. Its objective function is calculated as follows: ; In the formula, Let be the objective function. Based on experience and expectations, The ratio of the probabilities of the new and old strategies. This is the cutting factor; in, The probability ratio is calculated to control the new strategy and its magnitude. This represents the left-hand side of the formula for the objective function of pruning in near-end strategy optimization; This indicates the item on the right.

[0049] The policy network parameters are updated by calculating the loss of this policy update, and the value network loss is calculated based on the difference between the estimated value and the target value to update the parameters.

[0050] like Figure 4 The diagram shown primarily illustrates the task unloading and migration of mobile edge computing tasks and resource allocation. It can be seen that by treating the cloud as the upper layer and mobile edge computing as the lower layer, when a mobile user approaches the edge range of the edge server, the upper layer will migrate its resources through instructions, ultimately achieving the effects of low latency and low energy consumption.

[0051] To verify the effectiveness of this technical solution in addressing task offloading and resource allocation in user mobility scenarios, we compared it with current mainstream methods, including the MADDPG, COMA, and MATD3 algorithms. In the experiments, all algorithms used the same environmental settings and consistent experimental parameters. To evaluate the performance of this technical solution, average energy consumption, average latency, and individual latency were selected as evaluation metrics for the comparative experiments.

[0052] Table 1 quantifies the performance comparison of each method. Our method outperforms the compared methods in terms of migration cost, migration latency, and end-to-end latency of the function chain. Therefore, our method performs best in terms of overall model performance.

[0053] Table 1 The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for task offloading and resource allocation based on mobile edge scenario, characterized in that, The application relates to a method for task offloading and resource allocation in a mobile edge computing system. The method comprises: A static decision stage: modeling the task offloading and resource allocation problem as a mixed integer nonlinear programming model, and taking the weighted sum of the overall delay, overall energy consumption and individual delay as the target for minimization; 2.The method of task offloading and resource allocation based on mobile edge scenario according to claim 1, characterized in that, A dynamic decision stage: adopting a hierarchical learning framework to decompose the task offloading and resource allocation problem, and respectively optimizing the problem by a single upper agent and multiple lower agents, wherein the single upper agent is used for processing cloud data migration and resource allocation, and the multiple lower agents are used for task offloading and base station selection of multiple users. 3.The method of task offloading and resource allocation based on mobile edge scenario according to claim 2, characterized in that, The mixed integer nonlinear programming model comprises an optimization objective function and a plurality of constraint conditions, the optimization objective function is used for minimizing the overall delay, energy consumption and individual delay, and the constraint conditions comprise a computing resource allocation constraint, a user equipment transmission power constraint, a cloud base station migration power constraint, an individual delay and energy consumption range constraint, a task offloading selection constraint and a task migration selection constraint. ； wherein denotes the number of users and the inverse of the total time sum, is the time sum, is the number of users, is the delay at the current time instant, is the energy consumption at the current time instant, is a time instant, is a user index, , , , are coefficients. 4.The method of claim 1, wherein, The optimization objective function is as follows: The single upper agent is a cloud Actor-Critic structure, and is responsible for base station migration decision and resource allocation; 5.The method of task offloading and resource allocation based on mobile edge scenario according to claim 4, characterized in that, The multiple lower agents are multi-Actor-Critic structures, and are used for task offloading decision, transmission power control and base station selection of multiple users.

6. The method of task offloading and resource allocation based on mobile edge scenario according to claim 5, characterized in that, The multiple lower agents adopt a MAPPO algorithm to combine an LSTM and a MAT neural network to form an LMAT neural network architecture, perform state feature extraction and action decision, and output an action vector comprising discrete and continuous variables. The processing process of the LMAT neural network architecture comprises: Inputting time sequence state information of multiple users into an LSTM network to perform time sequence feature extraction and obtain a state time sequence dependence vector; Embedding the state time sequence dependence vector and adding position encoding to form an initial embedding matrix; Processing the initial embedding matrix through a self-attention mechanism to model the competition and cooperation relationship among multiple user tasks; Encoding the resource and position features of an edge server and a local device into a key-value pair, taking the features of a user task agent as a query, and interacting through a cross-attention mechanism; 7. The method of task offloading and resource allocation based on mobile edge scenario according to claim 1, characterized in that, The state of the upper layer single agent is wherein, represents the current number of all agents, represents the current computing resources of all edge servers, represents the current positions of all users, represents all actions of the lower layer delivered to the upper layer, represents the position of the migration base station; The state of the lower layer multi-agent is wherein, represents the data volume of the current agent, represents the current local computing resource, represents the current user's location, represents the channel quality transmitted to the base station, represents the location of the base station within the user range, is the transmission power of a certain user. 8.The method of task offloading and resource allocation based on mobile edge scenario according to claim 1, characterized in that, The action space of any agent in the lower layer multi-agent is wherein, is represented as an offloading strategy, is represented as an output radio frequency power, is represented as a base station selection; the cloud action space of the upper layer single agent is , and the dimension is L wherein , is represented as a migration strategy, is represented as a resource allocation ratio. 9.The method of task offloading and resource allocation based on mobile edge scenario according to claim 8, characterized in that, Performing feedforward neural network processing, residual connection and layer normalization on the output of the cross-attention mechanism to finally generate an action probability distribution of each user agent. ； wherein , are weighted averages, is the individual reward function value, is the individual delay, is the individual energy consumption. 10.The method of task offloading and resource allocation based on mobile edge scenario according to claim 9, characterized in that, The total reward formula of the multiple lower agents is as follows: The total reward formula of the single upper agent is as follows: ； wherein, R is a global reward function value, , are coefficients, is a migration distance of a certain user, is a base station distance constant value, is a migration decision, is a maximum migration distance, is a minimum migration distance, , are weighted average sums, M is a total number of users, is an internal reward function, i is a certain user index.

Images