A mobile edge computing task offloading optimization method and device

Abstract

The present application relates to the technical field of computer science and information engineering, in particular to a mobile edge computing task offloading optimization method and device, comprising the following steps: S1, environment modeling: including graph enhanced state space construction, action space construction and setting causal demand reward function; S2, constructing neural network structure: the neural network structure comprises a policy network and a Q network; S3, algorithm training: firstly, environment parameters are configured and network parameters are initialized, secondly, environment interaction and experience storage are carried out, and finally, network updating and loop iteration are carried out; S4, after the training of S3 is completed, the final policy network is deployed to a dynamic mobile edge computing system for real-time task offloading decision, wherein the graph embedding vector is provided.

Description

Technical Field

[0001] This invention relates to the fields of computer science and information engineering technology, and specifically to a method and apparatus for optimizing the offloading of mobile edge computing tasks. Background Technology

[0002] In an era of rapid development of Internet of Things (IoT) devices and 5G / 6G communication technologies, a large number of computationally intensive and latency-sensitive applications are emerging, such as augmented reality (AR), autonomous driving, and smart healthcare. These applications have extremely demanding requirements for computing resources and response latency. However, due to the limited computing and storage capabilities of mobile terminals, traditional cloud computing models suffer from significant network transmission latency and centralized bottlenecks, making it difficult to meet the demands for real-time performance and energy consumption control. To address this, mobile edge computing (MEC) has emerged, deploying computing nodes at the network edge close to the terminal to provide users with low-latency local computing and data services.

[0003] While the MEC architecture alleviates computational load and latency issues to some extent, its operating environment is highly dynamic and uncertain: mobile terminals frequently cross the coverage areas of different edge servers, leading to frequent task migrations and connection interruptions; edge node computing and bandwidth resources change constantly; and multi-user resource sharing causes the system load to exhibit strong time-varying characteristics. These factors make task offloading and resource allocation complex and dynamic.

[0004] In recent years, researchers have attempted to solve such problems using deep reinforcement learning (DRL) algorithms. DRL learns policies by interacting with the environment and can achieve automated task unloading and resource optimization in a high-dimensional state space. However, as research has deepened, existing DRL methods have gradually exposed two deep-seated technical bottlenecks that have not yet been effectively addressed when dealing with the inherent complexity of MEC environments: (1) The lack of structured representation of the state space. Existing methods usually model the state of the MEC environment as a stack of a series of independent physical parameters (such as server resources, channel bandwidth, user coordinates, etc.). This "flat" vector representation completely ignores the inherent topological connections and complex interaction relationships between users, tasks, and edge servers. The agent cannot perceive the overall structural information of the system, making it difficult for the learned policies to understand structural dynamics such as "changes in associated servers caused by user movement" or "multiple users competing for the same resource chain", resulting in poor generalization ability and decision failure when the environment undergoes non-stationary changes. (2) The problem of confusing bias in reward signals. Existing DRL methods rely heavily on artificially designed reward functions (such as weighted summation of time delay, energy consumption, etc.). Such reward signals are confounded by environmental fluctuations (confounding variables) and the true causal effects of decision-making actions. For example, the success of a task may be primarily attributed to the user moving to an area with better channel quality, rather than the offloading decision itself being superior. This biased reward can mislead the policy into learning false statistical associations rather than true causal patterns, severely limiting the policy's performance ceiling and learning stability, causing it to fluctuate wildly in dynamic environments.

[0005] To alleviate the above problems, some studies have attempted to introduce techniques such as imitation learning, but these have not fundamentally broken through the two bottlenecks mentioned above, and have introduced new problems such as expert sample dependence and training instability. Summary of the Invention

[0006] This invention addresses the two fundamental problems of shallow state awareness and biased reward signals in task offloading decisions in mobile edge computing (MEC) environments. It proposes a task offloading optimization method for mobile edge computing, comprising the following steps:

[0007] S1 performs environmental modeling: including graph-enhanced state space construction, action space construction, and setting causal demand reward function, specifically including the following steps: (1) Graph-enhanced state space construction: constructing heterogeneous graphs ,in A set of nodes, the set of nodes including user nodes Task Node and server nodes ; The edge set includes elements representing task generation relationships. Edges represent network connectivity. Edges and representing potential processing relationships Edges; a graph attention network is used as the encoder. Environmental heterogeneity diagram Encode the graph to generate a fixed-dimensional graph embedding vector. (1) State perception of the intelligent agent; (2) Construction of action space: including resources for unloading tasks. Bandwidth used during migration Allocation and in Selection of unloading target at any time The action space remains a hybrid design of continuous and discrete elements, expressed as follows: (3) Set up a causal demand reward function: used to eliminate the contamination of the reward signal by environmental confusion variables, including defining a causal graph, performing counterfactual reward calculation, and pure reward; the key variables of the causal graph include user context. Uninstallation action and task results Its causal dependence is: and ,in This indicates that C has a direct impact on A and Y; the counterfactual reward calculation specifically involves using a pre-trained counterfactual prediction model. To estimate baseline returns, used to predict returns in a given state. and user context At that time, take any action vector The expected return; the pure reward Actual benefits Counterfactual benchmark returns The difference is expressed as: ,in, This represents the state of the system at time t. Indicates the agent's state The following actions were taken. This indicates the action of the agent in the next moment. Represents the context variables of the agent at time t;

[0008] S2 constructs a neural network structure: The neural network structure includes an online policy network. Target-Policy Network Online Q network and target Q network ,in Given a state, For action vectors;

[0009] S3 performs algorithm training by first configuring environment parameters and initializing network parameters, then interacting with the environment and storing experience, and finally updating the network and iterating in a loop.

[0010] After S4 completes training in S3, it will become the final policy network. Deployed in dynamic mobile edge computing systems for real-time task offloading decisions, where The graph embedding vector.

[0011] Preferably, the policy network in step S2 is an online policy network. and target policy network It is responsible for generating actions based on the state; its input is a graph embedding vector. The network employs a multi-branch structure to adapt to a hybrid action space; specifically, it processes graph embedding vectors through a shared fully connected layer. A shared feature layer is formed, and the corresponding resource allocation amount is output as a continuous value. Resource allocation branch Output the bandwidth allocation corresponding to continuous values. Bandwidth allocation branch And the server selection branch that outputs the selection logic value for each available server. Resource allocation for continuous actions and bandwidth allocation Add normally distributed noise that decays over time:

[0012] ,

[0013] And it is mapped to the actual range [0, ] by cropping and scaling. ] and [0, ],in Indicates the amount of resources allocated The corresponding exploration noise samples, Indicates bandwidth allocation The corresponding exploration noise samples, and Representing the amount of resources allocated and bandwidth allocation Explore the variance of noise in a normal distribution.

[0014] Preferably, the Q-network in step S2 is an online Q-network. and target Q network Used to evaluate the value of state-action pairs, its input is a graph state embedding. With action vectors The network structure is a three-layer fully connected neural network, and its learning objective is to minimize causal reward-based... The timing difference error.

[0015] Preferably, step S3, the initialization phase in algorithm training, includes the following steps:

[0016] (1) Network parameter initialization: Randomly initialize the parameters of the policy network and the Q network;

[0017] (2) Initialization of experience replay buffer: Set up a unified experience replay buffer Experience quadruples used to store the interactions between intelligent agents and their environment ,in The next state obtained after a given action is performed;

[0018] (3) Counterfactual predictor initialization: Initialize the counterfactual prediction model The parameters are determined, and historical data is prepared for its pre-training.

[0019] (4) Hyperparameter settings: Determine the learning rate, soft update rate, reward discount factor, and exploration noise parameters;

[0020] (5) Environment settings: Configure mobile edge environment parameters, which are: edge server parameters: maximum available computing resources, maximum available bandwidth resources; mobile user attribute parameters: user location, task requirements; task characteristic parameters: data volume, deadline.

[0021] Preferably, step S3, which involves environmental interaction and experience storage, includes the following steps:

[0022] (1) State acquisition: Acquire current environment information and construct graph The graph embedding vector is obtained through the GAT encoder. ;

[0023] (2) Action generation: The action is generated by embedding vectors in the graph through a policy network. Generate Actions Add exploration noise;

[0024] (3) Reward calculation: The environment returns the original reward. And use a counterfactual predictor to calculate causal rewards. ;

[0025] (4) Experience storage: storing experience Store in the experience replay buffer .

[0026] Preferably, step S3, which involves network updating and iterative iteration, includes the following steps:

[0027] (1) From the buffer Experience with random sampling batches in China;

[0028] (2) Update the Q network: Update the parameters of the Q network by minimizing the temporal difference error based on causal reward. :

[0029] in, ( ) is the expectation function. It is a discount factor;

[0030] (3) Update the policy network: Update the parameters of the online policy network by gradient ascent of the policy gradient:

[0031]

[0032] (4) Soft update of the target network:

[0033] Among the symbols This indicates a parameter update. and These are the corresponding target network parameters. It is a soft update coefficient. These are the parameters of the current network;

[0034] (5) Iterative loop: Repeat the environmental interaction and network update steps until the preset maximum number of training iterations or performance convergence is reached.

[0035] A mobile edge computing task offloading optimization apparatus, to achieve the mobile edge computing task offloading optimization method as described above, is characterized by comprising the following modules:

[0036] Environment modeling module: used to construct graph-enhanced state space, construct action space and set causal demand reward function, specifically including the following steps: (1) Graph-enhanced state space construction: construct heterogeneous graph ,in A set of nodes, the set of nodes including user nodes Task Node and server nodes ; The edge set includes elements representing task generation relationships. Edges represent network connectivity. Edges and representing potential processing relationships Edges; a graph attention network is used as the encoder. Environmental heterogeneity diagram Encode the graph to generate a fixed-dimensional graph embedding vector. (1) State perception of the intelligent agent; (2) Construction of action space: including resources for unloading tasks. Bandwidth used during migration Allocation and in Selection of unloading target at any time The action space remains a hybrid design of continuous and discrete elements, expressed as follows: (3) Set up a causal demand reward function: used to eliminate the contamination of the reward signal by environmental confusion variables, including defining a causal graph, performing counterfactual reward calculation, and pure reward; the key variables of the causal graph include user context. Uninstallation action and task results Its causal dependence is: and ,in This indicates that C has a direct impact on A and Y; the counterfactual reward calculation specifically involves using a pre-trained counterfactual prediction model. To estimate baseline returns, used to predict returns in a given state. and user context At that time, take any action vector The expected return; the pure reward Actual benefits Counterfactual benchmark returns The difference is expressed as: ,in, This represents the state of the system at time t. Indicates the agent's state The following actions were taken. This indicates the action of the agent in the next moment. Represents the context variables of the agent at time t;

[0037] Constructing a neural network structure module: The neural network structure includes an online policy network. Target-Policy Network Online Q network and target Q network ,in Given a state, For action vectors;

[0038] Algorithm training module: First, configure environment parameters and initialize network parameters; second, perform environment interaction and experience storage; and finally, perform network updates and iterative iterations.

[0039] Model Deployment and Continuous Optimization Module: After the algorithm training module completes training, the final policy network is deployed. Deployed in dynamic mobile edge computing systems for real-time task offloading decisions, where The graph embedding vector.

[0040] By adopting the above scheme, the present invention has the following advantages and beneficial effects: The present invention provides a mobile edge computing task offloading optimization method based on graph state space and causal reward mechanism. This method fundamentally innovates the traditional deep reinforcement learning framework to address the high dynamism and uncertainty of the MEC environment: by constructing a heterogeneous graph model of user-task-server and using graph neural networks to encode the environment state, the problem of missing structured representations in the state space is solved; by designing a causal demand reward function based on counterfactual reasoning, the confusion bias in the reward signal is eliminated, ensuring that the policy learns the true causal effect of the action. Attached Figure Description

[0041] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the specific embodiments of the present invention will be briefly introduced below. It should be understood that the following drawings only show some specific embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0042] Figure 1 This is a structural diagram of the strategy network of this invention.

[0043] Figure 2 This is a graph showing the trend of task successful unloading rate under different training batches for the three algorithms: G-DDPG, DDPG, and G-DDPG.

[0044] Figure 3 This is a trend chart showing the connection interruption situation that occurred during the system operation of the present invention and the three algorithms G-DDPG and DDPG.

[0045] Figure 4 This is a graph showing the trend of cumulative rewards during system operation for the three algorithms: G-DDPG, DDPG, and DDPG. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to represent selected embodiments of the invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0048] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0049] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0050] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby providing a clearer and more explicit definition of the scope of protection of the present invention.

[0051] This invention proposes an optimization method for task offloading in mobile edge computing. This method first models the task offloading and resource allocation problem in a dynamic mobile edge computing (MEC) environment as a Markov decision process (MDP), and then uses a deep reinforcement learning framework to solve it. Its core innovation lies in fundamentally improving the agent's perception depth and decision-making quality by introducing a graph-enhanced state space and a causal demand reward function.

[0052] The mobile edge computing task offloading optimization method specifically includes the following steps:

[0053] 1. Environmental Modeling

[0054] 1.1 Graph-Enhanced State Space Construction: Traditional state spaces cannot represent the complex relationships between entities in an MEC system. Therefore, this invention constructs an environment heterogeneous graph. As the basis of state. The node set Includes: user nodes Features include coordinates Movement speed, equipment capabilities; task nodes Features include data volume, computational requirements, latency constraints, and the introduction of task type semantic encoding (obtained through a lightweight embedding layer to distinguish between tasks with different semantics, such as real-time and computational tasks); server nodes Features include available computing resources Available bandwidth resources Edge set include Edge: Represents the "task generation" relationship; Edge: Represents the "network connection" relationship, characterized by channel gain and signal-to-noise ratio; Edges: Represent "potential processing" relationships, characterized by the average success rate of historical unloading to this server. State encoding: Employs a graph attention network (GAT) as the encoder. , for heterogeneous graphs Encode the graph to generate a fixed-dimensional graph embedding vector. This is the state perception of the intelligent agent. Thus, state encoding... The definition is as follows: A graph attention network (GAT) is used as the encoder. , for heterogeneous graphs Encode the graph to generate a fixed-dimensional graph embedding vector. As a state perception of intelligent agents, State is defined as a dynamic state representation that reflects information such as node state, task requirements, and resource availability in the MEC environment, and can quickly capture environmental changes.

[0055] 1.2 Action Space: Actions define the operable variables of the system in task unloading and resource allocation decisions, including the amount of resources allocated to unloaded tasks. Bandwidth allocation used during migration as well as Selection of unloading target at any time The motion space remains a hybrid design of continuous and discrete elements.

[0056] 1.3 Causal Demand-Reward Function: To eliminate the contamination of the reward signal by environmental confounding variables (such as user mobility), a causal demand-reward function is designed. Cause-effect graph definition: Key variables include user context. (e.g., location, movement mode), uninstallation action Task Results (e.g., time delay). Its causal dependence is: and .

[0057] The causal demand reward function includes the following reward calculation: (1) Counterfactual reward calculation: through a pre-trained counterfactual prediction model This is used to estimate baseline returns. The model is trained on historical data to predict returns in a given state. and user context At that time, take any action Expected returns. (2) Pure reward: the causal reward obtained by the agent. Actual benefits Counterfactual benchmark returns The difference. This reward is stripped of user context. The influence purely reflects the action The true causal effect.

[0058]

[0059] 2. Neural Network Structure

[0060] This method uses Deep Deterministic Policy Gradient (DDPG) as its basic framework and consists of four core networks: an online policy network. Target-Policy Network Online Q network Target Q network The number of neurons in all hidden layers is set to 64.

[0061] 2.1 Policy Network (Actor): See the appendix of the instruction manual for details. Figure 1 The policy network is responsible for generating actions based on the state. Its input is a graph state embedding. The network employs a multi-branch structure to adapt to the mixed action space: Shared feature layer: includes a shared fully connected layer for processing. Output the corresponding resource allocation amount for continuous values. Resource allocation branch Output the bandwidth allocation corresponding to continuous values. Bandwidth allocation branch Output the server selection branch for each available server selection logic value (e.g., probabilities generated via Softmax). To facilitate exploration, the allocation of resources for continuous motion is adjusted. and bandwidth allocation Add normally distributed noise that decays over time:

[0062] ,

[0063] And it is mapped to the actual range [0, ] by cropping and scaling. ] and [0, ].in Indicates the amount of resources allocated The corresponding exploration noise samples, Indicates bandwidth allocation The corresponding exploration noise samples, and Representing the amount of resources allocated and bandwidth allocation Explore the variance of noise in a normal distribution.

[0064] 2.2 Q-Network (Critic)

[0065] Q-networks are used to evaluate the value of state-action pairs. Their input is a graph state embedding. With action vectors The network is a three-layer fully connected neural network. Its learning objective is to minimize causal reward-based... The timing difference error.

[0066] 3. Algorithm Training Phase

[0067] 3.1 Initialization Phase

[0068] This includes initializing the following parameters: Network parameter initialization: randomly initializing the parameters of the policy network and Q network; Experience replay buffer initialization: setting up a unified experience replay buffer. Experience quadruples used to store the interactions between intelligent agents and their environment Counterfact Predictor Initialization: Initialize the counterfact prediction model The parameters are set, and historical data is prepared for its pre-training; Hyperparameter settings: determine the learning rate, soft update rate, reward discount factor, exploration noise parameters, etc.; Environment settings: configure MEC environment parameters.

[0069] 3.2 Environmental Interaction and Experience Storage

[0070] State acquisition: Obtain current environment information and construct a graph. The graph state is obtained through the GAT encoder. Action generation: The policy network generates actions based on... Generate Actions Add exploration noise; reward calculation: environment returns the original reward. And use a counterfactual predictor to calculate causal rewards. Experience storage: storing experience Store in the experience replay buffer .

[0071] 3.3 Network Updates

[0072] From the buffer A batch of experience is randomly sampled; the Q-network is updated by minimizing the temporal difference error based on causal reward.

[0073]

[0074] Update the policy network: Update the parameters of the online policy network by ascending the policy gradient.

[0075] Soft update target network:

[0076] 3.4 Repeatedly execute the environment interaction and network update steps in a loop until the preset maximum number of training iterations or performance convergence is reached.

[0077] 4. After model deployment and continuous optimization training are completed, the final policy network will be... Deployed into the MEC system for real-time task offloading decisions. The system continuously collects runtime data for periodic fine-tuning and optimization of the model.

[0078] Reference manual attached Figure 2-4To demonstrate the performance advantages of this invention in mobile edge computing (MEC) environments, the classic deep reinforcement learning algorithm DDPG and a reinforcement learning algorithm that also uses graph neural networks for state processing (Graph-enhanced DDPG, G-DDPG) were selected as baselines for comparison. Comparative experiments were conducted in dynamic scenarios involving multiple users and multiple edge servers to comprehensively evaluate the algorithm's task offloading rate, connection interruption rate, and learning stability. Experimental results show that this method can gain a deeper understanding of system dynamics and achieve better offloading decisions, significantly outperforming traditional methods in terms of convergence speed, policy performance, and system robustness.

[0079] Reference manual attached Figure 2 Under the constraints of bandwidth, resources, and connectivity, a task that successfully returns a computational result is considered a "successful unloading." The figure shows the changing trends of the task unloading success rate of our proposed method, G-DDPG, and DDPG algorithms under different training batches. As can be seen from the figure, our proposed method stabilizes after approximately 600 batches, with the number of successful task unloadings stabilizing at around 15,200, demonstrating a significantly faster convergence speed than the comparative algorithms. In contrast, the DDPG algorithm, due to its limited state awareness, struggles to adapt to complex dynamic environments and is prone to getting trapped in local optima. While the G-DDPG algorithm improves its awareness by introducing a graph structure, the confusing nature of its reward signal limits further policy optimization. Our proposed method, through the synergistic effect of graph state space and causal rewards, achieves a deep understanding of the system topology and an accurate assessment of the true value of actions, thereby significantly improving the task unloading success rate and learning efficiency.

[0080] Reference manual attached Figure 3 An interruption event is considered to occur when a task fails to meet defined constraints and repeatedly enters a waiting state during task unloading (including migration). As shown in the figure, this method achieves a minimum interruption value of approximately 480 after about 800 batches, a reduction of about 10%-15% compared to the DDPG and G-DDPG algorithms. This indicates that the graph-augmented state space used in this method can effectively perceive potential changes in network topology and predict connection risks. Simultaneously, the causal reward mechanism guides policy learning to avoid invalid migrations caused by environmental confusion factors (such as random user movement), thus significantly reducing the probability of interruptions caused by network fluctuations and blind migrations from the decision-making source. The results demonstrate that this method exhibits stronger stability and task continuity assurance capabilities under dynamic topologies.

[0081] Reference manual attached Figure 4The reward curve of the method in this invention surpasses other algorithms after approximately 400 batches, ultimately reaching its peak with an average reward value of 1280, and the fluctuations throughout the learning process are significantly less than those of the comparative algorithms. This is because DDPG has high noise in its reward signal, while G-DDPG, although having better state perception, still suffers from reward bias. The unique causal requirement reward function of this method effectively filters out environmental confusion noise, providing a stable and pure optimization direction for policy learning, resulting in a smoother training process and higher and more reliable policy performance after convergence.

[0082] The experimental results above demonstrate that the proposed method significantly outperforms traditional DDPG and the state-only G-DDPG algorithm in key metrics such as task offloading success rate, connection interruption rate, and learning stability. This fully validates the technical advantages of combining graph-augmented state space with causal reward functions: the graph state space provides the agent with deep environmental awareness, while causal rewards ensure the correctness and efficiency of the optimization direction. The final results show that this method can achieve lower interruption and higher success rate task offloading in complex MEC environments through more intelligent perception and decision-making, providing a novel and efficient solution for edge intelligent decision-making.

[0083] The present invention provides the preferred embodiments described above. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principle of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for optimizing the offloading of mobile edge computing tasks, characterized in that, The following steps are included: S1 Environmental modeling: including graph augmentation state space construction, action space construction and setting causal demand reward function, specifically including the following steps: (1) Graph augmentation state space construction: constructing heterogeneous graphs ,in A set of nodes, the set of nodes including user nodes Task Node and server nodes ; The edge set includes elements representing task generation relationships. Edges represent network connectivity. Edges and representing potential processing relationships Edges; a graph attention network is used as the encoder. , for heterogeneous graphs Encode the graph to generate a fixed-dimensional graph embedding vector. (1) State perception of the intelligent agent; (2) Construction of action space: Actions define the operable variables of the system in task unloading and resource allocation decisions, including the amount of resource allocation for unloading tasks. Bandwidth allocation used during migration as well as Selection of unloading target at any time The action space remains a hybrid design of continuous and discrete elements, expressed as follows: (3) Set up a causal demand reward function: used to eliminate the pollution of reward signals by environmental confounding variables, including defining a causal graph, performing counterfactual reward calculation and pure reward calculation; the key variables of the causal graph include user context. Uninstallation action and task results Its causal dependence is: and ,in This indicates that C has a direct impact on A and Y; the counterfactual reward calculation specifically involves using a pre-trained counterfactual prediction model. To estimate baseline returns, used to predict returns in a given state. and user context At that time, take any action vector The expected return; the pure reward Actual benefits Compared with baseline returns The difference is expressed as: ,in, This represents the state of the system at time t. Indicates the agent's state The following actions were taken. This indicates the action of the agent in the next moment. Represents the context variables of the agent at time t; S2 constructs a neural network structure: The neural network structure includes an online policy network. Target-Policy Network Online Q network and target Q network ,in Given a state, Action vectors; online policy network and target policy network It is responsible for generating actions based on the state; its input is a graph embedding vector. The network employs a multi-branch structure to adapt to a hybrid action space; specifically, it processes graph embedding vectors through a shared fully connected layer. A shared feature layer is formed, and the corresponding resource allocation amount is output as a continuous value. Resource allocation branch Output the bandwidth allocation corresponding to continuous values. Bandwidth allocation branch And the server selection branch that outputs the selection logic value for each available server. Q network refers to the online Q network. and target Q network Used to evaluate the value of state-action pairs, its input is a graph embedding vector. With action vectors The network structure is a three-layer fully connected neural network, and its learning objective is to minimize the pure reward. The timing difference error; S3 performs algorithm training by first configuring environment parameters and initializing network parameters, then interacting with the environment and storing experience, and finally updating the network and iterating in a loop. After S4 completes training in S3, it will become the final policy network. Deployed in dynamic mobile edge computing systems for real-time task offloading decisions, where The graph embedding vector.

2. The mobile edge computing task offloading optimization method according to claim 1, characterized in that, Resource allocation for continuous motion and bandwidth allocation Add normally distributed noise that decays over time: , And it is mapped to the actual range [0, ] by cropping and scaling. ] and [0, ],in Indicates the amount of resources allocated The corresponding exploration noise samples, Indicates bandwidth allocation The corresponding exploration noise samples, and Representing the amount of resources allocated and bandwidth allocation Explore the variance of noise in a normal distribution.

3. The mobile edge computing task offloading optimization method according to claim 1, characterized in that, Step S3, the initialization phase in algorithm training, includes the following steps: (1) Network parameter initialization: Randomly initialize the parameters of the policy network and the Q network; (2) Initialization of experience replay buffer: Set up a unified experience replay buffer Experience quadruples used to store the interactions between intelligent agents and their environment ,in The next state obtained after a given action is performed; (3) Counterfactual prediction model initialization: Initialize the counterfactual prediction model The parameters are determined, and historical data is prepared for its pre-training. (4) Hyperparameter settings: Determine the learning rate, soft update rate, reward discount factor, and exploration noise parameters; (5) Environment settings: Configure mobile edge environment parameters, which are: edge server parameters: maximum available computing resources, maximum available bandwidth resources; mobile user attribute parameters: user location, task requirements; task characteristic parameters: data volume, deadline.

4. The mobile edge computing task offloading optimization method according to claim 3, characterized in that, Step S3, which involves environmental interaction and experience storage, includes the following steps: (1) State acquisition: Acquire current environment information and construct heterogeneous graph. And through graph attention networks to heterogeneous graphs Encode to obtain graph embedding vectors ; (2) Action generation: The action is generated by embedding vectors in the graph through a policy network. Generate Actions Add exploration noise; (3) Reward calculation: The environment returns actual benefits. And use a counterfactual prediction model to calculate the pure reward. ; (4) Experience storage: storing experience Store in the experience replay buffer .

5. A mobile edge computing task offloading optimization apparatus to implement the mobile edge computing task offloading optimization method as described in any one of claims 1-4, characterized in that, The following modules are included: Environment Modeling Module: used to construct the graph-enhanced state space, construct the action space, and set the causal demand reward function, specifically including the following steps: (1) Graph-enhanced state space construction: constructing a heterogeneous graph ,in A set of nodes, the set of nodes including user nodes Task Node and server nodes ; The edge set includes elements representing task generation relationships. Edges represent network connectivity. Edges and representing potential processing relationships Edges; a graph attention network is used as the encoder. Environmental heterogeneity diagram Encode the graph to generate a fixed-dimensional graph embedding vector. (1) State perception of the intelligent agent; (2) Construction of action space: Actions define the operable variables of the system in task unloading and resource allocation decisions, including the amount of resource allocation for unloading tasks. Bandwidth allocation used during migration as well as Selection of unloading target at any time The action space remains a hybrid design of continuous and discrete elements, expressed as follows: (3) Set up a causal demand reward function: used to eliminate the contamination of the reward signal by environmental confusion variables, including defining a causal graph, performing counterfactual reward calculation, and pure reward; the key variables of the causal graph include user context. Uninstallation action and task results Its causal dependence is: and ,in This indicates that C has a direct impact on A and Y; the counterfactual reward calculation specifically involves using a pre-trained counterfactual prediction model. To estimate baseline returns, used to predict returns in a given state. and user context At that time, take any action vector The expected return; the pure reward Actual benefits Compared with baseline returns The difference is expressed as: ,in, This represents the state of the system at time t. Indicates the agent's state The following actions were taken. This indicates the action of the agent in the next moment. Represents the context variables of the agent at time t; Constructing a neural network structure module: The neural network structure includes an online policy network. Target-Policy Network Online Q network and target Q network ,in Given a state, Action vectors; online policy network and target policy network It is responsible for generating actions based on the state; its input is a graph embedding vector. The network employs a multi-branch structure to adapt to a hybrid action space; specifically, it processes graph embedding vectors through a shared fully connected layer. A shared feature layer is formed, and the corresponding resource allocation amount is output as a continuous value. Resource allocation branch Output the bandwidth allocation corresponding to continuous values. Bandwidth allocation branch And the server selection branch that outputs the selection logic value for each available server. Q network refers to the online Q network. and target Q network Used to evaluate the value of state-action pairs, its input is a graph embedding vector. With action vectors The network structure is a three-layer fully connected neural network, and its learning objective is to minimize the pure reward. The timing difference error; Algorithm training module: First, configure environment parameters and initialize network parameters; second, perform environment interaction and experience storage; and finally, perform network updates and iterative iterations. Model Deployment and Continuous Optimization Module: After the algorithm training module completes training, the final policy network is deployed. Deployed in dynamic mobile edge computing systems for real-time task offloading decisions, where The graph embedding vector.

Images