A multi-agent collaborative task offloading method for a domain security mobile edge computing scene

Abstract

A multi-agent collaborative task offloading method for domain security mobile edge computing scenarios belongs to the technical fields of mobile edge computing, multi-agent reinforcement learning and privacy security. The method comprises a centralized training process, including obtaining a system state and performing characterization compression calculation collaborative auxiliary reward; calculating tail risk penalty; updating tail risk constraint multiplier; training policy and value network to obtain a trained offloading model. The process of decentralized execution comprises obtaining local observation and decision; executing safe offloading. The application explicitly controls the cost upper limit of the worst case, effectively improves the stability of service quality; through the information bottleneck and mutual information mechanism, the collaborative efficiency is enhanced, and the problem that the traditional multi-agent reinforcement learning is easy to fall into local optimum in the complex edge environment is solved; at the same time, a lightweight security protocol is adopted to guarantee the privacy of cross-domain data, and the security and computing efficiency are considered, which is suitable for intelligent task offloading optimization in heterogeneous edge computing environment.

Description

Technical Field

[0001] This invention belongs to the fields of mobile edge computing, multi-agent reinforcement learning and privacy security technology. It relates to a collaborative computing offloading method that combines variational information bottleneck and mutual information collaboration mechanism and introduces tail risk soft constraints in cross-domain offloading scenarios. In particular, it relates to a multi-agent collaborative task offloading method for domain-secure mobile edge computing scenarios. Background Technology

[0002] With the explosive growth of the Internet of Things (IoT) and mobile smart terminals, an increasing number of computationally intensive applications (such as augmented reality, wearable cognitive assistance, and in-vehicle real-time monitoring) are urgently demanding low-latency and energy-sensitive edge computing capabilities. Mobile edge computing, by providing services closer to users and offloading some tasks from user devices to nearby edge servers, can significantly alleviate the computational pressure on user devices and cloud centers. However, the resulting disparities in computing resources and the security of cross-domain data transmission also bring new challenges. In actual deployments, the computing power of edge servers varies, easily leading to load and resource mismatches; at the same time, the risk of user privacy leakage has become a constraint that cannot be ignored. This makes a task offloading strategy that balances efficiency and security a necessary issue. Existing mobile edge computing task offloading solutions face three major challenges: 1. Security and privacy risks: When task data is transmitted in open edge networks, especially across different trust domains (e.g., from private networks to public edge servers), there are serious data leakage and privacy risks. 2. Low collaborative efficiency: In complex scenarios with multiple users and multiple servers, multi-agent reinforcement learning is often used for distributed decision-making. However, existing multi-agent reinforcement learning methods often suffer from local optima due to the presence of a large amount of interfering information ("spurious associations") in the original system state and the lack of effective cooperation mechanisms, resulting in high overall system costs. 3. Tail Risk: Most existing offloading strategies aim to optimize the system's "average performance" (e.g., minimizing average latency or average energy consumption). However, in many latency-sensitive real-time applications, "good average performance" is not sufficient. If the system occasionally experiences extremely high latency or task failures, even if the average user experience is acceptable, such unstable quality of service is unacceptable.

[0003] Therefore, how to effectively suppress performance fluctuations in the worst-case scenario while ensuring secure collaboration and thus guaranteeing the stability of service quality is a technical challenge that urgently needs to be solved in this field. Summary of the Invention

[0004] The technical problem to be solved by this invention is to propose a novel computational offloading method that, while ensuring cross-domain transmission security, utilizes multi-agent collaborative optimization of the system's average cost and explicitly constrains tail risk to ensure that the system cost at the high percentile remains within an acceptable threshold.

[0005] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0006] A multi-agent collaborative task offloading method for domain-secure mobile edge computing scenarios includes a centralized training process and a decentralized execution process, specifically comprising the following steps:

[0007] Centralized training process:

[0008] S1. Obtain the global system state, compress the global system state using a variational information bottleneck encoder, and generate a task representation that contains only information related to the offloading decision.

[0009] S2. Based on the task representation and the joint actions of all agents, calculate the confidence mutual information and use it as an auxiliary reward signal;

[0010] S3. Collect a set of actual system costs for task execution from the training batch, calculate the high percentile statistic of the actual system costs for this set, set a cost threshold, and calculate the tail risk penalty item accordingly.

[0011] S4. Introduce an adaptive Lagrange multiplier and automatically update the multiplier based on the tail risk penalty term;

[0012] S5. Training Policy and Value Network: A composite objective function is used to update the policy network and value network in multi-agent reinforcement learning. This composite objective function includes minimizing the average system cost, maximizing the confidence mutual trust auxiliary reward, and minimizing the tail risk penalty term scaled by adaptive Lagrange multipliers, ultimately obtaining the trained offloading policy model.

[0013] Decentralized execution process:

[0014] S6. The agent loads the trained unloading policy model and obtains the local state required for decision-making based solely on local observations; the local state is input into the policy model to generate the unloading decision action.

[0015] S7. Determine whether the decision action triggers cross-domain transmission. If so, activate the domain security rules, use a lightweight key exchange protocol and a symmetric encryption algorithm to encrypt the unloading data, and execute the unloading task.

[0016] Furthermore, the task representation is obtained through the following process:

[0017] S11, Set global system state Input encoder network;

[0018] S12, Average output value of encoder network Sum of logarithmic variance The logarithmic variance is obtained through Calculations to control the range, This represents the hidden features extracted by the encoder. and These represent the weight matrix and the bias vector, respectively.

[0019] S13, Based on mean Sum of logarithmic variance The task representation z is generated through reparameterization techniques.

[0020] S14, By minimizing the variational information bottleneck loss To train encoder network parameters .

[0021] Furthermore, the confidence mutual information is obtained through the following steps:

[0022] S21. Set the positive sample buffer. and negative sample buffer The cumulative reward or cumulative cost of a task is used as a quality evaluation parameter. A threshold is set based on the quality evaluation parameter to distinguish between high-quality and low-quality experiences. High-quality experiences are stored in a positive sample buffer. Low-quality experience is stored in the negative sample buffer. ;

[0023] S22. Use a mutual information neural estimator to estimate the positive sample buffer. Medium task representation and Mutual information lower bound And train the mutual information neural estimator by minimizing its loss; where, This represents a joint action consisting of the actions of all agents.

[0024] S23. Use the contrast log-ratio upper bound estimator to estimate the negative sample buffer. middle and Upper bound of mutual information And train the comparison log ratio upper bound estimator by minimizing its loss;

[0025] S24. Combine the two estimators to obtain the final confidence mutual information-assisted reward. ,in and To balance the hyperparameters.

[0026] Furthermore, the tail risk penalty item is calculated as follows:

[0027] S31. Based on the unloading decision The latency and energy consumption of local execution, edge unloading, and cloud unloading were calculated respectively; among them , These represent the task number and the user device number, respectively.

[0028] S32. Calculate security costs This means penalizing cross-domain data transfers. Indicates the time slot number;

[0029] S33. Weighted summation yields the total cost: ,in and These represent the actual execution latency and energy consumption corresponding to the current unloading decision, respectively. , and These represent the non-negative weighting coefficients for latency, energy consumption, and security costs, respectively. ;

[0030] S34. Collect a set of total costs calculated by S33 from the training batch. Calculate the 95th percentile of the cost for this group. and set a cost threshold. Based on this, the tail risk penalty item is calculated. :

[0031] .

[0032] Furthermore, the adaptive Lagrange multiplier is specifically as follows:

[0033] In the k-th iteration, based on the tail risk penalty term Update the adaptive Lagrange multipliers using gradient ascent:

[0034]

[0035] in for Update learning rate, Let be the adaptive Lagrange multiplier for the k-th iteration.

[0036] Furthermore, the composite objective function specifically includes: the multi-agent reinforcement learning basic loss. The training objectives of the value network include: confidence mutual information and auxiliary rewards. Variational information bottleneck loss Loss of mutual information neural estimator ; Compare the loss of the logarithmic ratio upper bound estimator Tail risk loss ;in This step is treated as a constant and is only used to calculate the loss; its update is performed independently in S4.

[0037] By minimizing the total loss To train all network parameters, where This is the balance coefficient for the variational information bottleneck loss.

[0038] Furthermore, the variational information bottleneck loss is specifically the KL divergence loss, and the loss of the mutual information neural estimator is specifically the lower bound of the mutual information. The opposite of, i.e. The loss of the log-to-upper bound estimator is specifically the negative log-likelihood loss.

[0039] The beneficial effects of this invention are:

[0040] 1) By introducing tail risk soft constraints and adaptive multipliers This invention not only optimizes average performance but also explicitly penalizes the "worst-case scenario" (such as the cost of the 95th percentile), significantly reducing the probability of extreme high latency and improving the stability of service quality.

[0041] 2) The variational information bottleneck encoder filters out state noise, and the confidence mutual information-assisted reward and positive / negative sample buffer promote more accurate cooperation (rather than over-coupling) between agents, solving the problem of unstable training of multi-agent reinforcement learning in complex mobile edge computing environments.

[0042] 3) Through lightweight ECDH+AES encryption mechanism and intra-domain execution rules, the privacy and confidentiality of sensitive data during cross-domain offloading are effectively guaranteed, while maintaining the scalability of the system. Attached Figure Description

[0043] Figure 1 This is a flowchart of the centralized training process in an embodiment of the present invention;

[0044] Figure 2 This is a schematic diagram of the centralized training algorithm in an embodiment of the present invention;

[0045] Figure 3 This is a flowchart of the decentralized execution process in an embodiment of the present invention. Detailed Implementation

[0046] This invention aims to propose a domain-secure collaborative offloading method that considers tail risks, overcoming the shortcomings of existing methods that only optimize average performance while neglecting service quality stability, have low collaborative efficiency, and suffer from cross-domain security risks, thus enabling it to better complete mobile edge computing offloading tasks. Its overall implementation process includes two main stages: centralized training and decentralized execution. The overall process of centralized training is as follows: Figure 1As shown, the process includes: acquiring the system state and performing representation compression calculation of the collaborative auxiliary reward; calculating the tail risk penalty; updating the tail risk constraint multiplier; training the policy and value network to obtain the trained unloading model. The decentralized execution process is as follows: Figure 3 As shown, this includes acquiring local observations and making decisions; and performing secure unloading.

[0047] Example:

[0048] The present invention will now be described in detail. See below. Figure 2 The specific implementation process of centralized training is as follows:

[0049] S1. Obtain the global system state, compress the state using a variational information bottleneck encoder, and generate a low-dimensional task representation containing only information related to the offloading decision. This is to filter out random disturbances and irrelevant information in the state; specifically as follows:

[0050] S11, Set global system state Input encoder network;

[0051] S12, Average output value of encoder network Sum of logarithmic variance The logarithmic variance is obtained through Calculations are used to control the range. This represents the hidden features extracted by the encoder. and These represent the weight matrix and bias vector of the log-variance mapping layer, respectively;

[0052] S13. Generate task representation z using reparameterization techniques:

[0053]

[0054] in From the standard normal distribution noise in the mid-sample, This represents element-wise multiplication;

[0055] S14, By minimizing the KL divergence loss To train encoder parameters Overcoming information bottlenecks:

[0056]

[0057] in, This represents the KL divergence.

[0058] S2. Calculate collaborative assistance rewards: based on the task representation. Joint actions of all intelligent agents Calculate confidence mutual information This is used as an auxiliary reward signal to promote effective collaboration among agents; specifically as follows:

[0059] S21. Set the positive sample buffer. and negative sample buffer The cumulative total cost of each experience trajectory within a task cycle. As a quality evaluation parameter, among which... This represents the cost during the T-th task cycle. , and These represent the task number, user equipment number, and time slot number, respectively. The total cost is calculated based on the cumulative cost of each trajectory within the current training batch. the median of To determine the threshold, the following conditions will be met. High-quality experience storage , will satisfy Low-quality experience storage ;

[0060] S22. Using a Mutual Information Neural Estimation (MINE) estimator, through a neural network... Estimating the positive sample buffer middle and Mutual information lower bound And minimize the MINE estimation loss. Training was conducted; among them, This represents a joint action consisting of the actions of all agents. Indicates network parameters; The lower bound of the calculation of mutual information The opposite of, i.e. By minimizing Maximize the lower bound of mutual information.

[0061] S23. Using the Contrastive Log-ratio Upper Bound (CLUB) estimator, through a neural network... Estimating the negative sample buffer middle and Upper bound of mutual information And minimize the negative log-likelihood loss. Training was conducted; among them, For the network parameters of the estimator, the negative log-likelihood loss Used to fit conditional distribution And improve the accuracy of mutual information upper bound estimation.

[0062] S24. Combine the two estimators to obtain the final confidence mutual information-assisted reward. ,in and To balance the hyperparameters.

[0063] S3. Calculate tail risk penalty: Collect a set of actual system costs for task execution from the training batch. Calculate the high percentile statistic of the cost group, and set an acceptable cost threshold, based on which the tail risk penalty term is calculated; specifically as follows:

[0064] S31. Based on the unloading decision Calculate the latency for local execution, edge uninstallation, and cloud uninstallation respectively. , , and energy consumption , ;

[0065] S32. Calculate security costs That is, cross-domain transmission ( ) to be punished; among them, As an indicator variable for cross-domain transfer, when a task is executed within the same trust domain. When transmitting across different trust domains ; Indicates time slot The security costs of the corresponding tasks within the system.

[0066] S33. Weighted summation yields the total cost. ,in and These represent the actual execution latency and energy consumption corresponding to the current unloading decision, respectively; the actual execution latency is calculated as follows:

[0067]

[0068] in, Represents the total cost. , and These represent the non-negative weighting coefficients for latency, energy consumption, and security costs, respectively. ; , and These represent the effective latency for local execution, device-to-edge server unloading, and device-to-cloud server unloading, respectively. The latency for unselected execution paths is set to 0.

[0069] S34. Collect a set of total costs calculated by S33 from the training batch. Calculate the 95th percentile of the cost for this group. And set an acceptable cost threshold. Based on this, the tail risk penalty item is calculated. :

[0070]

[0071] S4. Introduce an adaptive Lagrange multiplier. ( The multiplier is automatically updated based on the tail risk penalty term to dynamically balance average cost optimization and tail risk constraints.

[0072] In the k-th iteration, the tail risk penalty term calculated according to S34 Update the adaptive Lagrange multipliers using gradient ascent:

[0073]

[0074] in for Update learning rate, For the k-th iteration, the adaptive Lagrange multiplier is used.

[0075] This update mechanism enables automatic adjustment: if (i.e., tail risk exceeds the threshold), then Increase to strengthen punishment; if (i.e., satisfying the constraints), then Reduce or keep it at 0 to reduce the penalty.

[0076] S5. Training the Policy and Value Network: A composite objective function is used to update the policy and value networks in multi-agent reinforcement learning. This composite objective function includes minimizing the average system cost, maximizing the confidence-mutual-trust auxiliary reward, and minimizing the [missing information - likely a factor in the original text]. The scaled tail risk penalty term ultimately yields the trained unloading strategy model.

[0077] S51, Multi-agent reinforcement learning basic loss For example, the policy network loss and value network loss of the MADDPG algorithm, where the training objective of the value network includes the confidence mutual information auxiliary reward calculated by S24. ;

[0078] S52, Variational Information Bottleneck Loss (From S14) and confidence mutual information estimator loss , (From S22, S23);

[0079] S53, Tail Risk Loss ,in This step is treated as a constant and is only used to calculate the loss; its update is performed independently in S4.

[0080] S54. By minimizing the total loss To train all network parameters, where This is the balance coefficient for the variational information bottleneck loss.

[0081] like Figure 3 As shown, the specific process of decentralized execution is as follows:

[0082] S6. Acquire local observations and make decisions:

[0083] S61. Agent m loads the offloading policy model trained in S5, relying solely on local observations. Obtain the local state required for decision-making;

[0084] S62, Change local state Input the strategy model to generate an unloading decision action. .

[0085] S7. Perform secure uninstallation: Determine the decision action. If cross-domain transmission is triggered, then domain security rules are activated, lightweight key exchange protocol and symmetric encryption algorithm are used to encrypt the offloaded data, and the offload task is executed.

[0086] S71, Determine the decision action. Does it trigger cross-domain transfer? );

[0087] S72, If so, user equipment Perform an Elliptic Curve Diffie-Hellman (ECDH) key exchange protocol with the target server (edge ​​server or cloud server) to negotiate and generate a temporary session key. ;

[0088] S73, User Equipment Use the session key The task data is encrypted using the Advanced Encryption Standard (AES) algorithm. Encrypt: ,in, This represents the original task data to be uninstalled. This represents the encrypted task data. Indicates the use of session keys right Perform AES encryption;

[0089] The target server received the ciphertext Then, use the same session key. Decryption: . Indicates the use of the same session key right Perform AES decryption.

[0090] If the determination is negative, then perform a normal local or domain uninstallation.

[0091] Although the present invention has been described herein with reference to embodiments thereof, the above embodiments are merely preferred embodiments of the present invention, and the implementation of the present invention is not limited to the above embodiments. It should be understood that those skilled in the art can design many other modifications and implementations, which will fall within the scope and spirit of the principles disclosed in this application.

Claims

1. A method for offloading multi-agent collaborative tasks in a domain-secure mobile edge computing scenario, characterized in that, It includes a centralized training process and a decentralized execution process, with the specific steps as follows: Centralized training process: S1. Obtain the global system state, compress the global system state using a variational information bottleneck encoder, and generate a task representation that contains only information related to the offloading decision. S2. Based on the task representation and the joint actions of all agents, calculate the confidence mutual information and use it as an auxiliary reward signal; S3. Collect a set of actual system costs for task execution from the training batch, calculate the high percentile statistic of the actual system costs for this set, set a cost threshold, and calculate the tail risk penalty item accordingly. S4. Introduce an adaptive Lagrange multiplier and automatically update the multiplier based on the tail risk penalty term; S5. Training Policy and Value Network: A composite objective function is used to update the policy network and value network in multi-agent reinforcement learning. This composite objective function includes minimizing the average system cost, maximizing the confidence mutual trust auxiliary reward, and minimizing the tail risk penalty term scaled by adaptive Lagrange multipliers, ultimately obtaining the trained offloading policy model. Decentralized execution process: S6. The agent loads the trained unloading policy model and obtains the local state required for decision-making based solely on local observations; the local state is input into the policy model to generate the unloading decision action. S7. Determine whether the decision action triggers cross-domain transmission. If so, activate the domain security rules, use a lightweight key exchange protocol and a symmetric encryption algorithm to encrypt the unloading data, and execute the unloading task.

2. The multi-agent collaborative task offloading method for a domain-secure mobile edge computing scenario according to claim 1, characterized in that, The task representation is obtained through the following process: S11, Set global system state Input encoder network; S12, Average output value of encoder network Sum of logarithmic variance The logarithmic variance is obtained through Calculations to control the range, This represents the hidden features extracted by the encoder. and These represent the weight matrix and the bias vector, respectively. S13, Based on mean Sum of logarithmic variance The task representation z is generated through reparameterization techniques. S14. By minimizing the variational information bottleneck loss To train encoder network parameters .

3. A multi-agent collaborative task offloading method for a domain-secure mobile edge computing scenario according to claim 2, characterized in that, The confidence mutual information is obtained through the following steps: S21. Set the positive sample buffer. and negative sample buffer The cumulative reward or cumulative cost of a task is used as a quality evaluation parameter. A threshold is set based on the quality evaluation parameter to distinguish between high-quality and low-quality experiences. High-quality experiences are stored in a positive sample buffer. Low-quality experience is stored in the negative sample buffer. ; S22. Use a mutual information neural estimator to estimate the positive sample buffer. Medium task representation and Mutual information lower bound And train the mutual information neural estimator by minimizing its loss; where, This represents a joint action consisting of the actions of all agents. S23. Use the contrast log-ratio upper bound estimator to estimate the negative sample buffer. middle and Upper bound of mutual information And train the comparison log ratio upper bound estimator by minimizing its loss; S24. Combine the two estimators to obtain the final confidence mutual information-assisted reward. ,in and To balance the hyperparameters.

4. A multi-agent collaborative task offloading method for a domain-secure mobile edge computing scenario according to claim 3, characterized in that, The specific calculation of the tail risk penalty item is as follows: S31. Based on the unloading decision The latency and energy consumption of local execution, edge unloading, and cloud unloading were calculated respectively; among them , These represent the task number and the user device number, respectively. S32. Calculate security costs This means penalizing cross-domain data transfers. Indicates the time slot number; S33. Weighted summation yields the total cost: ,in and These represent the actual execution latency and energy consumption corresponding to the current unloading decision, respectively. , and These represent the non-negative weighting coefficients for latency, energy consumption, and security costs, respectively. ; S34. Collect a set of total costs calculated by S33 from the training batch. Calculate the 95th percentile of the cost for this group. and set a cost threshold. Based on this, the tail risk penalty item is calculated. : 。 5. A multi-agent collaborative task offloading method for a domain-secure mobile edge computing scenario according to claim 4, characterized in that, The adaptive Lagrange multiplier is as follows: In the k-th iteration, based on the tail risk penalty term Update the adaptive Lagrange multipliers using gradient ascent: in for Update learning rate, Let be the adaptive Lagrange multiplier for the k-th iteration.

6. A multi-agent collaborative task offloading method for a domain-secure mobile edge computing scenario according to claim 5, characterized in that, The composite objective function specifically includes: the multi-agent reinforcement learning basic loss. The training objectives of the value network include: confidence mutual information and auxiliary rewards. Variational information bottleneck loss Loss of mutual information neural estimator ; Compare the loss of the logarithmic ratio upper bound estimator Tail risk loss ;in This step is treated as a constant and is only used to calculate the loss; its update is performed independently in S4. By minimizing the total loss To train all network parameters, where This is the balance coefficient for the variational information bottleneck loss.

7. A multi-agent collaborative task offloading method for a domain-secure mobile edge computing scenario according to claim 6, characterized in that, The variational information bottleneck loss is specifically the KL divergence loss, and the loss of the mutual information neural estimator is specifically the lower bound of mutual information. The opposite of, i.e. The loss of the log-to-upper bound estimator is specifically the negative log-likelihood loss.

8. A multi-agent collaborative task offloading method for a domain-secure mobile edge computing scenario according to claim 7, characterized in that, The actual execution delay is calculated as follows: in, , , This indicates the latency for local execution, edge uninstallation, and cloud uninstallation. The latency for execution paths that are not selected is set to 0.

9. A multi-agent collaborative task offloading method for a domain-secure mobile edge computing scenario according to claim 8, characterized in that, The cumulative total cost of each experience trajectory within a task cycle is used as the quality evaluation parameter, and the median of the cumulative total cost of all trajectories in the current training batch is used as the evaluation parameter. To determine the threshold, the following conditions will be met. High-quality experience storage , will satisfy Low-quality experience storage .

10. A multi-agent collaborative task offloading method for a domain-secure mobile edge computing scenario according to claim 9, characterized in that, The startup domain security rules employ a lightweight key exchange protocol and a symmetric encryption algorithm to encrypt the uninstallation data and execute the uninstallation task, as detailed below: User equipment Perform an elliptic curve Diffie-Hellman key exchange protocol with the target server to negotiate and generate a temporary session key; User equipment The task data is encrypted using the session key and the Advanced Encryption Standard (AES) encryption algorithm. After receiving the ciphertext, the target server decrypts it using the same session key.

Images