Multi-agent driven cloud-edge-end security task multi-objective adaptive scheduling method

Abstract

The application discloses a multi-agent driven cloud-edge-end security task multi-objective adaptive scheduling method, relates to the technical field of network space security, and overcomes the defects of poor adaptability of a traditional static scheduling strategy in a heterogeneous resource environment by introducing a real-time perception and dynamic decision mechanism in a cloud-edge-end three-layer architecture in cooperation with multiple agents. The system takes the average processing time delay of a task, a security detection accuracy and a privacy entropy of a scheduling strategy as parallel optimization targets, constructs a multi-objective reward function, and enables the scheduling strategy to realize adaptive balance between efficiency, security and privacy protection.

Description

Technical Field

[0001] This invention relates to the field of cyberspace security technology, specifically to a multi-agent driven cloud-edge-device security task multi-objective adaptive scheduling method. Background Technology

[0002] With the widespread adoption of 5G and IoT technologies, mobile terminals have become a core component of modern information systems. However, terminal devices are limited by computing, storage, and energy resources, making it difficult to independently deploy complex security detection mechanisms. This makes them highly vulnerable to diverse attack methods. Traditional solutions primarily employ a centralized cloud computing architecture, transmitting network security data (including network traffic, system logs, and other critical information) to the cloud for centralized detection. While this architecture alleviates the resource bottleneck of mobile terminal devices to some extent, it inevitably introduces significant task latency. High latency can hinder real-time security detection responses, thereby increasing security risks. Inspired by the edge computing paradigm, recent research has begun exploring the Terminal-Edge-Cloud Collaboration (EECC) architecture. This architecture effectively alleviates the pressure on terminal resources by offloading security tasks to the more resource-rich edge and cloud.

[0003] However, these EECC methods typically rely on static task allocation strategies, where security tasks are pre-assigned to specific levels on devices and remain fixed during system operation. This static allocation paradigm cannot adapt to the dynamic changes in resource availability among heterogeneous devices. On the one hand, while over-concentrating detection tasks in the cloud can achieve finer detection granularity, it inevitably introduces significant task latency; conversely, allocating a large number of tasks to edge devices wastes available cloud resources and reduces overall detection granularity. Furthermore, the specific allocation patterns in task scheduling strategies can be captured by external attackers through long-term observation or traffic analysis, such as the selection of task offloading paths and target devices. Attackers can infer device task offloading preferences and even sensitive behavioral patterns based on the regularities exhibited in such scheduling behavior, thereby increasing the risk of privacy breaches.

[0004] In summary, current task scheduling technologies in the terminal-edge-cloud environment still lack effective solutions in achieving the three key objectives of balancing and optimizing task processing latency, security detection accuracy, and scheduling behavior privacy. Summary of the Invention

[0005] To overcome the shortcomings of the above technologies, this invention provides a method for guiding intelligent agents to make decisions with the unified goals of collaboratively optimizing the average processing latency of tasks, the accuracy of security detection, and the privacy entropy of scheduling strategies.

[0006] The technical solution adopted by this invention to overcome its technical problems is:

[0007] A multi-agent driven cloud-edge-device security task multi-objective adaptive scheduling method includes:

[0008] S1. Deploy an intelligent agent on each cloud device, edge device, and terminal device. Each intelligent agent collects... Observation data of the corresponding device at any time , , For periodicity;

[0009] S2. Deploy a policy network on each agent to process the observed data. The input is fed into the policy network, and the output is the security task scheduling policy. ;

[0010] S3. Calculation Global privacy entropy at any moment ;

[0011] S4. Utilizing global privacy entropy Calculated Real-time security task scheduling rewards ;

[0012] S5. Each agent collects... Observation data of the corresponding device at any time ;

[0013] S6. Constructing a collection ,gather ,gather ;

[0014] S7. Deploy a twin network consisting of an encoder, a projector head, and a prediction head on each agent, and aggregate... ,gather Enhancement is performed by inputting the enhanced view into the encoder of the Siamese network, and the output is the feature. ,feature ,feature ,feature Utilizing features ,feature ,feature ,feature Task scheduling rewards The optimized twin network is obtained by training the twin network.

[0015] S8. Set The input is fed into the encoder of the optimized Siamese network, and the output is the feature. , will set The input is fed into the encoder of the optimized Siamese network, and the output is the feature. ;

[0016] S9. Deploy a value network on each agent, integrating features ,feature The inputs are fed into the value network respectively, and the outputs are the state values. State value ;

[0017] S10. Utilizing State Value State value ,gather Calculate the loss function, train the policy network using the loss function, and obtain the optimized policy network;

[0018] S11. Input the new observation data collected by the agent into the optimized policy network, and output the final safe task scheduling policy.

[0019] Furthermore, in step S1, the first Each device is , , A value of 0 indicates that the device deploying the intelligent agent is a terminal device. A value of 1 indicates that the device deploying the intelligent agent is an edge device. A value of 2 indicates that the device deploying the intelligent agent is a cloud-based device. ,in For the number of terminal devices, For the number of edge devices, For the number of cloud devices, the first One device intelligent agent collection Observational data at time Observational data For tuples, , For the first One device exist Task queue length at any given time For edge devices and cloud devices The set of task queue lengths at any given time. For edge devices and cloud devices CPU utilization at any given moment For edge devices and cloud devices Set of memory utilization at any given moment for Time of the first One device A collection of bandwidth between edge devices and cloud devices.

[0020] Furthermore, step S2 includes the following steps:

[0021] S2-1. In the first One device Deploy policy networks in intelligent agents;

[0022] S2-2. The policy network consists of a first fully connected layer, a ReLU activation function, a second fully connected layer, a Tanh activation function, a mean output branch, and a standard deviation output branch;

[0023] S2-3. Observation data The inputs are sequentially fed into the first fully connected layer of the policy network and the ReLU activation function, and the output is the feature. ;

[0024] S2-4. Features The inputs are sequentially fed into the second fully connected layer of the policy network and the Tanh activation function, and the output is the feature. ;

[0025] S2-5. The mean output branch of the policy network consists of a fully connected layer and a Tanh activation function, which sequentially converts the features... The input is fed into the mean output branch, and the output is the mean vector. ;

[0026] S2-6. The standard deviation output branch of the policy network consists of a fully connected layer and a SoftPlus activation function, which sequentially converts features... The input is fed into the standard deviation output branch, and the output is the standard deviation vector. ;

[0027] S2-7. Based on mean vector and standard deviation vector Construct a Gaussian probability distribution, and use the sample() method of the torch.distributions.Normal class in the PyTorch library to sample the first element from the Gaussian probability distribution. One device The original policy vector ;

[0028] S2-8. For the original policy vector After normalization, we obtain the first... One device At any moment Security task scheduling strategy .

[0029] Furthermore, step S3 includes the following steps:

[0030] S3-1. Through formula Calculated Time of the first Terminal devices Dispatch to the Target device probability In the formula, for Time of the first Terminal devices Dispatch to the Target device The number of security tasks, , for Time of the first Terminal devices The total number of scheduled security tasks;

[0031] S3-2. Using the formula Calculation yields the first Terminal devices At any moment Information entropy ;

[0032] S3-3. Through formula The calculation of the terminal device at time Average information entropy ;

[0033] S3-4. Through formula The calculation of the terminal device at time Normalized information entropy ;

[0034] S3-5. Through formula Calculation yields the first Target device At any moment global probability In the formula, For at any time The scheduling to the first Target device The number of security tasks, The total number of all scheduled security tasks;

[0035] S3-6. Through formula The calculated time global entropy ;

[0036] S3-7. Through formula The calculated time Normalized global entropy ;

[0037] S3-8. Through formula The calculated time Global privacy entropy .

[0038] Furthermore, in step S4, the formula is used... Calculated Real-time security task scheduling rewards In the formula, The weighting coefficient for average scheduling delay. For the central controller to return time The average scheduling delay generated by all devices completing the safety task scheduling This is a weighting coefficient for the average security detection accuracy. For the central controller to return time The average safety detection accuracy rate generated by all devices completing safety task scheduling. This represents the weighting coefficient of the global privacy entropy.

[0039] Furthermore, in step S5... One device intelligent agent collection Observational data at time Observational data For tuples, , For the first One device exist Task queue length at any given time For edge devices and cloud devices The set of task queue lengths at any given time. For edge devices and cloud devices CPU utilization at any given moment For edge devices and cloud devices Set of memory utilization at any given moment for Time of the first One device A collection of bandwidth between edge devices and cloud devices.

[0040] Furthermore, step S6 includes the following steps:

[0041] S6-1. Pass indivual Constructing a set of observation data at each time point , ;

[0042] S6-2. Pass A collection of security task scheduling strategies , ;

[0043] S6-3. Pass indivual Constructing a set of observation data at each time point , .

[0044] Furthermore, step S7 includes the following steps:

[0045] S7-1. Use the `torch.rand_like()` function from the PyTorch library to generate a set. Isomorphic vectors ,vector Each element in the set is a random number within the interval [0, 1). When an element's value is less than 10%, its value is set to 0; when its value is greater than or equal to 10%, its value is set to 1. with vector Perform element-wise multiplication to obtain a randomly discarded vector. Then, use the `torch.randint()` function from the PyTorch library to randomly generate a new vector with a 5% probability. Given a unique index, set the value of the element corresponding to that index to 0 to obtain the enhanced view. , will vector Multiply by standard deviation Gaussian noise is obtained. Add Gaussian noise to the set Get the enhanced view above. ;

[0046] S7-2. Use the `torch.rand_like()` function from the PyTorch library to generate a set. Isomorphic vectors ,vector Each element in the set is a random number within the interval [0, 1). When an element's value is less than 10%, its value is set to 0; when its value is greater than or equal to 10%, its value is set to 1. with vector Perform element-wise multiplication to obtain a randomly discarded vector. Then, use the `torch.randint()` function from the PyTorch library to randomly generate a new vector with a 5% probability. Given a unique index, set the value of the element corresponding to that index to 0 to obtain the enhanced view. , will vector Multiply by standard deviation Gaussian noise is obtained, and the Gaussian noise is added to the set. Get the enhanced view above. ;

[0047] S7-3. The encoder of the Siamese network consists of a first fully connected layer, a first LN layer, a first ReLU activation function, a second fully connected layer, a second LN layer, a second ReLU activation function, and a third fully connected layer, which enhances the view. The input is fed into the encoder, and the output is the feature. This will enhance the view. The input is fed into the encoder, and the output is the feature. This will enhance the view. The input is fed into the encoder, and the output is the feature. This will enhance the view. The input is fed into the encoder, and the output is the feature. ;

[0048] S7-4. The projection head of the Siamese network consists of a first fully connected layer, a Batch Normalization (BN) layer, a ReLU activation function, and a second fully connected layer, which sequentially transmits features. The input is fed into the projection head, and the output is the feature. , will feature The input is fed into the projection head, and the output is the feature. , will feature The input is fed into the projection head, and the output is the feature. ,feature The input is fed into the projection head, and the output is the feature. ;

[0049] S7-5. The prediction head of the Siamese network consists of a first fully connected layer, a Batch Normalization (BN) layer, a ReLU activation function, and a second fully connected layer, which integrates features. The input is fed into the prediction head, and the output is the feature. , will feature The input is fed into the prediction head, and the output is the feature. , will feature The input is fed into the prediction head, and the output is the feature. ,feature The input is fed into the prediction head, and the output is the feature. ;

[0050] S7-6. Through formula

[0051] The basic cosine similarity loss was calculated. In the formula, For negative cosine similarity, To stop the gradient operation;

[0052] S7-7. Rewards for Task Scheduling Normalization is performed to obtain the normalized task scheduling reward. ;

[0053] S7-8. Through formula Calculated Real-time task scheduling rewards Weight ;

[0054] S7-9. Through formula The weighted cosine similarity loss function is calculated. Gradient descent is used with a weighted cosine similarity loss function. Train the twin network to obtain the optimized twin network.

[0055] Furthermore, step S9 includes the following steps:

[0056] S9-1. The value network consists of a first fully connected layer, a first ReLU activation function, a second fully connected layer, a second ReLU activation function, a third fully connected layer, and a third ReLU activation function, in sequence.

[0057] S9-2. Features The input is fed into the value network, and the output is the state value. ;

[0058] S9-3. Features The input is fed into the value network, and the output is the state value. ;

[0059] S9-4. Utilizing State Value State value Safety task scheduling rewards Calculate timing difference error By calculating the timing difference error The loss function of the value network is obtained from the mean squared error, and the value network is trained using the loss function of the value network through gradient descent.

[0060] Furthermore, step S10 includes the following steps:

[0061] S10-1. Based on time-series difference error Calculate the advantage function ;

[0062] S10-2. Based on sets The importance sampling ratio is calculated using an importance sampling method, and then combined with the dominance function. We construct the loss function of the policy network by using the pruning mechanism in multi-agent proximal policy optimization.

[0063] The beneficial effects of this invention are as follows: By introducing real-time perception and dynamic decision-making mechanisms into a multi-agent collaborative cloud-edge-device three-layer architecture, the shortcomings of traditional static scheduling strategies in adaptability to heterogeneous resource environments are overcome. The system constructs a multi-objective reward function with average task processing latency, security detection accuracy, and scheduling strategy privacy entropy as parallel optimization objectives, enabling the scheduling strategy to achieve an adaptive balance between efficiency, security, and privacy protection. Privacy entropy quantifies the uniformity and uncertainty of task distribution, effectively resisting attackers' behavioral inferences based on scheduling patterns and enhancing user privacy protection. A reward-driven Siamese network mechanism is designed. The Siamese network performs data augmentation and consistency encoding on high-dimensional environment states, effectively filtering out observation noise and extracting robust core feature representations related to task decision-making. The reward-guided mechanism further uses secure task scheduling rewards to weight the state representation learning process, giving high-reward states a greater weight in feature learning, thus improving the quality and discriminative power of feature representations. This mechanism, in conjunction with a multi-agent reinforcement learning framework, enables each device agent to learn and dynamically adjust its policies based on clearer and more discriminative features, significantly enhancing the system's overall scheduling performance, adaptability, and robustness in dynamic and complex environments. Attached Figure Description

[0064] Figure 1 This is a network framework diagram of the present invention;

[0065] Figure 2 This is a flowchart of the method of the present invention. Detailed Implementation

[0066] The following is in conjunction with the appendix Figure 1 Appendix Figure 2 The present invention will be further described below.

[0067] A multi-agent-driven multi-objective adaptive scheduling method for cloud-edge-device security tasks, with the unified goal of collaboratively optimizing average task processing latency, security detection accuracy, and scheduling policy privacy entropy, guides agents to make decisions, and includes the following steps:

[0068] S1. Deploy an intelligent agent on each cloud device, edge device, and terminal device. Each intelligent agent collects... Observation data of the corresponding device at any time , , It is a periodicity.

[0069] S2. Deploy a policy network on each agent to process the observed data. The input is fed into the policy network, and the output is the security task scheduling policy. .

[0070] S3. Calculation Global privacy entropy at any moment .

[0071] S4. Utilizing global privacy entropy Calculated Real-time security task scheduling rewards .

[0072] S5. Each agent collects... Observation data of the corresponding device at any time .

[0073] S6. Constructing a collection ,gather ,gather .

[0074] S7. Deploy a twin network consisting of an encoder, a projector head, and a prediction head on each agent, and aggregate... ,gather Enhancement is performed by inputting the enhanced view into the encoder of the Siamese network, and the output is the feature. ,feature ,feature ,feature Utilizing features ,feature ,feature ,feature Task scheduling rewards The optimized twin network is obtained by training the twin network.

[0075] S8. Set The input is fed into the encoder of the optimized Siamese network, and the output is the feature. , will set The input is fed into the encoder of the optimized Siamese network, and the output is the feature. .

[0076] S9. Deploy a value network on each agent, integrating features ,feature The inputs are fed into the value network respectively, and the outputs are the state values. State value .

[0077] S10. Utilizing State Value State value ,gather Calculate the loss function, train the policy network using the loss function, and obtain the optimized policy network.

[0078] S11. Input the new observation data collected by the agent into the optimized policy network, and output the final safe task scheduling policy.

[0079] To address the challenge of existing scheduling schemes adapting to dynamic resource environments, this invention constructs a three-layer collaborative computing architecture encompassing terminals, edge computing, and the cloud. Intelligent agents are deployed on each device to monitor local task queues, resource utilization, and network bandwidth in real time. Dynamic scheduling decisions are generated through a policy network, with average task processing latency, security detection accuracy, and scheduling policy privacy entropy serving as parallel optimization objectives. A multi-objective reward function is constructed to achieve an adaptive balance between efficiency, security, and privacy protection in the scheduling policy. Building upon this, a reward-driven Siamese network mechanism is designed. By applying noise perturbation and enhancement transformations to the state vector, feature consistency is maintained across different enhancement perspectives to filter out environmental noise and obtain robust feature representations. Furthermore, scheduling rewards are used to adaptively weight the feature learning process, giving higher weight to high-reward states in representation learning, significantly improving feature discrimination capabilities. Finally, a multi-agent reinforcement learning framework is used to collaboratively train all agents, enabling the system to achieve an optimal balance between task processing efficiency, security detection accuracy, and user privacy protection in dynamic and complex environments, comprehensively improving scheduling performance and system robustness.

[0080] In one embodiment of the present invention, the first step S1 Each device is , , A value of 0 indicates that the device deploying the intelligent agent is a terminal device. A value of 1 indicates that the device deploying the intelligent agent is an edge device. A value of 2 indicates that the device deploying the intelligent agent is a cloud-based device. ,in For the number of terminal devices, For the number of edge devices, For the number of cloud devices, the first One device intelligent agent collection Observational data at time Observational data For tuples, , For the first One device exist Task queue length at any given time For edge devices and cloud devices The set of task queue lengths at any given time. For edge devices and cloud devices CPU utilization at any given moment For edge devices and cloud devices Set of memory utilization at any given moment for Time of the first One device A collection of bandwidth between edge devices and cloud devices.

[0081] In one embodiment of the present invention, step S2 includes the following steps:

[0082] S2-1. In the first One device Deploy policy networks in intelligent agents.

[0083] S2-2. The policy network consists of a first fully connected layer, a ReLU activation function, a second fully connected layer, a Tanh activation function, a mean output branch, and a standard deviation output branch.

[0084] S2-3. Observation data The inputs are sequentially fed into the first fully connected layer of the policy network and the ReLU activation function, and the output is the feature. .

[0085] S2-4. Features The inputs are sequentially fed into the second fully connected layer of the policy network and the Tanh activation function, and the output is the feature. .

[0086] S2-5. The mean output branch of the policy network consists of a fully connected layer and a Tanh activation function, which sequentially converts the features... The input is fed into the mean output branch, and the output is the mean vector. .

[0087] S2-6. The standard deviation output branch of the policy network consists of a fully connected layer and a SoftPlus activation function, which sequentially converts features... The input is fed into the standard deviation output branch, and the output is the standard deviation vector. .

[0088] S2-7. Based on mean vector and standard deviation vector Construct a Gaussian probability distribution, and use the sample() method of the torch.distributions.Normal class in the PyTorch library to sample the first element from the Gaussian probability distribution. One device The original policy vector .

[0089] S2-8. For the original policy vector After normalization, we obtain the first... One device At any moment Security task scheduling strategy .

[0090] In one embodiment of the present invention, step S3 includes the following steps:

[0091] S3-1. Through formula Calculated Time of the first Terminal devices Dispatch to the Target device probability In the formula, for Time of the first Terminal devices Dispatch to the Target device The number of security tasks, , for Time of the first Terminal devices Total number of scheduled security tasks.

[0092] S3-2. Using the formula Calculation yields the first Terminal devices At any moment Information entropy .

[0093] S3-3. Through formula The calculation of the terminal device at time Average information entropy .

[0094] S3-4. Through formula The calculation of the terminal device at time Normalized information entropy .

[0095] S3-5. Through formula Calculation yields the first Target device At any moment global probability In the formula, For at any time The scheduling to the first Target device The number of security tasks, This represents the total number of all scheduled security tasks.

[0096] S3-6. Through formula The calculated time global entropy .

[0097] S3-7. Through formula The calculated time Normalized global entropy .

[0098] S3-8. Through formula The calculated time Global privacy entropy .

[0099] In one embodiment of the present invention, step S4 is performed using the formula Calculated Real-time security task scheduling rewards In the formula, The weighting coefficient for average scheduling delay. For the central controller to return time The average scheduling delay generated by all devices completing the safety task scheduling This is a weighting coefficient for the average security detection accuracy. For the central controller to return time The average safety detection accuracy rate generated by all devices completing safety task scheduling. This represents the weighting coefficient of the global privacy entropy.

[0100] In one embodiment of the present invention, in step S5... One device intelligent agent collection Observational data at time Observational data For tuples, , For the first One device exist Task queue length at any given time For edge devices and cloud devices The set of task queue lengths at any given time. For edge devices and cloud devices CPU utilization at any given moment For edge devices and cloud devices Set of memory utilization at any given moment for Time of the first One device A collection of bandwidth between edge devices and cloud devices.

[0101] In one embodiment of the present invention, step S6 includes the following steps:

[0102] S6-1. Pass indivual Constructing a set of observation data at each time point , .

[0103] S6-2. Pass A collection of security task scheduling strategies , .

[0104] S6-3. Pass indivual Constructing a set of observation data at each time point , .

[0105] In one embodiment of the present invention, step S7 includes the following steps:

[0106] S7-1. Use the `torch.rand_like()` function from the PyTorch library to generate a set. Isomorphic vectors ,vector Each element in the set is a random number within the interval [0, 1). When an element's value is less than 10%, its value is set to 0; when its value is greater than or equal to 10%, its value is set to 1. with vector Perform element-wise multiplication to obtain a randomly discarded vector. Then, use the `torch.randint()` function from the PyTorch library to randomly generate a new vector with a 5% probability. Given a unique index, set the value of the element corresponding to that index to 0 to obtain the enhanced view. , will vector Multiply by standard deviation Gaussian noise is obtained. Add Gaussian noise to the set Get the enhanced view above. .

[0107] S7-2. Use the `torch.rand_like()` function from the PyTorch library to generate a set. Isomorphic vectors ,vector Each element in the set is a random number within the interval [0, 1). When an element's value is less than 10%, its value is set to 0; when its value is greater than or equal to 10%, its value is set to 1. with vector Perform element-wise multiplication to obtain a randomly discarded vector. Then, use the `torch.randint()` function from the PyTorch library to randomly generate a new vector with a 5% probability. Given a unique index, set the value of the element corresponding to that index to 0 to obtain the enhanced view. , will vector Multiply by standard deviation Gaussian noise is obtained, and the Gaussian noise is added to the set. Get the enhanced view above. .

[0108] S7-3. The encoder of the Siamese network consists of a first fully connected layer, a first LN layer, a first ReLU activation function, a second fully connected layer, a second LN layer, a second ReLU activation function, and a third fully connected layer, which enhances the view. The input is fed into the encoder, and the output is the feature. This will enhance the view. The input is fed into the encoder, and the output is the feature. This will enhance the view. The input is fed into the encoder, and the output is the feature. This will enhance the view. The input is fed into the encoder, and the output is the feature. .

[0109] S7-4. The projection head of the Siamese network consists of a first fully connected layer, a Batch Normalization (BN) layer, a ReLU activation function, and a second fully connected layer, which sequentially transmits features. The input is fed into the projection head, and the output is the feature. , will feature The input is fed into the projection head, and the output is the feature. , will feature The input is fed into the projection head, and the output is the feature. ,feature The input is fed into the projection head, and the output is the feature. .

[0110] S7-5. The prediction head of the Siamese network consists of a first fully connected layer, a Batch Normalization (BN) layer, a ReLU activation function, and a second fully connected layer, which integrates features. The input is fed into the prediction head, and the output is the feature. , will feature The input is fed into the prediction head, and the output is the feature. , will feature The input is fed into the prediction head, and the output is the feature. ,feature The input is fed into the prediction head, and the output is the feature. .

[0111] S7-6. Through formula

[0112] The basic cosine similarity loss was calculated. In the formula, For negative cosine similarity, To stop the gradient operation.

[0113] S7-7. Rewards for Task Scheduling Normalization is performed to obtain the normalized task scheduling reward. .

[0114] S7-8. Through formula Calculated Real-time task scheduling rewards Weight .

[0115] S7-9. Through formula The weighted cosine similarity loss function is calculated. Gradient descent is used with a weighted cosine similarity loss function. Train the Siamese network until the predetermined number of training iterations or the weighted cosine similarity loss function is reached. The optimized twin network is obtained by continuing until convergence.

[0116] In one embodiment of the present invention, step S9 includes the following steps:

[0117] S9-1. The value network consists of a first fully connected layer, a first ReLU activation function, a second fully connected layer, a second ReLU activation function, a third fully connected layer, and a third ReLU activation function.

[0118] S9-2. Features The input is fed into the value network, and the output is the state value. .

[0119] S9-3. Features The input is fed into the value network, and the output is the state value. .

[0120] S9-4. Utilizing State Value State value Safety task scheduling rewards Calculate timing difference error By calculating the timing difference error The loss function of the value network is obtained from the mean squared error, and the value network is trained using the loss function of the value network through gradient descent.

[0121] In one embodiment of the present invention, step S10 includes the following steps:

[0122] S10-1. Based on time-series difference error Calculate the advantage function .

[0123] S10-2. Based on sets The importance sampling ratio is calculated using an importance sampling method, and then combined with the dominance function. The loss function of the policy network is constructed by using the pruning mechanism in Multi-Agent Proximal Policy Optimization (MAPPO).

[0124] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-agent driven cloud-edge-end security task multi-objective adaptive scheduling method, characterized in that, Comprising: S1. Deploy an agent on each cloud device, edge device, terminal device, each agent collects Observation data of the corresponding device at the moment , , is a period; S2. Deploy a policy network on each agent, input observation data into the policy network, and output a safe task scheduling policy ; S3. Compute Global privacy entropy at time instant ; S4. Utilizing global privacy entropy The calculation obtains The security task scheduling reward at the moment ; S5. Each agent collects Observation data of the corresponding device at the time ; S6. Constructing a collection , collection , collection ; S7. Deploying a twin network composed of an encoder, a projection head and a prediction head on each agent, inputting the set , the set into the encoder of the twin network for enhancement, inputting the enhanced view into the encoder of the twin network, and outputting the obtained feature , the feature , the feature , the feature , using the feature , the feature , the feature , the feature , the task scheduling reward , and training the twin network to obtain an optimized twin network; S8. inputting the set to an encoder of the optimized twin network, outputting the features , inputting the set to an encoder of the optimized twin network, outputting the features ; S9. Deploy a value network on each agent, input the features , features into the value network respectively, and output the state value , state value respectively; S10. Utilize state value , state value , set Calculate a loss function, train the policy network using the loss function, and obtain an optimized policy network; S11. input the new observation data collected by the intelligent agent into the optimized policy network, and output the final safety task scheduling policy; In step S1 Each device is , , A value of 0 indicates that the device deploying the intelligent agent is a terminal device. A value of 1 indicates that the device deploying the intelligent agent is an edge device. A value of 2 indicates that the device deploying the intelligent agent is a cloud-based device. ,in For the number of terminal devices, For the number of edge devices, For the number of cloud devices, the first One device intelligent agent collection Observational data at time Observational data For tuples, , For the first One device exist Task queue length at any given time For edge devices and cloud devices Set of task queue lengths at any given time For edge devices and cloud devices CPU utilization at any given moment For edge devices and cloud devices Set of memory utilization at any given moment for Time of the first One device The bandwidth combination of edge devices and cloud devices; Step S6 includes the following steps: S6-1. By one observation data at the moment of construction set , ; S6-2. By a set of security task scheduling policies , ; S6-3. By one observation data at the moment of construction set , . 2.The multi-agent driven cloud-edge-end security task multi-objective adaptive scheduling method of claim 1, wherein, Step S2 includes the following steps: S2-1. In the first device agent, deploying a policy network; S2-2. The policy network is composed of a first fully connected layer, a ReLU activation function, a second fully connected layer, a Tanh activation function, a mean output branch, and a standard deviation output branch. S2-3. The observation data is input into the first fully connected layer of the policy network, a ReLU activation function, and output to obtain features ; S2-4. The features are sequentially input into a second fully connected layer of the policy network, a Tanh activation function, and output to obtain features ; S2-5. The mean output branch of the policy network is sequentially composed of a fully connected layer and a Tanh activation function, which inputs the features into the mean output branch and outputs the mean vector ; S2-6. The standard deviation output branch of the policy network is sequentially composed of a fully connected layer and a SoftPlus activation function, which inputs the features into the standard deviation output branch and outputs a standard deviation vector ; S2-7. Based on the mean vector and the standard deviation vector a Gaussian probability distribution is constructed, and the first original policy vector of the device is sampled from the Gaussian probability distribution using the sample() method of the torch.distributions.Normal class of the PyTorch library ; S2-8. Normalizing the original policy vector to obtain the first device security task scheduling policy at time . 3.The multi-agent driven cloud-edge-end security task multi-objective adaptive scheduling method of claim 1, wherein, Step S3 includes the following steps: S3-1. The probability that the i-th terminal device is scheduled to the j-th target device at time t is calculated by the formula ​​​​​​​​​​​​​​​​​​ S3-2. Through the formula The first terminal device At time The information entropy ; S3-3. The average information entropy of the terminal device at time t is calculated by the formula ;​​ S3-4. The normalized information entropy of the terminal device at time point t is calculated by the formula ;​​ S3-5. By formula The first target device The global probability at time The global probability at time The global probability at time , wherein, The number of safety tasks scheduled to the first target device The number of safety tasks scheduled to the first target device The number of safety tasks scheduled to the first target device The number of safety tasks scheduled to the first target device The total number of all scheduled safety tasks S3-6. The global entropy H(t) is calculated by the formula at time t is calculated as ;​ S3-7. By formula The normalized global entropy at time is calculated; S3-8. The global privacy entropy at time is calculated by the formula .​ 4. The multi-agent driven cloud-edge-end security task multi-objective adaptive scheduling method according to claim 3, characterized in that: The reward of the security task scheduling at the time t is calculated by the formula , wherein, is a weight coefficient of the average scheduling delay, is the time when the central controller returns, is the average scheduling delay generated by the security task scheduling of all devices, is a weight coefficient of the average security detection accuracy, is the time when the central controller returns, is the average security detection accuracy generated by the security task scheduling of all devices, is a weight coefficient of the global privacy entropy.​​ 5.The multi-agent driven cloud-edge-end security task multi-objective adaptive scheduling method of claim 1, wherein: In step S5 One device intelligent agent collection Observational data at time Observational data For tuples, , For the first One device exist Task queue length at any given time For edge devices and cloud devices Set of task queue lengths at any given time For edge devices and cloud devices CPU utilization at any given moment For edge devices and cloud devices Set of memory utilization at any given moment for Time of the first One device A collection of bandwidth between edge devices and cloud devices. 6.The multi-agent driven cloud-edge-end security task multi-objective adaptive scheduling method of claim 1, wherein, Step S7 includes the following steps: S7-1. Generate a vector with the same shape as the set using the torch.rand_like() function of the PyTorch library , and set each element value in the vector to a random number in the interval [0, 1). When the element value is less than 10%, set the element value to 0; when the element value is greater than or equal to 10%, set the element value to 1. Multiply the set element by element with the vector to obtain a randomly discarded vector. Use the torch.randint() function of the PyTorch library to randomly generate non-repeating indexes with a 5% probability from the randomly discarded vector, and set the element value corresponding to the index to 0 to obtain an enhanced view . Multiply the vector by the standard deviation to obtain Gaussian noise . Add the Gaussian noise to the set to obtain an enhanced view . S7-2. Generate a vector of the same shape as the set using the torch.rand_like() function of the PyTorch library , and set each element value in the vector to a random number in the interval [0, 1). When the element value is less than 10%, set the element value to 0; when the element value is greater than or equal to 10%, set the element value to 1. Multiply the set element by element with the vector to obtain a randomly dropped vector. Use the torch.randint() function of the PyTorch library to randomly generate non-repeating indexes with a 5% probability from the randomly dropped vector, and set the element value corresponding to the index to 0 to obtain the enhanced view . Multiply the vector by the standard deviation to obtain Gaussian noise, and add the Gaussian noise to the set to obtain the enhanced view . S7-3. The encoder of the twin network is sequentially composed of a first fully connected layer, a first LN layer, a first ReLU activation function, a second fully connected layer, a second LN layer, a second ReLU activation function, and a third fully connected layer, and the enhanced view is input into the encoder, and the output obtains the feature , the enhanced view is input into the encoder, and the output obtains the feature , the enhanced view is input into the encoder, and the output obtains the feature , the enhanced view is input into the encoder, and the output obtains the feature ; S7-4. The projection head of the twin network is sequentially composed of a first full connection layer, a BN layer, a ReLU activation function, and a second full connection layer. The feature is input into the projection head, and the feature is output. The feature is input into the projection head, and the feature is output. The feature is input into the projection head, and the feature is output. S7-5. The prediction head of the twin network is sequentially composed of a first fully connected layer, a BN layer, a ReLU activation function, and a second fully connected layer, and the features are input into the prediction head, and the features are output. The features are input into the prediction head, and the features are output. The features are input into the prediction head, and the features are output. S7-6. Through the formula The base cosine similarity loss is calculated wherein is the negative cosine similarity, is the stop gradient operation; S7-7. Task scheduling reward Normalization is performed to obtain a normalized task scheduling reward ; S7-8. Through the formula The calculation obtains Moment task scheduling reward The weight ; S7-9. The weighted cosine similarity loss function is calculated by the formula The weighted cosine similarity loss function is calculated by the formula The weighted cosine similarity loss function is calculated by the formula The twin network is trained using the gradient descent method to obtain an optimized twin network. 7.The multi-agent driven cloud-edge-end security task multi-objective adaptive scheduling method of claim 1, wherein, Step S9 includes the following steps: S9-1. The value network is composed of a first fully connected layer, a first ReLU activation function, a second fully connected layer, a second ReLU activation function, a third fully connected layer, and a third ReLU activation function in sequence. S9-2. Feature input into the value network, and output the state value ; S9-3. Feature input into the value network, and output the state value ; S9-4. Utilizing state value , state value , safety task scheduling reward computing the timing difference error , obtaining the loss function of the value network by computing the mean square error of the timing difference error , training the value network by using the loss function of the value network through the gradient descent method. 8.The multi-agent driven cloud-edge-end security task multi-objective adaptive scheduling method of claim 7, wherein, Step S10 includes the following steps: S10-1. Based on timing difference error Computing advantage function ; S10-2. Based on the set The importance sampling ratio is calculated in a manner of importance sampling, and the importance sampling ratio is combined with the advantage function The loss function of the policy network is constructed through a clipping mechanism in multi-agent proximal policy optimization.

Images