A method for SDN-MEC security offloading and resource scheduling based on double-agent collaborative decision-making

Abstract

The application discloses an SDN-MEC security offloading and resource scheduling method based on double-agent collaborative decision-making, and belongs to the field of network security and mobile edge computing. In view of the problems of separation design of security detection and computing offloading, high action space dimension and the like in the prior art, the method proposes collaborative decision-making by a control agent and a scheduling agent. First, the attack probability is estimated based on the traffic change rate, and a programmable data plane is used for malicious flow pre-blocking. Then, the control agent outputs the suspicious flow sampling ratio, and the scheduling agent receives the suspicious flow sampling ratio and outputs the security resource allocation vector of each node. Finally, combined with the attack probability bucketing random forest runtime detector, the CPU cycle domain queue evolution model and the multi-objective reward function, the joint optimization of security detection and computing offloading is realized. The method can reduce the time delay and the default rate while ensuring security, improve the resource utilization efficiency, and is suitable for edge scenarios such as industrial internet and vehicle networking.

Description

Technical Field

[0001] This invention relates to the fields of network security, software-defined networking, programmable data plane, mobile edge computing, and reinforcement learning collaborative optimization, and particularly to a method, system, device, and storage medium for SDN-MEC secure offloading and resource scheduling based on dual-agent collaborative decision-making. Background Technology

[0002] With the rapid development of industrial internet, vehicle-to-everything (V2X) networks, private networks in industrial parks, and the convergence of computing and network, more and more latency-sensitive services are being offloaded to the edge for execution. At the same time, network threats such as DDoS attacks, scanning and probing, and malicious traffic spoofing are also beginning to directly affect the edge control plane and edge computing plane, causing the two objectives of "whether to strengthen detection" and "how to ensure low-latency execution" to be strongly coupled on the same time scale.

[0003] The existing technology has the following main problems:

[0004] 1. Existing edge security solutions often separate traffic detection and task offloading into separate processes. The former is handled by fixed thresholds or a single security policy, while the latter is handled by static resource scheduling or local heuristic algorithms, lacking a cross-layer linkage mechanism.

[0005] 2. Existing joint optimization methods typically incorporate traffic sampling, routing, node resource allocation, and task execution into the same joint action space, resulting in excessively high action dimensions, difficulty in training convergence, and complexity in online inference.

[0006] 3. Existing security scheduling methods often use manually set detection rate curves or fixed attack level thresholds, which are difficult to reflect the fluctuations in detector performance as attack intensity changes under real traffic trajectories.

[0007] 4. Existing single-agent or static strategies are difficult to express the causal relationship that "global sampling intensity first affects the node-side input load, and then affects the internal resource allocation of the node", which can easily lead to congestion in both the safety queue and the execution queue.

[0008] 5. Existing methods are insufficient in modeling the coupling relationship between security queues, execution queues, transmission latency, detection latency, execution latency, energy consumption, and attack leakage risk, making it difficult to achieve true security and latency co-optimization under limited computing power constraints.

[0009] Therefore, there is an urgent need for a technical solution that can achieve joint optimization of threat perception, security detection and computation offloading with a lower action dimension under the SDN-MEC collaborative architecture, so as to improve the limitations of existing technologies. Summary of the Invention

[0010] The purpose of this invention is to provide a method, system, device, and storage medium for secure offloading and resource scheduling of SDN-MEC based on dual-agent collaborative decision-making, in order to solve the following technical problems:

[0011] 1. Address the decision-making disconnect caused by the separation of edge security detection and computational offloading design.

[0012] 2. Solve the problems of training instability and deployment complexity caused by single-stage high-dimensional joint action space.

[0013] 3. To address the problem that fixed detection rate modeling is insufficient to reflect the relationship between the actual attack situation and detection performance.

[0014] 4. Address the challenge of simultaneously achieving low latency, low leakage, and high efficient throughput when security queues and execution queues are coupled.

[0015] 5. Address the issue of the lack of clear causal transmission and collaborative control between global detection intensity and node-side resource allocation.

[0016] To achieve the above objectives, the present invention adopts the following technical solution:

[0017] This invention proposes a dual-agent cooperative control framework, wherein:

[0018] 1. The control agent is responsible for outputting the global sampling ratio of suspicious flows. This is used to control the size of suspicious flows entering the edge security deep detection link after the P4 data plane;

[0019] 2. The scheduling agent is responsible for outputting node-level resource allocation vectors. This is used to control the proportion of remaining idle computing power that each MEC node allocates to the security detection queue and the service execution queue.

[0020] In a preferred embodiment, the present invention does not directly incorporate routing decisions into the reinforcement learning joint actions. Instead, it retains the inverse load forwarding logic on the environmental infrastructure side, so that the reinforcement learning actions retain only two truly high-control-value and strongly coupled continuous variable sets, namely... and This design significantly reduces the difficulty of policy learning while preserving the adaptability of network forwarding.

[0021] In another preferred embodiment, the present invention constructs the attack probability by measuring the source-end traffic change rate and the destination-end traffic change rate. Furthermore, by combining malicious flow pre-blocking at layer P4, suspicious flow marking, and edge-side random forest runtime detector, a two-layer security processing link of "coarse filtering + deep detection" is constructed. Notably, the random forest runtime detector does not employ a fixed detection probability; instead, it uses a lookup table of true positive and false positive rates, trained with labeled public trajectories, to vary with attack probability bucketing, thus more realistically describing detection performance.

[0022] In another preferred embodiment, the present invention models the security queue and the execution queue in the CPU cycle domain, drives the service rate update with the proportion of remaining idle computing power, and constructs a reward function together with transmission latency, detection latency, execution latency, attack leakage rate, risk loss, energy consumption and deadline default, thereby achieving unified optimization of security and service quality.

[0023] In another preferred embodiment, the present invention employs a dual-agent proximal policy optimization algorithm, in which both agents output bounded continuous actions based on a Beta distribution, and combine heuristic auxiliary targets, entropy coefficient annealing, behavior constraint coefficient annealing, and learning rate annealing mechanisms to improve stability and deployability during long-term training.

[0024] Compared with the prior art, the present invention has at least the following beneficial effects:

[0025] 1. By adopting a sequential decoupling structure of “controlling agent outputting global suspicious flow sampling ratio + scheduling agent outputting node-level resource partitioning vector”, the complex cross-layer joint control problem is decomposed into a two-level continuous decision problem, which reduces the action space complexity while preserving the coupling relationship.

[0026] 2. By explicitly introducing in the state of the scheduling agent This ensures a clear causal link between the global detection intensity and the node input load and node resource allocation, thereby improving collaborative control capabilities.

[0027] 3. By using the attack probability bucket lookup table mechanism of the random forest runtime detector, the security detection performance can be dynamically calibrated based on public traffic trajectories, making the detection rate curve closer to the real network environment than manual detection.

[0028] 4. By separating the routing strategy from the reinforcement learning action space and adopting an inverse load forwarding mechanism to take over the distribution behavior of the infrastructure side, the multi-node load balancing capability is preserved, while the exponential expansion of the joint action space is avoided.

[0029] 5. The dual-queue evolution model of the CPU cycle domain can simultaneously characterize the resource contention relationship between the security detection link and the computation execution link, making it more suitable for real-time control and resource scheduling in edge scenarios.

[0030] 6. By integrating a multi-objective reward function that considers latency, leakage, risk, energy consumption, and penalties for breach of contract, an explicit trade-off between security and service quality is achieved.

[0031] 7. By integrating coarse-grained detection and dual-agent collaborative decision-making on the P4 switch side with fine-grained detection and task computation on the MEC server side, network security protection and edge task processing can operate collaboratively under a unified control framework.

[0032] 8. This invention is applicable to various edge computing scenarios such as industrial internet, vehicle network, and campus network, and is also easy to integrate with existing SDN controllers, P4 programmable switches and MEC servers, making it highly feasible and worthy of promotion. Attached Figure Description

[0033] Figure 1 This diagram illustrates the steps of the SDN-MEC security offloading and resource scheduling system based on dual-agent collaborative decision-making of the present invention, and is used to show the connection relationship between the SDN controller, P4 programmable switch, MEC server cluster, control agent, scheduling agent and status feedback link.

[0034] Figure 2 This is a diagram of the SDN-MEC security offloading and resource scheduling system based on dual-agent collaborative decision-making of the present invention.

[0035] Figure 3 This is a workflow diagram of the SDN-MEC security offloading and resource scheduling based on dual-agent collaborative decision-making in this invention. It is used to illustrate the overall process of the system from network architecture deployment, P4 rule initialization, dual-agent modeling, attack probability calculation, sampling decision, traffic forwarding, resource scheduling, edge execution to policy update. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the present invention will be further described below in conjunction with the accompanying drawings and embodiments. It should be understood that the following embodiments are only for explaining the present invention and are not intended to limit the scope of protection of the present invention. Equivalent substitutions and conventional improvements made by those skilled in the art without departing from the concept of the present invention should all fall within the scope of protection of the present invention. Unless otherwise defined, the technical or scientific terms used herein should have the ordinary meaning understood by those skilled in the art to which this invention pertains. The terms "comprising" and similar expressions used herein mean that the element or object preceding the word covers the element or object listed following the word and its equivalents, but does not exclude other elements or objects.

[0037] This invention provides a method for secure offloading and resource scheduling of SDN-MEC based on dual-agent collaborative decision-making, comprising the following steps:

[0038] S1. Configure an SDN-MEC three-layer network architecture and deploy an SDN controller, a P4 programmable switch, and a MEC server;

[0039] S2. Initialize the P4 switch detection rules, configure the Count-Min Sketch statistical structure and blacklist / whitelist rules;

[0040] S3. Construct a dual-agent reinforcement learning model, defining the state space, action space, and reward function of the control agent and the scheduling agent;

[0041] S4. Statistically analyze real-time traffic and calculate the attack probability. The control agent then outputs the sampling ratio of suspicious flows after observing the status.

[0042] S5. The P4 switch performs traffic filtering and forwarding based on the detection results and the sampling ratio of suspicious flows.

[0043] S6. The scheduling agent observes the MEC side status and outputs the resource allocation ratio of each MEC server.

[0044] S7. The MEC server performs security checks and task calculations, and returns the execution results and status feedback to the control layer. After completing the checks and calculations, the MEC server only periodically reports the aggregated latency statistics and CPU utilization to the SDN controller through the southbound interface, without transmitting single-task-level queue fine-grained jitter, in order to avoid control plane signaling congestion.

[0045] S8. Calculate the reward value based on latency, security, energy consumption, and queue status, and update the dual-agent policy network.

[0046] S9. Repeat the above steps until the dual-agent strategy converges, and output the target detection suspicious flow sampling ratio and resource allocation ratio for online deployment.

[0047] In step S1, a collaborative working architecture is constructed for the SDN control layer, P4 switching layer, and MEC execution layer. This collaborative working architecture includes at least a network architecture deployment module, a P4 detection and statistics module, a control agent decision-making module, a traffic forwarding module, a scheduling agent decision-making module, an MEC execution feedback module, and a policy update module.

[0048] The overall system architecture diagram of the SDN-MEC security offloading and resource scheduling method based on dual-agent cooperative decision-making provided by this invention is as follows: Figure 1 As shown.

[0049] The workflow framework diagram of the SDN-MEC security offloading and resource scheduling method based on dual-agent collaborative decision-making provided by this invention is as follows: Figure 2 As shown.

[0050] 1) Network Architecture Deployment Module: This module is used to deploy and initialize the SDN-MEC three-layer network environment. The SDN controller is used to collect network status, maintain global control logic, and interact with the dual-agent policy module; the P4 programmable switch is used to perform fast detection, traffic labeling, and forwarding on the data plane side; and the MEC server is used to perform deep security detection and edge task computation. Through the three-layer collaborative architecture, control decisions, traffic processing, and edge execution can operate in a closed loop within a unified system.

[0051] 2) P4 Detection and Statistics Module: This module performs traffic statistics, suspicious flow identification, and pre-filtering control on the P4 switch side. The module uses Count-Min Sketch to approximate the count of source and destination traffic to obtain traffic change trends; simultaneously, it combines blacklist rules to directly block deterministic malicious traffic, combines whitelist rules to quickly allow trusted traffic, and marks remaining suspicious traffic to provide input for subsequent sampling and deep detection.

[0052] 3) Control agent decision-making module: This module is used to output the sampling ratio of suspicious flows based on the global network state. Preferably, the input states of the control agent include at least the attack probability, link bandwidth state, previous time-lapse latency state, previous time-lapse resource allocation state, and the security queue state and execution queue state of each MEC node. The output states of the control agent... This is used to control the proportion of suspicious flows entering the MEC-side deep security detection link, enabling the system to dynamically adjust the detection intensity based on the threat situation.

[0053] In a preferred embodiment, the input state of the control agent includes at least the attack probability, link bandwidth state, previous time delay state, previous time resource allocation state, and the security queue state and execution queue state of each MEC node.

[0054] In a preferred embodiment, attack probability Based on the change in source rate and the change in the rate at the destination end Calculated.

[0055] 4) Traffic Forwarding Module: This module performs traffic forwarding based on the detection results of the P4 switch and the sampling ratio of suspicious flows output by the control agent. For malicious traffic matching the blacklist, the switch directly discards or blocks it; for normal traffic matching the whitelist, the switch directly forwards it to the task execution link; for traffic in the suspicious set, the switch... Sampling is performed, and the sampled suspicious traffic is sent to the MEC security detection link. The unsampled traffic is sent to the normal computing link according to the preset strategy.

[0056] 5) Scheduling agent decision-making module: This module is used to output resource allocation ratios based on the queue status and task load status on the MEC side. The output of the scheduling agent middle, Indicates the first The proportion of remaining idle computing power allocated to each MEC node in the security detection link.

[0057] In a preferred embodiment, the input state of the scheduling agent includes at least the input load, attack probability, link bandwidth, task bit count, task CPU cycle requirement, deadline, and control agent output. The status of the security queue and execution queue of each MEC node.

[0058] 6) MEC Execution Feedback Module: This module performs security checks, task computation, and status feedback on each MEC server side. For suspicious traffic entering the security check link, deep detection can be performed using a random forest detector or an equivalent classification model; edge computing is performed on legitimate traffic, allowed traffic, and task data. Each MEC node calculates the proportion of remaining idle computing power based on the current total load, and further calculates the security check service rate and task execution service rate.

[0059] 7) Policy Update Module: This module is used to calculate the reward value and update the dual-agent policy network based on the system operation results.

[0060] In a preferred embodiment, the reward function comprehensively considers task transmission delay, queuing delay, detection delay, execution delay, attack and leakage risk, energy consumption, queue overflow, and deadline violation penalty, thereby enabling the control agent and the scheduling agent to update collaboratively under a unified optimization objective.

[0061] During steps S3 to S8, a dual-agent reinforcement learning model is constructed. The control agent is used to output the sampling ratio of suspicious flows. The scheduling agent is used to output the resource allocation ratio. .

[0062] In a preferred embodiment, both agents adopt an Actor-Critic structure and output corresponding control quantities in a continuous action distribution, so that the sampling ratio of suspicious flows and the resource allocation ratio can be continuously changed within a preset range.

[0063] During step S4, the system collects real-time traffic statistics, queue length, link bandwidth, task scale, and attack status information. The control agent first makes a global detection strength decision. This pre-decision structure enables... It can directly participate in the construction of subsequent MEC-side input loads and further influence the decision-making state of the scheduling agent.

[0064] When executing step S6, the scheduling agent is based on the included The status information is used to allocate resources to each MEC node, enabling the security detection link and task execution link to dynamically adjust the computing power ratio according to the load status of different nodes, thereby alleviating the problems of single node congestion and global resource mismatch.

[0065] During step S7, the MEC server maintains both the safety queue and the execution queue based on the scheduling results, and feeds back the detection results, task completion results, and status information to the policy update module. This feedback is used both for reward calculation at the current moment and for state construction at the next moment.

[0066] During steps S8 and S9, the system iteratively updates the dual-agent policy network based on reward feedback until a suspicious flow sampling ratio policy and resource scheduling policy that meet the predetermined control objectives are obtained. After training is complete, the control agent and scheduling agent can be deployed in the SDN controller or equivalent control unit, and during online operation, the suspicious flow sampling ratio and resource allocation ratio can be directly output based on the real-time status.

[0067] It should be noted that the above embodiments are merely preferred embodiments of the present invention. Those skilled in the art can make equivalent adjustments to the P4 switch detection rules, dual-agent network structure, edge detection model, queue modeling method, and reward function form without departing from the principles of the present invention. However, it should be understood that all such equivalent adjustments should be considered to fall within the protection scope of the present invention. Furthermore, the present invention described herein can have other embodiments and can be implemented or realized in various ways.

Claims

1. A method for secure offloading and resource scheduling of SDN-MEC based on dual-agent collaborative decision-making, characterized in that, Includes the following steps: Obtain network traffic status, task load status, link bandwidth status, and security queue status and execution queue status of each mobile edge computing node; construct the input status of the control agent and the input status of the scheduling agent. The attack probability at the current moment is estimated based on the source traffic change rate and the destination traffic change rate, and malicious flows are pre-blocked and suspicious flows are marked based on the programmable data plane; The control agent outputs the current sampling ratio of suspicious flows based on the input state of the control agent. This is used to control the sampling scale of suspicious flows entering the edge-side deep security detection link; The As a prerequisite, the environment is injected to determine the actual traffic load entering the edge side. Subsequently, the state containing the actual traffic load is written into the input state of the scheduling agent, which then outputs the resource allocation vector for each mobile edge computing node. ,in This represents the proportion of remaining idle computing power allocated to the security detection queue for the m-th mobile edge computing node; According to the preset forwarding strategy, the tasks to be processed are distributed to each mobile edge computing node. At each mobile edge computing node, the random forest runtime detector is used to perform malicious detection on the sampled suspicious streams, and to perform computation offloading on legitimate services, unsampled suspicious services, and services that are allowed after detection. Instant rewards are calculated based on factors such as security queue length, execution queue length, transmission latency, security detection latency, execution latency, attack leakage rate, attack blocking rate, risk loss, energy consumption normalization value, queue overflow ratio, and deadline default. A dual-agent reinforcement learning algorithm is used to collaboratively train the control agent and the scheduling agent. During the deployment phase, the trained control agent and the scheduling agent jointly output a safe offloading and resource allocation decision for the current network environment.

2. The method according to claim 1, characterized in that, The method for estimating the attack probability is as follows: The preprocessed malicious task count and total task count are mapped to source-end statistics and destination-end statistics, respectively, and the source-end rate change value is calculated. Change in rate at the destination end The attack probability is determined by the following formula. : in, Weighted by the rate of change at the source. This is the normalized upper limit of the source-end rate. This is the upper limit of the normalized rate at the destination.

3. The method according to claim 1, characterized in that, The input state of the control agent includes at least the attack probability, the normalized link bandwidth value, the normalized macroscopic CPU utilization value of each node at the previous time, and the normalized average system latency feedback value at the previous time; the input state of the scheduling agent includes at least the normalized input load value, the attack probability, the normalized link bandwidth value, the normalized task bit quantity value, the normalized task CPU cycle requirement value, the normalized deadline value, and the sampling ratio of suspicious flows at the current time. The CPU utilization of MEC; and the control agent output first. After scheduling the intelligent agent, based on Output .

4. The method according to claim 1, characterized in that, The random forest runtime detector is trained based on public traffic trajectory data with attack labels. After training, it outputs the global true positive rate, global false positive rate, attack probability bucket boundaries, and true positive rate lookup table and false positive rate lookup table corresponding to each attack probability bucket. During runtime, the corresponding detection parameters are retrieved or interpolated from the lookup tables according to the current attack probability to replace the fixed detection probability curve.

5. The method according to claim 1, characterized in that, The forwarding strategy adopts a pre-set reverse load forwarding mechanism or uniform forwarding mechanism on the environment side. The forwarding strategy does not directly participate in the joint action space as a reinforcement learning action variable. The reverse load forwarding mechanism calculates the forwarding weight based on the current security queue load and execution queue load of each mobile edge computing node, and filters out candidate nodes that do not meet the current task deadline constraint, so as to reduce the dimension of the joint action space and retain the load balancing behavior of the network infrastructure side.

6. The method according to claim 1, characterized in that, Both the security queue and the execution queue are modeled within the CPU cycle domain; for the first Each mobile edge computing node first calculates the proportion of remaining idle computing power based on the total queue load, then determines the security detection service rate and execution service rate, and subsequently updates the security queue and execution queue based on the newly arrived workload and service volume.

7. The method according to claim 1, characterized in that, The instant reward adopts a multi-objective weighted form, including at least the average total latency term, energy consumption normalization term, risk loss term, attack leakage rate term, queue overflow term, and deadline default penalty term; wherein, the risk loss term is calculated according to the following formula: in, As a risk penalty coefficient, This represents the effective detection rate at the current moment.

8. The method according to claim 1, characterized in that, The described dual-agent reinforcement learning algorithm is a proximal policy optimization algorithm based on a centralized training and distributed execution (CTDE) architecture. Both the control agent and the scheduling agent adopt an Actor-Critic structure. The Critic network introduces a global state for temporal differential updates during the training phase, while relying only on local states during the execution phase. The action distribution adopts a Beta distribution to ensure the sampling ratio of suspicious flows. and resource allocation vector Each component always lies within a preset continuous closed interval.

9. The method according to claim 8, characterized in that, During training, auxiliary objectives based on state heuristic rules are introduced for both the control agent and the scheduling agent. Combined with policy entropy coefficient annealing, behavior constraint coefficient annealing, actor learning rate annealing, and critic learning rate annealing mechanisms, the dual-agent policy is stabilized during training.

10. The method according to claim 1, characterized in that, Before model deployment, multiple sub-evaluations are performed on multiple training checkpoints, and target checkpoints are selected for online deployment based on a comprehensive evaluation index consisting of effective throughput, deadline satisfaction rate, attack blocking rate, edge detection recall rate, average latency, tail latency, attack leakage rate, and false positive rate.

11. The method according to claim 1, characterized in that, The publicly available traffic trajectory data is generated by preprocessing DDoS public datasets (such as CIC-DDoS2019) and includes at least attack probability, task arrival rate, link bandwidth, task bit count, task CPU cycle requirement, and attack label fields. It is used to simultaneously drive the evolution of the attack situation and the training of the random forest runtime detector.

Images