SCN control forwarding plane dynamic scheduling method and system based on DRQN and time-sensitive compiler

Abstract

The invention discloses an SCN control forwarding plane dynamic scheduling method and system based on a DRQN and a time-sensitive compiler, and belongs to the field of service customization networks. Comprising resource state collection and vector construction; generating a strategy intention; in a control plane, a distributed recursive Q learning DRQN model is adopted, and a resource allocation strategy is generated in real time based on the advanced strategy intention; a time-sensitive flow table compiler is designed, and the time-sensitive flow table compiler converts the advanced strategy intention into an atomic instruction set irrelevant to equipment in real time and supports a time-sensitive network TSN scheduling operation code; and issuing the atomic instruction and updating the flow table. The method can adapt to the rapid change of the network state, and reduces the failure time of the scheduling strategy.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of service customization network (SCN) and deep reinforcement learning (DRL), and mainly relates to an SCN control-forwarding plane dynamic scheduling method and system based on DRQN and time-sensitive compiler. BACKGROUND

[0002] With the rapid development of new network systems (such as computing power networks, TSN, and 5G slice networks), the architecture design of decoupling network control plane and forwarding plane has become a mainstream trend. This paradigm aims to simplify network management through centralized control, but its popularity has also brought new challenges. Traditional SDN scheduling schemes are difficult to adapt to highly dynamic business and network states due to their limitations based on static rules or shallow optimization. In particular, in service customization networks (SCN), there is a lack of effective elastic mapping capability between business-level SLA requirements and underlying physical resource configurations, resulting in the system being unable to respond quickly and accurately when facing resource bottlenecks or sudden loads. This lag seriously affects the real-time performance, reliability, and service quality assurance of the network.

[0003] On the other hand, with the popularity of programmable capabilities of network devices (such as P4 language) and time-sensitive scheduling technologies (such as IEEE802.1Qbv, 802.1Qch) in the industrial field, the controllability and determinism of the network forwarding plane have been greatly enhanced. However, these advanced forwarding plane functions have not been fully utilized, because the upper control plane lacks a mechanism for intelligent reasoning of high-level strategies. Current orchestration and control still highly depend on pre-set rule configurations or manual strategies, and cannot fully exploit the potential of programmable data planes, let alone achieve intelligent and adaptive scheduling for dynamic changes. In view of this, there is an urgent need for a unified scheduling mechanism that can intelligently map SLA, resource state, policy intent, and underlying atomic flow tables, by combining intelligent policy generation based on deep reinforcement learning and automatic time-sensitive flow table compilation, to comprehensively improve the intelligence of the control plane and the controllability of the forwarding plane, and meet the urgent needs of the next generation of high-reliability, low-latency, and dynamically adjustable network architecture. SUMMARY

[0004] The application aims to solve the problems of incomplete resource awareness, delayed policy generation response, lack of time-sensitive policy compilation capability, and long control-forwarding loop delay in existing service customization networks (SCN) in dynamic business scheduling. The application proposes an SCN control-forwarding plane dynamic scheduling method based on distributed recursive Q-learning (DRQN) and time-sensitive flow table compiler.

[0005] Technical solution: To achieve the above-mentioned purpose, the technical solution adopted by the application is:

[0006] A SCN control forwarding plane dynamic scheduling method based on DRQN and time-sensitive compiler, comprising the following steps:

[0007] Step 1, resource state acquisition and vector construction: real-time acquisition of resource states of wireless domain, wired domain and computing domain in multi-domain network, and recording the resource states as a multi-domain resource state vector.

[0008] Step 2, policy intention generation: based on the multi-domain resource state vector, a multi-domain cooperative network operating system CNOS is constructed, which converts service level agreement SLA into resource constraint equation and outputs high-level policy intention.

[0009] Step 3, resource allocation strategy generation: in the control plane, a distributed recursive Q learning DRQN model is used to generate resource allocation strategy in real time based on the high-level policy intention.

[0010] Step 4, policy intention compilation: a time-sensitive flow table compiler is designed, which converts the high-level policy intention into a device-independent atomic instruction set in real time and supports time-sensitive network TSN scheduling operation code.

[0011] Step 5, atomic instruction issuing and flow table updating: the atomic instruction set is issued to the programmable data plane, and the programmable data plane updates its internal pipeline and flow table according to the atomic instruction set.

[0012] Preferably, the resource state acquisition and vector construction method in step 1 is as follows:

[0013] By deploying lightweight awareness agent components in the multi-domain network, the core resource states of the wireless domain, wired domain and computing domain are collected in real time. The resource states include link bandwidth, round-trip delay, CPU usage, GPU memory occupancy and storage input / output operation times. All resource indicators are uniformly packaged into a multi-domain resource state vector in each sampling period

[0014] Preferably, the form of the multi-domain resource state vector is as follows:

[0015]

[0016] Wherein, Bandwidth, delay, CPU, GPU and storage performance indicators of the i-th node, respectively. All indicators will be normalized to the interval.

[0017] Preferably, the method of generating policy intention in step 2 is as follows:

[0018] ​The CNOS multi-domain cooperative network operating system running in the control plane receives resource state vectors. The process involves parsing and transforming the Service Level Agreement (SLA) contained in the upper-layer business requests into a formalized resource constraint model. SLA metrics include maximum latency. Minimum bandwidth This is expressed in a structured form as a resource constraint equation:

[0019]

[0020] in, Representing the resource constraint equation, This indicates the maximum CPU utilization.

[0021] The multi-domain collaborative network operating system (CNOS) uses resource constraint equations to map state vectors, identifies performance bottleneck nodes in the current system, and generates policy intent outputs based on historical trends and current business priorities. The policy intent is represented using a graph structure or tensors and is provided to subsequent decision-making models for perceiving constraints and scheduling objectives.

[0022] Preferred method: The method for generating the resource allocation strategy in step 3 is as follows:

[0023] This implementation is based on the Distributed Recursive Q-Learning (DRQN) model, using the current state vector... With strategic intent The input is fed into a Distributed Recursive Q-Learning (DRQN) model to predict the optimal scheduling action. The Distributed Recursive Q-Learning (DRQN) model utilizes an LSTM structure to enhance its ability to remember temporal state changes, adapting to dynamic fluctuations in the network environment.

[0024] The preferred reward function for the Distributed Recursive Q-Learning (DRQN) model is:

[0025]

[0026] in, This represents the hidden state at time t. Represents the Long Short-Term Memory network. Let represent the multi-domain resource state vector at time t. Indicates the state The expected cumulative reward that can be obtained by taking action 'a'. This indicates the high-level strategic intent at time t. This represents the optimal resource allocation strategy / optimal scheduling action at time t. Indicates that choosing can make The largest operation, represents the state at time t performing an action the instant reward obtained afterwards, represents the weight coefficient, used to balance the relative importance of the four different objectives (SLA achievement, SLA violation, resource cost, stability) in the formula, satisfying . represents the SLA achievement reward, which is a positive incentive. When the scheduling action taken by the DRQN model makes the traffic flow fully satisfy its SLA constraints, the model will obtain a positive score reward. represents the SLA violation penalty, which is a negative penalty. When the traffic flow violates the SLA constraints (such as too high latency or insufficient bandwidth), the model will be punished with a negative score penalty according to the severity of the violation. represents the resource energy efficiency cost, which is a negative cost. It measures the total consumption of CPU, bandwidth, GPU, etc. of the nodes involved after the scheduling strategy is executed. This item is used to encourage the model to find the most "energy-saving" or resource-occupying solution while meeting the SLA. represents the scheduling stability cost, which is a negative cost. It is specifically used to punish high-cost scheduling actions (e.g. causing large-scale traffic rerouting or computing task migration). The purpose of setting this item is to suppress the frequent "oscillation" of the strategy, improve the stability of the entire system, and reduce the policy invalidation time.

[0027] Preferably, the method for policy intent compilation in step 4 is as follows:

[0028] The policy intent is converted by the time-sensitive flow table compiler, which has device independence and time control ability. Its working principle is: receiving the policy intent and converting it into a standard intermediate instruction set, and then compiling it into atomic operation instructions according to the network device type. The operation instruction structure is as follows:

[0029] [OPCODE] [MATCH_FIELDS] [ACTION_SET] [TIME_SLOT]

[0030] Wherein, OPCODE represents an operation code, i.e. indicating the type of instruction, such as ADD_FLOW (adding flow table entry), MOD_FLOW (modifying flow table entry), GATE_CONTROL (gate control scheduling), etc. [ ] represents a symbol for defining and separating various fields, MATCH_FIELDS represents a matching field, i.e. defining the matching rule of the data packet, such as source / desination IP address, port number, etc. ACTION_SET represents an action set, i.e. defining the operation to be performed after the matching is successful, such as FORWARD (forwarding), SET_VLAN_ID (setting VLAN ID), etc. TIME_SLOT represents a time slot, which supports TSN scheduling semantics, encodes the time constraint as a time parameter in the instruction, such as gate_open_time and gate_close_time, and ensures that the key data flow passes in a specific, reserved time slot.

[0031] Preferably, the method for issuing atomic instructions and updating flow table in step 5 is as follows:

[0032] The finally generated atomic instruction set is issued to the data plane device through the SDN controller. The instruction transmission adopts a high priority channel. After receiving the instruction, the device updates its internal flow table, queue priority and scheduling time sequence, and executes in a pipeline structure, supporting concurrent data path processing. The flow table format is as follows:

[0033] flow_id | match_fields | egress_port | gate_open_time | gate_close_time

[0034] Wherein, flow_id represents the unique identifier of the flow, | represents the visual separator between different fields, match_fields represents the matching rule of the data packet (such as source / desination IP address, protocol type, etc.), egress_port represents the specified egress port of the data packet, and gate_open_time and gate_close_time represent the key fields for supporting TSN, which define the valid time window of the flow table entry.

[0035] Another object of the present application is to provide an SCN control forwarding plane dynamic scheduling system based on DRQN and time-sensitive compiler, comprising a resource state acquisition and vector construction unit, a policy intention generation unit, a resource allocation policy generation unit, a policy intention compilation unit, an atomic instruction issuing and flow table updating unit, wherein:

[0036] The resource state acquisition and vector construction unit is used to acquire the resource states of the wireless domain, the wired domain and the computing domain in the multi-domain network in real time, and record the resource states as a multi-domain resource state vector.

[0037] The policy intention generation unit is configured to construct a multi-domain collaborative network operating system (CNOS) based on the multi-domain resource state vector, the multi-domain collaborative network operating system (CNOS) converts a service level agreement (SLA) into a resource constraint equation, and outputs a high-level policy intention.

[0038] The resource allocation strategy generation unit is configured to generate a resource allocation strategy in real time based on the high-level policy intention by using a distributed recursive Q-learning (DRQN) model on a control plane.

[0039] The policy intention compiling unit is configured to save a time-sensitive flow table compiler, the time-sensitive flow table compiler converts the high-level policy intention into a device-independent atomic instruction set in real time, and supports a time-sensitive network (TSN) scheduling operation code.

[0040] The atomic instruction issuing and flow table updating unit is configured to issue the atomic instruction set to a programmable data plane, and the programmable data plane updates an internal pipeline and a flow table according to the atomic instruction set.

[0041] Another object of the present application is to provide a non-transitory computer readable storage medium storing computer instructions, the computer instructions causing the computer to execute the SCN control forwarding plane dynamic scheduling method based on DRQN and time-sensitive compiler.

[0042] Compared with the prior art, the present application has the following beneficial effects:

[0043] The present application is based on the collaborative design of multi-domain perception, intelligent decision-making, time-sensitive compilation and high-speed execution, can adapt to rapid changes in network state, reduce the invalidation time of scheduling strategies, and is widely applicable to data center networks, 5G / 6G slices, industrial internet, vehicle networking and other application scenarios with strict requirements for low delay and high reliability. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 It is a system overall architecture schematic diagram of the present application.

[0045] Figure 2 It is a method flow chart of the present application.

[0046] Figure 3 It is a resource scheduling model structure schematic diagram based on DRQN.

[0047] Figure 4 It is a time-sensitive compiler working principle schematic diagram.

[0048] Figure 5 It is a time slot scheduling schematic diagram of the present application. DETAILED DESCRIPTION

[0049] The present application will be further clarified by the following examples, which should not be construed as limiting the scope of the application. Various modifications of the application in accordance with the teaching of the application will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments without departing from the scope of the application as defined in the appended claims.

[0050] Example 1

[0051] In view of the problems of incomplete resource awareness, slow policy generation response, lack of time-sensitive ability in policy compilation, and long delay in control-forwarding closed loop in the existing service customization network (SCN) in dynamic service scheduling, the embodiment provides an SCN control-forwarding plane dynamic scheduling method based on DRQN and time-sensitive compiler. First, a multi-domain resource awareness framework across wireless domain, wired domain and computing domain is constructed, real-time multi-dimensional resource state information including bandwidth, delay, CPU utilization, GPU memory usage and storage IOPS is collected, and it is packaged as a multi-domain resource state vector to ensure the comprehensiveness and consistency of the scheduling decision input data. Subsequently, in the multi-domain collaborative network operating system (CNOS), the service level agreement (SLA) is formalized as a resource constraint equation, combined with the real-time state vector, and a high-level policy intent is generated to clearly define the scheduling optimization target and ensure the direct relevance of the policy and the service demand. A distributed recursive Q-learning (DRQN) model is introduced in the control plane, which uses its time series modeling capability to associate and analyze the historical state, dynamically generates the optimal resource allocation policy based on the high-level policy intent, and realizes efficient response to complex and variable business scenarios. After the policy generation, a time-sensitive flow table compiler is designed to convert the high-level policy intent into a device-independent atomic instruction set in real time, and supports IEEE 802.1Qbv and other time-sensitive network (TSN) scheduling semantics in the compilation process to ensure that critical business flows can obtain deterministic forwarding time slot allocation and priority protection. Finally, the atomic instruction set obtained by compilation is issued to the programmable data plane through the southbound interface, and the hardware pipeline is used to quickly update the flow table to realize the closed-loop scheduling of the control-forwarding plane. Through the synergistic effect of high-frequency acquisition, intelligent policy generation, time-sensitive compilation and fast execution, the scheduling response delay is significantly reduced, and the SLA achievement rate and resource utilization efficiency are improved.

[0052] As shown in Figures 1-5 , specifically comprising the following steps:

[0053] Step 1, resource state acquisition and vector construction: real-time acquisition of resource states in the wireless domain, wired domain and computing domain in the multi-domain network, and recording the resource states as a multi-domain resource state vector.

[0054] The method for resource status acquisition and vector construction is as follows:

[0055] By deploying lightweight awareness agent components in multi-domain networks, the core resource status of wireless domains (such as Wi-Fi and 5G base stations), wired domains (such as SDN switches and routers), and computing domains (such as edge nodes and computing centers) is collected in real time. The data collection cycle can be as low as 100ms, with a transmission latency of less than 5ms. Resource status includes: link bandwidth (Mbps), round-trip time (RTT, ms), CPU utilization (%), GPU memory utilization (%), and memory I / O operations (IOPS). All resource metrics are displayed in each sampling period. Internally, it is uniformly encapsulated as a multi-domain resource state vector, providing a unified input format for subsequent policy generation.

[0056] The form of a multi-domain resource state vector is:

[0057]

[0058] Where t is the sampling timestamp, and n is the number of monitored nodes. These represent the bandwidth, latency, CPU, GPU, and storage performance metrics of the i-th node, respectively. All metrics will be normalized to a common standard. This process establishes a unified dimensionless index within a given interval, ensuring the convergence stability of subsequent reinforcement learning models. This step constructs a computationally achievable and learnable foundation for system state representation, providing a basis for generating subsequent scheduling strategies.

[0059] Step 2, Policy Intent Generation: Based on the multi-domain resource state vector, a multi-domain collaborative network operating system (CNOS) is constructed. The CNOS will include service level agreements (SLAs) such as maximum latency. Minimum bandwidth Maximum CPU utilization This is transformed into a resource constraint equation, and combined with the current multi-domain resource state vector, a high-level strategy intent that reflects resource bottlenecks, latency sensitivity, and priority requirements is generated. The degree of default at each node is then calculated. The system is based on all The distribution of information generates advanced strategy intent. This intent, represented in a structured format such as JSON or YAML, contains information such as traffic redistribution rules and computation task migration instructions to guide subsequent intelligent decision-making. This strategy intent clarifies the optimization objectives and constraints, providing directional guidance for subsequent intelligent scheduling.

[0060] The method for generating strategy intent is as follows:

[0061] The CNOS multi-domain cooperative network operating system running in the control plane receives resource state vectors. The process involves parsing and transforming the Service Level Agreement (SLA) contained in the upper-layer business requests into a formalized resource constraint model. SLA metrics include maximum latency. (For example Minimum bandwidth (For example Contents such as these are structured and expressed as resource constraint equations:

[0062]

[0063] in, Represent the resource constraint equation. This indicates the maximum CPU utilization.

[0064] The CNOS (Multi-Domain Collaborative Network Operating System) uses resource constraint equations to map state vectors, identify performance bottleneck nodes in the current system, and generate policy intentions based on historical trends and current service priorities. For example, it might require increasing bandwidth weight on node N1 or avoiding high-latency paths. These policy intentions are typically represented using graph structures or tensors and are provided to subsequent decision-making models for understanding constraints and scheduling objectives.

[0065] Step 3, Resource Allocation Strategy Generation: The Distributed Recursive Q-Learning (DRQN) model deployed in the control plane receives the high-level policy intent and real-time state vector as input. It uses LSTM units to extract time-series features, evaluates and outputs the Q-values ​​of each available resource scheduling action, thereby selecting the optimal action for the current state. The reward function comprehensively considers SLA achievement rate and resource utilization rate to enhance the global optimality of the scheduling strategy. Based on the aforementioned high-level policy intent, the DRQN ​​model generates resource allocation strategies in real time.

[0066] like Figure 3 As shown, the method for generating resource allocation strategies is as follows:

[0067] This implementation is based on the Distributed Recursive Q-Learning (DRQN) model, using the current state vector... With strategic intent The input is fed into a distributed recursive Q-learning (DRQN) model to predict the optimal scheduling action. The Distributed Recursive Q-Learning (DRQN) model utilizes an LSTM structure to enhance its ability to remember state changes over time, adapting to dynamic fluctuations in the network environment. Its core computational process includes:

[0068]

[0069]

[0070]

[0071] Simultaneously, construct the following reward function:

[0072]

[0073] in, This represents the hidden state at time t. Represents the Long Short-Term Memory network. Let represent the multi-domain resource state vector at time t. Indicates the state The expected cumulative reward that can be obtained by taking action 'a'. This indicates the high-level strategic intent at time t. This represents the optimal resource allocation strategy / optimal scheduling action at time t. Indicates that choosing can make The largest operation, This indicates the state of the DRQN ​​model at time t. Next action The instant reward obtained afterward This represents the weighting coefficient, used to balance the relative importance of the four different objectives (SLA achievement, SLA default, resource costs, and stability) in the formula, satisfying... . This indicates that achieving the SLA (Solution Level Agreement) is a reward, serving as a positive incentive. When the DRQN ​​model takes scheduling actions... The model receives a positive reward when the business flow fully satisfies its SLA constraints. This represents a negative penalty for SLA breach. When a service flow violates an SLA constraint (such as excessive latency or insufficient bandwidth), the model will impose a negative penalty based on the severity of the breach. This represents resource efficiency cost, which is a negative cost. It measures the total consumption of resources such as CPU, bandwidth, and GPU on the nodes involved after executing the scheduling strategy. This item is used to incentivize the model to find the most "energy-efficient" or lowest resource-consuming solution while meeting the SLA. This represents the scheduling stability cost, which is a negative cost. It is specifically used to penalize high-cost scheduling actions (e.g., those causing large-scale traffic rerouting or computation task migration). The purpose of setting this is to suppress frequent policy "oscillations," improve the overall system stability, and reduce policy failure time. This model supports distributed training and online updates, and can adapt to the dynamic resource allocation needs of various business scenarios.

[0074] The Distributed Recursive Q-Learning (DRQN) model captures multi-domain resource state vectors using LSTM units. timing characteristics, and combined with high-level policy intent to calculate the Q value of each optional action in the current state The final optimal action with the maximum Q value is selected Its reward function Comprehensive consideration of SLA achievement reward and scheduling cost reward And introduce the weight coefficient , Balance to ensure the global optimality of the policy.

[0075] Step 4, policy intent compilation: design a time-sensitive flow table compiler that converts the high-level policy intent into a device-independent atomic instruction set in real time, and supports TSN scheduling operation codes. The resource allocation strategy output by DRQN is converted into a device-independent intermediate representation (Policy IR), and further compiled into an atomic instruction set containing TSN scheduling operation codes, flow matching fields, and time slot control parameters, ensuring uniform execution on programmable switching devices from different manufacturers.

[0076] As shown in Figure 4 , the method of policy intent compilation is as follows:

[0077] The policy intent is converted by the time-sensitive flow table compiler, which has device independence and time control capability. Its working principle is: receiving the policy intent and converting it into a standard intermediate instruction set, and then compiling it into an atomic operation instruction according to the network device type. The operation instruction structure is as follows:

[0078] [OPCODE] [MATCH_FIELDS] [ACTION_SET] [TIME_SLOT]

[0079] Where OPCODE represents the operation code, that is, the type of instruction, such as ADD_FLOW (add flow table entry), MOD_FLOW (modify flow table entry), GATE_CONTROL (gate scheduling), etc. [ ] represents the symbol that delimits and separates each field, MATCH_FIELDS represents the matching field, that is, the matching rule of the data packet, such as source / destination IP address, port number, etc. ACTION_SET represents the action set, that is, the operation to be performed after the matching is successful, such as FORWARD (forward), SET_VLAN_ID (set VLAN ID), etc. TIME_SLOT represents the time slot, which supports TSN scheduling semantics, and encodes the time constraints as time parameters in the instruction, such as gate_open_time and gate_close_time, to ensure that critical data flows pass within a specific, reserved time slot.

[0080] For example:

[0081] SET_QUEUE_PRIORITY dst_ip=192.168.1.1 prio=5 slot=7

[0082] The time-sensitive flow table compiler converts the resource allocation strategy output by the DRQN into a device-independent intermediate representation (IR). The IR is then compiled into an atomic instruction set, each instruction in the instruction set having a clear format, including OPCODE (operation code), MATCH_FIELDS (match field), ACTION_SET (action set), and TIME_SLOT (time slot) key fields. In particular, by supporting the TIME_SLOT field, seamless support for TSN scheduling semantics is achieved.

[0083] The time-sensitive flow table compiler supports the TSN (Time-Sensitive Network) standard, can add a time control field to the instructions, and achieve deterministic execution of scheduling actions. In addition, it can also extend support for P4 language description, adapt to modern programmable switch environments, and improve system portability and running efficiency.

[0084] Step 5, atomic instruction issuing and flow table updating: the atomic instruction set is issued to the programmable data plane, and the programmable data plane updates its internal pipeline and flow table according to the atomic instruction set. The compiled atomic instruction set is issued to the programmable data plane device through a southbound interface protocol (such as P4Runtime or extended OpenFlow), and the data plane completes flow table updating based on the pipeline mechanism at the microsecond level, and performs priority scheduling and time window control according to the policy, thereby forming an efficient and low-latency control-forwarding closed-loop scheduling mechanism.

[0085] As shown in Figure 5 , the method of atomic instruction issuing and flow table updating is as follows:

[0086] The finally generated atomic instruction set is issued to the data plane device, such as a P4 switch or a forwarding device supporting OpenFlow, by the SDN controller. Instruction transmission adopts a high-priority channel to ensure that instruction execution delay control is within 50 microseconds. After receiving the instructions, the device updates its internal flow table, queue priority, and scheduling timing, and executes based on the pipeline structure, supporting concurrent data path processing. The flow table format is as follows:

[0087] flow_id | match_fields | egress_port | gate_open_time | gate_close_time

[0088] Wherein, flow_id represents a unique identifier of the flow, | represents a visual separator between different fields, match_fields represents a matching rule defined for the data packet (such as source / destination IP address, protocol type, etc.), egress_port represents an egress port specified for the data packet, and gate_open_time and gate_close_time represent key fields for supporting TSN, defining a valid time window for the flow table entry.

[0089] The mechanism combines time window configuration (such as GCL) to achieve strict bandwidth guarantee and delay control for real-time services. In a high concurrency scenario, the controller can issue flow tables through a batch strategy to avoid configuration drift. This step ensures accurate mapping between control strategies and actual forwarding behaviors, achieving closed-loop scheduling control for the SCN system.

[0090] The atomic instruction set is issued to the programmable data plane through a high-priority channel and completes flow table updating through atomic operations. The data plane uses the updated flow table for fast matching and action execution based on a hardware pipeline mechanism. For service flows that require time-sensitive guarantees, the device updates its gate control list (GCL) according to the TIME_SLOT parameter in the instruction to ensure forwarding within a specific, reserved time slot, thereby achieving deterministic scheduling.

[0091] Embodiment 2

[0092] This embodiment provides an SCN control forwarding plane dynamic scheduling method based on DRQN and time-sensitive compiler, as shown in Figure 1 、 2 , including the following steps:

[0093] Step 1, combined with Figure 1 , a multi-domain resource state acquisition and vector construction technical architecture. In a multi-domain network environment, including wireless domain, wired domain and computing domain as three core parts. In the wireless domain, resource states include base station channel bandwidth utilization, air interface transmission delay, wireless link packet loss rate, etc. In the wired domain, resource states include link bandwidth occupation, node forwarding delay, queue length, etc. In the computing domain, resource states include CPU utilization, GPU memory occupation, IOPS, etc. Resource acquisition agents are deployed in each domain, which can obtain real-time indicators through driver interfaces (such as eBPF hook), operating system APIs (such as / proc / stat) or hardware SDKs (such as CUDA API).

[0094] The wireless domain includes: link bandwidth utilization (unit: Mbps, defined as the ratio of current used bandwidth to total link bandwidth), average end-to-end delay (unit: ms, measured by round-trip time RTT), packet loss rate (unit: %, calculated by the ratio of lost packet number to total sent packet number).

[0095] Wired domain collection: link bandwidth utilization , switch node queue length (unit: packet number or byte number, read through switch queue monitoring interface), end-to-end latency .

[0096] Computing domain collection: central processing unit (CPU) utilization (unit: %, calculated by the ratio of CPU usage time to total running time), graphics processing unit (GPU) memory occupancy (unit: %, calculated by the ratio of GPU memory usage to total GPU memory capacity), storage IOPS utilization (unit: times / second, counted through storage device driver).

[0097] All indicators are normalized, and the normalization formula is as follows:

[0098]

[0099] Wherein: is the original indicator value collected. is the historical minimum value of the indicator. is the historical maximum value of the indicator. is the normalized dimensionless indicator value (range 0~1).

[0100] After mapping to the interval, a multi-domain resource state vector is formed , which is in the form of:

[0101]

[0102] Wherein, t represents the sampling timestamp, and n represents the number of monitored nodes. The vector serves as the input for subsequent policy calculation and scheduling decision. The collection period can be as low as 100ms, and the transmission delay is less than 5ms, thereby ensuring the real-time and feasibility of scheduling.

[0103] Step 2, the present application realizes policy intent generation in a multi-domain collaborative network operating system (CNOS). The system first receives the multi-domain resource state vector generated in step 1 , and converts the service level agreement (SLA) into a formal resource constraint equation.

[0104] The SLA indicators include: maximum latency (unit: ms, such as ), minimum bandwidth (unit: Mbps, e.g. ), maximum CPU occupancy (unit: %, e.g. ), etc. Structured as:

[0105]

[0106] Calculate the default level of each node:

[0107]

[0108] Wherein: is the comprehensive default value of node i (dimensionless). , , are the weight coefficients of delay, bandwidth, and CPU utilization (satisfying ). represents only taking positive values when defaulting, and 0 in normal state.

[0109] CNOS generates high-level policy intent according to the distribution of all , which is expressed in a structured format (such as JSON or YAML format), and contains information such as traffic redistribution rules and computing task migration instructions, facilitating subsequent compilation and delivery.

[0110] The intent field includes:

[0111] Objective (Objective): Indicates the main purpose of this scheduling, such as "reducing latency", "increasing bandwidth", "reducing packet loss", or a combination of multiple goals. For example, minimize_latency (minimize latency), maximize_throughput (maximize throughput).

[0112] Scope (Scope): Clearly defines the objects to be affected by the scheduling, which can be specific devices, links, or a certain type of data flow (identified by the five-tuple information of the flow <src, dst, proto, port, DSCP>, such as source IP, destination IP, protocol, port number, priority, or slice ID), or even the entire slice network.

[0113] Constraints (Constraints): Defines hard conditions that must be met during scheduling, such as: delay must be ≤10ms, bandwidth must be ≥50Mbps, CPU occupancy cannot exceed 80%, etc. These constraints are directly extracted from SLA and converted into a form that can be calculated by the system.

[0114] Hints: Optional suggestions for the scheduler, such as: Should rerouting be allowed? Should computational tasks be migrated to other nodes? Can TSN (Time-Sensitive Network) scheduling be prioritized?

[0115] The advantage of generating advanced policy intents is that the system can flexibly switch scheduling strategies in different scenarios. For example, it can quickly prioritize ensuring the delay of critical services during sudden congestion and improve the overall throughput during idle times, without the need for manual configuration of complex network parameters.

[0116] Step 3, as follows Figure 3 As shown, a distributed recursive Q-learning (DRQN) model is run within the control plane of the multi-domain cooperative network operating system (CNOS), aiming to transmit high-level policy intentions. With real-time resource state vector This transforms into the optimal resource allocation strategy.

[0117] The DRQN ​​model mainly consists of two parts: a recurrent neural network (RNN) for processing time-series state data, typically employing a Long Short-Term Memory (LSTM) network or a gated recurrent unit (GRU). This invention uses an LSTM network to capture the influence of historical states on the current decision. The other is a deep Q-network (DQN) for policy evaluation and action selection.

[0118] Its hidden state The calculation formula is as follows:

[0119]

[0120] in: This represents the hidden state at time t. This represents the multi-domain resource state vector at time t. This indicates the hidden state at the previous moment. This represents the set of parameters for an LSTM network.

[0121] LSTM layer output The input is then fed into the DQN layer and combined with high-level policy intent. Q is used to calculate the Q-value of taking all possible actions in the current state. Indicates the state The expected cumulative reward that can be obtained by taking action 'a' is calculated using the following formula:

[0122]

[0123] in: This represents a deep neural network function. This represents the set of parameters for the DQN network.

[0124] The agent selects the optimal action based on the Q-value. During the deployment phase, the agent adopts a greedy strategy, choosing the action with the highest Q-value as the optimal resource allocation strategy. .

[0125]

[0126] in: This represents the optimal resource allocation strategy at time t.

[0127] reward function The design comprehensively considers both the satisfaction of business SLAs and the economy of resource utilization. The calculation formula is as follows:

[0128]

[0129] in, This represents the hidden state at time t. Represents the Long Short-Term Memory network. Let represent the multi-domain resource state vector at time t. Indicates the state The expected cumulative reward that can be obtained by taking action 'a'. This indicates the high-level strategic intent at time t. This represents the optimal resource allocation strategy / optimal scheduling action at time t. Indicates that choosing can make The largest operation, This indicates the state of the DRQN ​​model at time t. Next action The instant reward obtained afterward This represents the weighting coefficient, used to balance the relative importance of the four different objectives (SLA achievement, SLA default, resource costs, and stability) in the formula, satisfying... . This indicates that achieving the SLA (Solution Level Agreement) is a reward, serving as a positive incentive. When the DRQN ​​model takes scheduling actions... The model receives a positive reward when the business flow fully satisfies its SLA constraints. This represents a negative penalty for SLA breach. When a service flow violates an SLA constraint (such as excessive latency or insufficient bandwidth), the model will impose a negative penalty based on the severity of the breach. This represents resource efficiency cost, which is a negative cost. It measures the total consumption of resources such as CPU, bandwidth, and GPU on the nodes involved after executing the scheduling strategy. This item is used to incentivize the model to find the most "energy-efficient" or lowest resource-consuming solution while meeting the SLA. Represents the cost of scheduling stability, which is a negative cost. It is specifically used to penalize those high-cost scheduling actions (e.g., causing massive traffic rerouting or compute task migration). The purpose of setting this is to suppress the frequent "oscillation" of the policy, improve the stability of the entire system, and reduce the policy failure time.

[0130] Step 4, the present application designs a time-sensitive flow table compiler, as shown in Figure 4 , which converts high-level policy intentions into device-independent atomic instruction sets in real time and supports time-sensitive network (TSN) scheduling operation codes. This compiler ensures that high-level policy intentions can be accurately and efficiently converted into bottom-layer device-executable, time-constrained atomic instructions, providing a key bridge for the close coordination of the control-forwarding plane.

[0131] The time-sensitive flow table compiler receives the high-level policy intentions generated in step 2 and the resource allocation strategy generated in step 3 , first extracts all relevant parameters through the front-end parser. Then, these parameters are converted into a unified intermediate representation (IR), which is designed to decouple the policy from the underlying hardware. Finally, the back-end code generator compiles the IR into an atomic instruction set that the target data plane device (such as a P4 switch) can directly recognize and execute, according to the device type.

[0132] The structure of the atomic instruction set is as follows:

[0133] [OPCODE] [MATCH_FIELDS] [ACTION_SET] [TIME_SLOT]

[0134] Where:

[0135] OPCODE (Operation Code): Indicates the type of instruction, such as ADD_FLOW (add flow table entry), MOD_FLOW (modify flow table entry), GATE_CONTROL (gate control scheduling), etc.

[0136] MATCH_FIELDS (Matching Fields): Defines the matching rules of the data packet, such as source / destination IP address, port number, etc.

[0137] ACTION_SET (Action Set): Defines the operations to be performed after a successful match, such as FORWARD (forward), SET_VLAN_ID (set VLAN ID), etc.

[0138] TIME_SLOT (Time Slot): This is the unique feature of the time-sensitive compiler of the present invention. This field supports TSN scheduling semantics, encoding time constraints as time parameters in instructions such as gate_open_time and gate_close_time, ensuring that critical data flows pass within specific, reserved time slots.

[0139] Step 5, as shown in Figure 5 , the atomic instruction set is issued to the programmable data plane, which updates its internal pipeline and flow table according to the atomic instruction set, realizing the close cooperation of the control-forwarding plane.

[0140] The compiled atomic instruction set is issued from the CNOS control plane to the data plane device through an efficient southbound interface (such as the P4Runtime protocol). Instruction transmission is given the highest priority to reduce transmission queuing delay. After the data plane device receives the instruction, its control unit immediately parses the instruction and translates it into read-write operations on the high-speed TCAM or SRAM, completing the addition, modification, or deletion of flow table entries at the microsecond level.

[0141] The update and application process of the flow table is as follows:

[0142] After the data plane device receives the atomic instruction set, its control unit (Control Plane Agent) will immediately parse the instruction. For example, a MOD_FLOW instruction will be parsed as a specific modification operation for a certain flow table.

[0143] The flow table in the programmable data plane is the core of dynamic forwarding and scheduling. The flow table format used in the present invention is as follows:

[0144] flow_id | match_fields | egress_port | gate_open_time | gate_close_time

[0145] Wherein:

[0146] flow_id: It is the unique identifier of the flow.

[0147] match_fields: Defines the matching rules of the data packet (such as source / destination IP address, protocol type, etc.).

[0148] egress_port: Specifies the egress port of the data packet.

[0149] gate_open_time and gate_close_time: They are the key fields of the present invention for supporting TSN, which define the valid time window of the flow table entry.

[0150] The mathematical expression of the flow table update is as follows:

[0151]

[0152] Wherein: represents the updated flow table. represents the current flow table state. InstructionSet represents the issued atomic instruction set.

[0153] The updated flow table is applied to the relevant stage of the data packet processing pipeline in real time. The data packet is matched and action is executed in the pipeline, and the process is as follows:

[0154]

[0155] Wherein: represents that the data packet is processed at the jth stage of the pipeline. represents that the header field H is used for matching at the jth stage. represents the action executed after successful matching.

[0156] In particular, for service flows that require time-sensitive guarantees, the atomic instructions issued by the control plane update the gate_open_time and gate_close_time fields in the flow table. For example, an instruction may set the gate_open_time of a specific high-priority flow to and the gate_close_time to , which means that the flow can only pass through the switch within this time window , thus achieving time-deterministic scheduling and avoiding queuing delays for regular flows.

[0157] The end-to-end response delay of the entire closed loop, from resource state collection to flow table update taking effect, is controlled within milliseconds or even microseconds, which enables the system to adapt to changing network conditions with extremely high real-time performance, meeting the core requirements of the next generation of high-reliability, low-latency networks.

[0158] Embodiment 3

[0159] The embodiment provides an SCN control forwarding plane dynamic scheduling system based on DRQN and time-sensitive compiler, comprising a resource state collection and vector construction unit, a policy intention generation unit, a resource allocation strategy generation unit, a policy intention compilation unit, an atomic instruction issuing and flow table updating unit, wherein:

[0160] The resource state collection and vector construction unit is used to collect the resource states of the wireless domain, the wired domain and the computing domain in the multi-domain network in real time, and record the resource states as a multi-domain resource state vector.

[0161] The policy intention generation unit is configured to construct a multi-domain collaborative network operation system (CNOS) based on the multi-domain resource state vector, the multi-domain collaborative network operation system (CNOS) converting a service level agreement (SLA) into a resource constraint equation and outputting a high-level policy intention.

[0162] The resource allocation policy generation unit is configured to generate a resource allocation policy in real time based on the high-level policy intention by using a distributed recursive Q-learning (DRQN) model on a control plane.

[0163] The policy intention compiling unit is configured to save a time-sensitive flow table compiler, the time-sensitive flow table compiler converting the high-level policy intention into a device-independent atomic instruction set in real time and supporting a time-sensitive network (TSN) scheduling operation code.

[0164] The atomic instruction issuing and flow table updating unit is configured to issue the atomic instruction set to a programmable data plane, the programmable data plane updating its internal pipeline and flow table according to the atomic instruction set.

[0165] The embodiment provides a non-transitory computer readable storage medium storing computer instructions, the computer instructions causing the computer to execute the SCN control forwarding plane dynamic scheduling method based on the DRQN and time-sensitive compiler.

[0166] The above only describes the preferred embodiments of the present application, and it should be noted that those skilled in the art can make several improvements and refinements without departing from the principles of the present application, and these improvements and refinements should also be considered within the protection scope of the present application.

Claims

1. A dynamic scheduling method for the SCN control forwarding plane based on DRQN ​​and a time-sensitive compiler, characterized in that, Includes the following steps: Step 1, Resource Status Acquisition and Vector Construction: Real-time acquisition of resource status in the wireless domain, wired domain, and computing domain of the multi-domain network, and recording the resource status as a multi-domain resource status vector; Step 2, Policy Intent Generation: Based on the multi-domain resource state vector, a multi-domain collaborative network operating system (CNOS) is constructed. The CNOS converts the Service Level Agreement (SLA) into resource constraint equations and outputs high-level policy intents. Step 3, Resource allocation strategy generation: In the control plane, a distributed recursive Q-learning DRQN ​​model is used to generate resource allocation strategies in real time based on the high-level strategy intent; Step 4, Policy Intent Compilation: Design a time-sensitive flow table compiler that translates the high-level policy intent into a device-independent atomic instruction set in real time and supports Time-Sensitive Network (TSN) scheduling opcodes. Step 5, Atomic instruction issuance and flow table update: The atomic instruction set is issued to the programmable data plane, and the programmable data plane updates its internal pipelines and flow tables according to the atomic instruction set.

2. The SCN control forwarding plane dynamic scheduling method based on DRQN ​​and a time-sensitive compiler according to claim 1, characterized in that: The method for resource status acquisition and vector construction in step 1 is as follows: By deploying lightweight awareness agent components in multi-domain networks, the core resource status of the wireless domain, wired domain, and computing domain is collected in real time. The resource status includes link bandwidth, round-trip latency, CPU utilization, GPU memory utilization, and number of storage input / output operations. All resource metrics in each sampling period Internally, it is uniformly encapsulated as a multi-domain resource state vector.

3. The SCN control forwarding plane dynamic scheduling method based on DRQN ​​and a time-sensitive compiler according to claim 1, characterized in that: The form of a multi-domain resource state vector is: in, These represent the bandwidth, latency, CPU, GPU, and storage performance metrics of the i-th node, respectively; all metrics will be normalized to be consistent with... Interval.

4. The SCN control forwarding plane dynamic scheduling method based on DRQN ​​and a time-sensitive compiler according to claim 1, characterized in that: The method for generating the strategy intent in step 2 is as follows: The CNOS multi-domain cooperative network operating system running in the control plane receives resource state vectors. The process involves parsing and transforming the Service Level Agreement (SLA) contained in the upper-layer business requests into a formalized resource constraint model; the SLA metrics include maximum latency. Minimum bandwidth This is expressed in a structured form as a resource constraint equation: in, Represent the resource constraint equation. Indicates the maximum CPU utilization; The multi-domain collaborative network operating system CNOS uses resource constraint equations to map state vectors, identify performance bottleneck nodes in the current system, and generate policy intent outputs based on historical trends and current business priorities. The policy intent is represented by a graph structure or tensor and is used to provide subsequent decision-making models with the ability to perceive constraints and schedule objectives.

5. The SCN control forwarding plane dynamic scheduling method based on DRQN ​​and a time-sensitive compiler according to claim 1, characterized in that: The method for generating the resource allocation strategy in step 3 is as follows: This implementation is based on the Distributed Recursive Q-Learning (DRQN) model, using the current state vector... With strategic intent The input is fed into a distributed recursive Q-learning (DRQN) model to predict the optimal scheduling action. ; The Distributed Recursive Q-Learning (DRQN) model utilizes an LSTM structure to enhance the ability to remember temporal state changes and adapt to dynamic fluctuations in the network environment.

6. The SCN control forwarding plane dynamic scheduling method based on DRQN ​​and a time-sensitive compiler according to claim 1, characterized in that: The reward function for the Distributed Recursive Q-learning DRQN ​​model is: in, This represents the hidden state at time t. Represents the Long Short-Term Memory network. Let represent the multi-domain resource state vector at time t. Indicates the state The expected cumulative reward that can be obtained by taking action 'a'. This indicates the high-level strategic intent at time t. This represents the optimal resource allocation strategy / optimal scheduling action at time t. Indicates that choosing can make The largest operation, This indicates the state of the DRQN ​​model at time t. Next action The instant reward obtained afterward Indicates the weighting coefficient. Indicates that the SLA has been achieved, This indicates the penalty for breach of the SLA. Indicates resource energy efficiency cost, This represents the cost of scheduling stability.

7. The SCN control forwarding plane dynamic scheduling method based on DRQN ​​and a time-sensitive compiler according to claim 1, characterized in that: The method for compiling the strategy intent in step 4 is as follows: The policy intent is transformed by the time-sensitive flow table compiler, which possesses device independence and timeliness control capabilities. Its working principle is as follows: it receives the policy intent, transforms it into a standard intermediate instruction set, and then compiles it into atomic operation instructions based on the network device type. The operation instruction structure is as follows: [OPCODE] [MATCH_FIELDS] [ACTION_SET] [TIME_SLOT] Wherein, OPCODE represents the opcode, [ ] represents the symbol that delimits and separates each field, MATCH_FIELDS represents the matching field, ACTION_SET represents the action set, and TIME_SLOT represents the time slot.

8. The SCN control forwarding plane dynamic scheduling method based on DRQN ​​and a time-sensitive compiler according to claim 1, characterized in that: The methods for issuing atomic instructions and updating the flow table in step 5 are as follows: The final generated atomic instruction set is distributed to the data plane devices via the SDN controller; instruction transmission uses a high-priority channel; after receiving the instruction, the device updates its internal flow table, queue priority, and scheduling sequence; the execution method is based on a pipeline structure and supports concurrent data path processing; the flow table format is as follows: flow_id | match_fields | egress_port | gate_open_time | gate_close_time Here, flow_id represents the unique identifier of the flow, | represents the visual separator between different fields, match_fields represents the matching rules for packets, egress_port represents the egress port of the packet, and gate_open_time and gate_close_time represent key fields used to support TSN, defining the effective time window of the flow table entry.

9. A scheduling system based on the SCN control forwarding plane dynamic scheduling method based on DRQN ​​and a time-sensitive compiler as described in claim 1, characterized in that: It includes a resource status acquisition and vector construction unit, a policy intent generation unit, a resource allocation policy generation unit, a policy intent compilation unit, and an atomic instruction issuance and flow table update unit, wherein: The resource status acquisition and vector construction unit is used to acquire the resource status of the wireless domain, wired domain and computing domain in the multi-domain network in real time, and record the resource status as a multi-domain resource status vector. The policy intent generation unit is used to construct a multi-domain collaborative network operating system (CNOS) based on the multi-domain resource state vector. The CNOS converts the Service Level Agreement (SLA) into resource constraint equations and outputs advanced policy intents. The resource allocation strategy generation unit is used to generate resource allocation strategies in real time in the control plane using a distributed recursive Q-learning DRQN ​​model based on the high-level strategy intent. The policy intent compilation unit is used to store the time-sensitive flow table compiler, which converts the high-level policy intent into a device-independent atomic instruction set in real time and supports Time-Sensitive Network (TSN) scheduling opcodes. The atomic instruction issuance and flow table update unit is used to issue the atomic instruction set to the programmable data plane, and the programmable data plane updates its internal pipelines and flow tables according to the atomic instruction set.

10. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions that cause the computer to execute the SCN control forwarding plane dynamic scheduling method based on DRQN ​​and a time-sensitive compiler as described in any one of claims 1 to 8.

Images