TSN scheduling optimization method and device based on flow sensing autonomous learning, equipment and medium

Abstract

The invention discloses a TSN scheduling optimization method and device based on flow sensing autonomous learning, equipment and a medium. The method comprises the following steps: deploying a lightweight flow detection module in a switch or a router, and after a controller receives a data request, automatically identifying a newly arrived unknown service flow by using the lightweight flow detection module, and judging whether the newly arrived unknown service flow is a periodic TT flow or an unpredictable burst flow; the controller performs classification management on the identified service flow types, collects topological information and flow requirements of the whole network and issues the topological information and the flow requirements to the terminal nodes through a network management interface; the controller constructs an intelligent queue scheduling task based on the collected network topology information and traffic demand and converts the task into a Markov decision process MDP, network resources, queue states and priorities are used as a state space, a scheduling strategy is used as an action space, a reward function is designed in combination with throughput and delay indexes, and an intelligent queue scheduling task is obtained. Driving a dynamic environment through real-time data and training a DRL model; an enhanced queue scheduling mechanism Pro-CQF is adopted, different priority labels are configured according to classified flow types, and then mixed flow scheduling is carried out; the controller periodically collects time delay, packet loss rate and end-to-end transmission delay indexes and feeds back the indexes to the DRL model, and a scheduling strategy is updated online. According to the method, the traffic sensing and scheduling efficiency is greatly improved in a network environment in which multiple service flows coexist and end-side equipment functions are different, and the reliability and the expandability of the TSN in industrial Internet of Things, edge computing and other high-real-time application scenes are remarkably enhanced.

Description

Technical Field

[0001] This invention belongs to the field of traffic classification and scheduling optimization technology related to Time-Sensitive Networking (TSN) in communication technology. In particular, it is a method for identifying multiple types of network traffic and coordinating the scheduling of time-triggered (TT) flows and burst flows in complex mixed service flow scenarios by means of self-learning technology. Background Technology

[0002] Time-Sensitive Networking (TSN) is a new suite of Ethernet enhancement standards developed by the IEEE 802.1 working group. Its core objective is to provide deterministic quality-of-service (QoS) guarantees for heterogeneous service scenarios through network architecture innovation. TSN employs time-aware shapers, frame preemption mechanisms, and flow reservation protocols to achieve end-to-end resource allocation, ensuring that flow latency, bandwidth, and timing are guaranteed. By introducing time synchronization and traffic scheduling mechanisms, it enables data packets to meet requirements such as low latency, low jitter, and low packet loss during network transmission. Against the backdrop of the accelerated evolution of the Industrial Internet and intelligent systems, and with the deep application of interconnected networks in areas such as telemedicine, autonomous driving systems, and digital twins, new technology paradigms with high computing power dependence and strong real-time response characteristics are constantly emerging. This has led to heterogeneous and scalable network characteristics, posing multi-dimensional technical challenges to TSN's deterministic transmission and low latency.

[0003] In addition to traditional industrial control nodes and various sensors, advanced terminal devices with deep learning or inference capabilities are constantly emerging in the current network. These terminal devices generate various service flows in the network, including time-triggered (TT) flows with strict time synchronization and periodic requirements, and short-lived, unpredictable, non-periodic burst flows. TT flows have the highest priority in TSNs and are suitable for real-time, high-reliability scenarios such as industrial control. Burst flows, on the other hand, have uncertain and unpredictable transmission characteristics and are typically transmitted as low-priority flows when network resources allow. How to ensure low-latency, high-reliability, and scalable transmission of these mixed flows is a crucial problem that needs to be solved in current network research and applications.

[0004] In the process of large-scale deployment of the Industrial Internet, with the exponential growth of network topology complexity and the continuous evolution of heterogeneous terminal ecosystems, TT flows need deterministic priority guarantees, while bursty flows are often difficult to manage through simple static configuration or fixed rules. Traditional Time-Sensitive Network (TSN) architectures face multi-dimensional technical challenges. Furthermore, the limited computing and storage resources of many terminal nodes make it difficult to deploy overly complex traffic identification methods. At the same time, manually maintaining and updating traffic rules is time-consuming and error-prone when facing dynamic topologies and changing business requirements. Therefore, achieving accurate and automated traffic classification at the network level remains a challenge.

[0005] TSN, as a core technology of deterministic networking, provides deterministic transmission capabilities for local area networks (LANs) by relying on high-precision clock synchronization and bounded delay guarantee mechanisms. However, in scenarios with large-scale networks and diverse service flows, current typical scheduling mechanisms (such as CQF and CSQF) are optimized for the periodic characteristics of TT traffic and lack the ability to dynamically adapt to non-periodic burst traffic. The CQF model was initially designed primarily for deterministic guarantees of TT traffic and lacks dynamic adaptability when dealing with burst traffic. Due to the characteristics of burst traffic, such as random arrival and high peak traffic, without proper scheduling and resource management, burst traffic may compete with TT traffic for bandwidth and queuing resources, leading to high queuing latency and packet loss rate, thereby affecting the overall real-time performance and stability of the network.

[0006] In summary, the existing technology has the following problems:

[0007] (1) TSN adaptation challenges caused by the differentiation of end-side devices in TSN local area scenarios. High-performance devices can adapt to complex and diverse business flows and provide accurate traffic management with strong hardware support and good upgradeability. However, the limited hardware performance and bandwidth bottlenecks of Internet of Things (IoT) and video terminals make it impossible to efficiently perform complex traffic classification tasks. The heterogeneity of end-side devices brings severe challenges to the network traffic perception and management capabilities. At the same time, compared with network upgrades, hardware updates of terminal devices face greater difficulties due to multiple constraints such as power consumption, size and cost. (2) The scheduling algorithm faces complexity problems, mainly due to its limitations in three aspects: insufficient unified coordination mechanism, blind spots in state perception, and lack of timeliness in the global perspective. Existing scheduling algorithms are difficult to effectively integrate and utilize network resources, and there are blind spots in the perception of network state, leading to decision bias. In addition, the lack of real-time response capability makes the network inefficient when handling burst traffic, affecting stability. Its reliance on fixed configuration and local decision-making and lack of dynamic adaptability reflect the limitations of traditional static scheduling algorithms. (3) Performance conflict problem of CQF (Circular Queuing and Forwarding) model in hybrid flow joint scheduling. The CQF model faces performance conflicts in hybrid streaming scenarios. Its support for bursty flows is insufficient, especially on terminal devices, where limited flow awareness and scheduling flexibility lead to uneven resource allocation and difficulties in balancing high reliability and low latency. When handling bursty flows, the CQF model exacerbates TT (transmission-time) latency by extending the reception duration, causing critical performance conflicts. Designing effective scheduling algorithms in hybrid streaming environments to ensure high reliability while maintaining low latency is a crucial challenge that the CQF model urgently needs to address. Summary of the Invention

[0008] This invention aims to address the technical challenge of efficiently scheduling time-triggered (TT) flows with strict latency protection requirements and burst flows with high latency tolerance in mixed service flow scenarios with limited and dynamically changing network resources, when end-side devices lack TSN support. It proposes a TSN scheduling optimization method based on flow-aware classification.

[0009] The technical solution to achieve the purpose of this invention is: a TSN scheduling optimization method based on flow-aware autonomous learning, comprising the following steps:

[0010] Step 1: Deploy a lightweight traffic detection module in the switch or router. After the controller receives a data request, it uses the lightweight traffic detection module to automatically identify the newly arrived unknown service flow and determine whether it is a periodic TT flow or an unpredictable burst flow.

[0011] Step 2: The controller classifies and manages the identified service flow types, collects the topology information and traffic requirements of the entire network, and distributes them to the terminal nodes through the network management interface.

[0012] Step 3: The controller constructs an intelligent queue scheduling task based on the collected network topology information and traffic demand, and transforms it into a Markov Decision Process (MDP). The state space is network resources, queue status and priority, and the action space is scheduling policy. The reward function is designed by combining throughput and latency indicators. The dynamic environment is driven by real-time data and the DRL model is trained.

[0013] Step 4: The enhanced queue scheduling mechanism Pro-CQF is adopted, and different priority labels are configured according to the classified traffic types before mixed flow scheduling is performed.

[0014] Step 5: The controller periodically collects latency, packet loss rate, and end-to-end transmission delay metrics and feeds them back to the DRL model to update the scheduling strategy online.

[0015] Further, the specific implementation of step 1 is as follows: when servers need to exchange data, the sending end initiates a data forwarding request to the controller of the control layer through the API protocol; after receiving the data request, the controller uses the lightweight traffic detection module deployed in the switch or router to automatically identify the newly arrived service flow, determine whether it is a periodic time-triggered TT flow or an unpredictable burst flow, and save the identification result together with the relevant feature parameters.

[0016] Furthermore, the specific implementation of determining whether a newly arrived service flow is a periodic time-triggered (TT) flow or an unpredictable burst flow is as follows:

[0017] A lightweight traffic detection module based on a convolutional neural network (CNN) is constructed to extract the time interval sequence T = {t1, t2, ..., t} of the traffic flow. n}, Data packet length distribution L={l1,l2,...,l n} and periodic characteristic index C;

[0018] After preprocessing the traffic, it is classified according to a preset classification function: Where T represents transmission time, L is data packet length, and C is burst index. Combined with the θc threshold, the traffic type is determined to be either TT flow or unpredictable burst flow.

[0019] Furthermore, step 2 is specifically implemented as follows:

[0020] While periodically sampling network information, the controller marks TT flows as high priority and controls their latency and packet loss; burst flows are recorded for their potential bandwidth requirements and flow type; subsequently, the controller initiates an intelligent queue scheduling process based on the current network congestion status and historical traffic statistics to ensure that different types of service flows receive appropriate Quality of Service (QoS) guarantees.

[0021] Furthermore, the specific implementation of the controller's classification and management of the identified service flow types is as follows:

[0022] If it is determined to be a TT flow, then it is marked as a high-priority category P. TT Set the maximum allowable delay threshold and zero packet loss target If it is a burst of traffic, record its maximum bandwidth requirement B. peak and priority label P burst Subsequently, the controller determines the network congestion index I based on the current network congestion level. cong Based on the values ​​∈[0,1] and historical traffic statistics, the queue scheduling process is initiated to ensure that different types of business flows receive differentiated quality of service guarantees.

[0023] Furthermore, in step 3, when intelligent queue scheduling of the current flow is required, the controller collects real-time queue length, priority mapping and packet loss rate information on the switch or router, and constructs the corresponding state matrix as the environment input of the deep reinforcement learning DRL model.

[0024] Specifically, when intelligent queue scheduling of the current business flow is required, the controller obtains the real-time queue length Q from the router. i Priority Mapping Table M p and packet loss rate L i Information is collected, and the following state matrix is ​​constructed:

[0025]

[0026] This serves as the environment input for the deep reinforcement learning (DRL) model; each row represents the queue length, packet loss rate, and priority level of each service flow at the current moment; the DRL model learns policies and makes action decisions based on this state space, outputting the scheduling action 'a' for the next moment. t .

[0027] Further, step 4 is implemented as follows: After considering the priority requirements of both TT and burst flows, the DRL outputs the optimal queue allocation and priority adjustment scheme based on constraints; an enhanced queue scheduling mechanism, Pro-CQF, is adopted, with a three-queue structure at the switching node and a three-queue structure {Q0,Q1,2} at the network node; the controller dynamically configures the gating time windows w0, w1, w2 and priority weights α0, α1, α2 of each queue through scheduling instructions generated by the DRL, and sends them to the network devices through the interface, driving them to adjust the buffer size and priority mapping update rules in real time; then the DRL dynamically allocates the time gating and priority of the queues; the controller sends the DRL decision results to the network layer in the form of scheduling instructions, and the network devices adjust the gating time, priority, and buffer space of the corresponding queues according to the instructions, ensuring that different service flows achieve differentiated resource scheduling at the queue level.

[0028] A TSN scheduling optimization device based on flow-aware autonomous learning includes:

[0029] Lightweight traffic detection module; deployed in switches or routers, the deployed lightweight detection module automatically identifies periodic time-triggered TT flows and unpredictable burst flows;

[0030] Classification Management Module: The controller classifies and manages the identified service flow types, collects the topology information and traffic requirements of the entire network, and distributes them to the terminal nodes through the network management interface;

[0031] Construct a transformation module; the controller constructs an intelligent queue scheduling task based on the collected network topology information and traffic demand, and transforms it into a Markov Decision Process (MDP). The state space is based on network resources, queue status and priority, and the action space is based on the scheduling strategy. The reward function is designed by combining throughput and latency indicators. The dynamic environment is driven by real-time data and the DRL model is trained.

[0032] The scheduling module employs the enhanced queue scheduling mechanism Pro-CQF, which configures different priority labels based on the classified traffic types before performing mixed flow scheduling.

[0033] Update module: The controller periodically collects latency, packet loss rate, and end-to-end transmission delay metrics and feeds them back to the DRL model to update the scheduling strategy online.

[0034] Compared with the prior art, the present invention has the following significant advantages: (1) While meeting the deterministic transmission requirements of TT streams, the present invention effectively copes with traffic peaks in the network. By deploying a lightweight traffic detection module in a resource-constrained network environment and combining it with a dynamic queue scheduling strategy for adaptive optimization, the invention effectively improves the adaptability of heterogeneous networks to diverse business scenarios and achieves more efficient network resource management and business traffic processing capabilities. (2) The invention enhances the overall schedulability and load balancing capabilities of the network. The dynamic scheduling algorithm based on self-learning and automatic optimization can achieve optimal resource allocation in complex network environments by sensing traffic changes in real time, accurately predicting service demands, and making adaptive adjustments. This significantly improves network performance and system stability, and provides technical support for building an efficient and flexible network architecture. (3) Significantly improves burst flow transmission performance. Compared with the traditional scheduling mechanism based on TT flow, this invention adopts the technical measure of deploying self-learning service flow detection in switches or routers. Compared with the scheduling mechanism based on TT flow, this invention adopts the method of deploying self-learning service flow detection in the network, and combined with the enhanced Pro-CQF three-queue structure and intelligent strategy, it realizes accurate identification and dynamic scheduling of burst flow, thereby greatly reducing the frame loss rate and end-to-end latency of burst flow. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of an enhanced feasibility time-sensitive network centralized control architecture according to an embodiment of the present invention.

[0036] Figure 2 This is a schematic diagram of the time-sensitive networking enhanced Pro-CQF scheduling mechanism according to an embodiment of the present invention.

[0037] Figure 3 This is a schematic diagram illustrating a CNN-based traffic classification method for unknown service flows according to an embodiment of the present invention.

[0038] Figure 4 This is a flowchart illustrating the classification and scheduling communication process for end-to-end enhanced feasibility in an embodiment of the present invention. Detailed Implementation

[0039] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0040] Example 1:

[0041] See Figure 1 This diagram illustrates an enhanced time-sensitive network (TSN) centralized control architecture. It primarily improves TSN's adaptability in complex environments by deploying new traffic flow detection modules in switches or routers and optimizing traffic awareness and management on the centralized network controller (CNC). This method is widely applicable to industrial internet, smart grids, automotive control networks, and 5G low-latency applications.

[0042] Step 1: End-user equipment (such as automation control equipment in industrial control systems or sensors in smart grids) reports service requirements to the Deterministic Service Manager (CUC) via API protocols, initiating a user configuration request. The Deterministic Service Manager parses the service requirements and extracts the Quality of Service (QoS) requirements, including key parameters such as latency, bandwidth, and jitter. Subsequently, the CUC transmits these requirements to the Central Network Controller (CNC) through the User Network Interface (UNI) for network configuration and traffic scheduling.

[0043] Step 2: As the core control node of the TSN network, the CNC is responsible for unified management and scheduling. It collects network topology information, traffic requirements, and performance metrics (such as end-to-end latency, packet loss rate, jitter, etc.) and calculates the optimal traffic scheduling scheme based on optimization algorithms (such as Time-Triggered Scheduling (TT), Credit-Shaped Broadcasting (CBS), and preemption mechanisms). After calculation, the CNC generates a global traffic scheduling scheme and distributes it to switches and routers through the network management interface to ensure minimal data transmission latency and efficient utilization of network resources.

[0044] Step 3: The CNC sends traffic scheduling policies to switches and routers, ensuring that each device forwards data along the optimized path. To cope with network changes in complex environments, the CNC dynamically adjusts routing policies based on network status, device load, traffic demand, and topology. For example, when network congestion occurs, the CNC can dynamically adjust traffic paths based on path recalculation mechanisms to reduce end-to-end latency and improve data transmission reliability.

[0045] Step 4: TSN switches and routers strictly execute the scheduling plan issued by the CNC, using a time-based gating mechanism (GateControlList, GCL) or a frame preemption-based approach to ensure that critical service flows (such as industrial control signals and smart grid data) can be transmitted within strict time constraints. Through optimized latency calculation and resource allocation mechanisms, low-latency, high-reliability end-to-end data transmission is achieved, ensuring effective scheduling of data flows of different priorities within the TSN network.

[0046] Step 5: The CNC continuously monitors the data flow transmission status within the TSN network and calculates the end-to-end data forwarding latency. Intra-Domain Latency refers to the transmission delay of data flows within the same network domain, while Inter-Switch Latency represents the transmission time of data between different switches. By aggregating this latency information, the CNC optimizes the overall scheduling strategy and dynamically adjusts the configuration when network congestion or device load changes, ensuring optimal transmission performance of service flows.

[0047] Example 2:

[0048] To make the technical details of steps 4 and 5 in Example 1 clearer, Figure 2 This section provides a more detailed and systematic introduction to the scene and planar flow aspects. The enhanced Pro-CQF model achieves efficient coordinated transmission of time-triggered streams (TT streams) and burst streams in Time-Sensitive Networks (TSNs) through dynamic priority tags and enhanced queue scheduling mechanisms. Its core flow is as follows:

[0049] Configure three priority queues ((Q_0, Q_1, ..., Q_{K-1})) on each switch port, arranged in descending order of priority, with (Q0) being the highest priority queue. Each time interval (T)... ξ Within a given queue, only the highest-priority queue is allowed to open its transmit gate (Tx-gate) to send data; other queues can only receive data through their receive gates (Rx-gate). Queue priorities are determined by dynamically assigned priority tags (ψ). j The value determines the priority; a smaller value indicates a higher priority. For example, within a time interval (T0), if the priority label (ψ0 = 0) of (Q0) is 0, then its Tx-gate is enabled, and the Rx-gates of other queues are enabled to ensure that data transmission is conflict-free.

[0050] TT stream scheduling always has the highest priority. When a TT stream enters the network, its priority label is fixed at (ψ). r =0), directly mapped to queue (Q0). In each time interval, the TT stream preferentially occupies the sending window, ensuring deterministic low latency for end-to-end transmission. If non-critical TT streams exist in the network (such as tasks that allow for some delay), their priority label can be dynamically downgraded to (ψ). r =1), the right to send is given to a higher priority task, while still using the formula Limit its maximum latency to prevent performance from spiraling out of control.

[0051] The scheduling of burst flows achieves a balance between flexibility and reliability through dynamic priority adjustment. In the initial phase, burst flows select lower priority labels (ψ). r =2), mapped to queue Q2, to avoid competition for the sending window with the high-priority TT stream. During the dead time T d Within this context, sudden flow must meet certain constraints. Frame loss is prevented by expanding the receive window. For example, when a burst stream is sent from host H1 to the first-hop switch S1, (ψx = 2) is selected to ensure high reliability; once the first hop is passed, subsequent switches (such as (S2) to (S4)) raise the priority label of the burst stream to (ψx = 2). r=1), which maps it to queue Q1, thereby reducing queuing delay and achieving fast forwarding. This dynamic adjustment strategy avoids conflicts in the initial stage and optimizes latency in subsequent transmissions.

[0052] The hybrid flow scheduling theorem provides strict guarantees for end-to-end performance. For any flow (f) i Its maximum delay is constrained to be ). Where N is the number of hops. The priority label represents the nth hop; jitter is limited to... Within. For example, a burst flow with 4 hops, if the first hop is chosen as (ψ) r =2), subsequent jump selection (ψ) r =1), and its end-to-end delay upper limit is (2T+3×1T+T). d =5T+T d Meanwhile, the jitter is controlled within ((1+1)T=2T) to ensure real-time requirements.

[0053] In practical applications, when the host (H1) sends both TT streams and burst streams simultaneously, the TT stream occupies the (Q0) queue for the entire duration, using a fixed priority label (ψ). r =0) achieves deterministic transmission (e.g., end-to-end delay of (3T)); bursty flow is achieved through dynamic priority adjustment, with the first hop using (Q2)((ψ)). r =2)) Avoid conflict, then switch to (Q1)((ψ) r =1)) Accelerate forwarding, and the final latency is (2T+3T=5T). If the network load is low, the TT flow can be temporarily downgraded to Q1 to free up Q0 resources for other burst flows, while mathematical theorems still ensure that its latency does not exceed the constraint value after downgrading.

[0054] The enhanced Pro-CQF model achieves seamless coordination of TT (Time-to-Time) and bursty flows in complex traffic scenarios through dynamic allocation of priority labels, precise control of queue gates, and constraints on mathematical performance boundaries. Its advantages include: replacing traditional time offsets with priorities to avoid strict time synchronization dependencies; dynamically adjusting responses to network state changes; and providing quantifiable performance guarantees for both types of traffic. This model significantly improves the flexibility and reliability of TSN (Time-to-Speed ​​Network) in high-real-time scenarios such as industrial automation and edge computing.

[0055] Example 3:

[0056] To make the technical details of step 2 in Example 1 clearer Figure 3 This section provides a more detailed and systematic introduction to the optimization algorithm for the traffic-aware component. The core algorithm flow of the efficient classification and labeling method for unknown traffic flows in Time-Sensitive Networks is as follows:

[0057] Step 1: Definition and Objectives of TSN Traffic Classification Problem

[0058] This invention requires classifying traffic into TT flows and burst flows in a TSN (Traffic Streaming Network) based on network traffic characteristics. Given a set of traffic data... Each data sample x i It is a feature representation of traffic, and its corresponding label y i The flow characteristics are 0 (TT flow) or 1 (burst flow). We hope to use a convolutional neural network (CNN) to automatically extract useful spatiotemporal information from these flow characteristics and then make classification decisions.

[0059] The goal of the mathematical model is to learn a function mapping from traffic features using a CNN:

[0060] f θ :R n →0,1 where θ is the parameter of the CNN model and n is the feature dimension of the traffic data.

[0061] Input representation: The traffic data in TSN is represented as a two-dimensional matrix, where each row of the matrix represents a traffic feature at a certain moment in the time series.

[0062] In this invention, the output of the CNN model can be further mapped to the aforementioned classification function to achieve the final determination of the traffic type. Specifically, the CNN model outputs two probability values ​​in the last layer through the Softmax function: PTT and PBurst, which represent the probability that the traffic belongs to a time-triggered flow (TT flow) and a burst flow, respectively.

[0063] This invention uses the class confidence score output by the CNN as a weight parameter in the classification function, defining the following mapping relationship:

[0064]

[0065] Where T represents the average time interval of the traffic; L represents the average length of the data packets; C represents the traffic burstiness index, calculated from the fluctuation rate of the number of data packets arriving per unit time; θ c It is a set periodic threshold used to determine whether there is a stable period; α and β are empirical coefficients that are adjusted according to the statistical characteristics of the training set.

[0066] Step 2: Traffic Data Representation and Input Features

[0067] Each traffic sample x i This can be represented as a time series feature set. Assume each traffic data point contains (T) time points, and the feature dimension of each time point is (D) (e.g., time interval, packet size, traffic rate, etc.). Therefore, each traffic data point can be represented as a (T×D) matrix:

[0068] Where, x i，j ∈R D Let represent the features of the (i)th traffic sample at time point (j). This matrix will be used as the input to the CNN.

[0069] CNN architecture: Extracts spatiotemporal features of traffic data through convolutional layers, reduces feature dimensionality through pooling layers, and performs final classification through fully connected layers.

[0070] Step 3: CNN Network Structure and Convolution Operations

[0071] The core of a convolutional neural network is the convolutional layer and the pooling layer, which are designed to automatically extract high-order spatiotemporal features from traffic data.

[0072] Let the input of the Lth layer be X(l)∈RT. l ×D l That is, the feature map of that layer. The convolution operation uses a convolution kernel (...). It is k l (This refers to the kernel size) and the bias term b l To calculate the output of this layer:

[0073] X(l+1)=f(Conv(X(l), W) l )+b l Wherein, Conv(X(l), W l ) represents the input X(l) and the convolution kernel W. l The convolution operation yields a feature map, where f(·) is the activation function (such as ReLU), and b l This is the bias term for this layer.

[0074] Convolution operations are performed by sliding the convolution kernel W. l Extracting local spatiotemporal features from traffic data. This process is performed through convolution across the time dimension T and the feature dimension D, thereby effectively capturing temporal and spatial patterns in the traffic.

[0075] Step 4: Pooling Layer

[0076] Pooling operations are commonly used to downsample feature maps, reducing computational complexity. Assuming max pooling is used, the output X of the pooling layer... (l+1) It can be represented as:

[0077] X(l+1)=Pool(X(l))

[0078] Pooling operations allow you to choose the size of the pooling window as needed. Pooling operations can preserve important spatiotemporal features while reducing the risk of overfitting.

[0079] Step 5: Fully Connected Layer

[0080] The features extracted by the convolution and pooling layers are flattened and fed into the fully connected layer. Let the output of this layer be X(f)∈RM, where (M) is the number of neurons in the fully connected layer. The output of the fully connected layer:

[0081] y=σ(W(f)X(f)+b (f) )

[0082] Where W(f)∈RM×N is the weight matrix, b (f) σ is the bias, and σ(·) is the activation function (such as Sigmoid or Softmax). The output (y) is the prediction result of whether the flow belongs to TT flow or burst flow.

[0083] Step 6: Loss Function: Optimize the model using the cross-entropy loss function.

[0084] loss function

[0085] To train the CNN model, we use the cross-entropy loss function. Given the true label of a sample (y... i ) and model predictions The cross-entropy loss function is defined as:

[0086]

[0087] in, This is the predicted probability output by the CNN model. The loss function is optimized, and the CNN model is trained to minimize the classification error.

[0088] Step 7: Optimization method: Update the model parameters and optimize the loss function using gradient descent.

[0089] Optimization Algorithm

[0090] To minimize the loss function L(θ), we can use gradient descent (such as stochastic gradient descent, SGD) to update the network parameters. The formula for parameter update is: Where η is the learning rate. It is the gradient of the loss function with respect to the model parameters.

[0091] Step 8: During training, use techniques such as Batch Normalization and Dropout to accelerate training and reduce overfitting.

[0092] Evaluation metrics: The model performance is evaluated using metrics such as accuracy, precision, recall, and F1 score to ensure the reliability of traffic classification.

[0093] Step 9: Model Evaluation and Performance Metrics

[0094] During training and testing, we need to evaluate the model's performance. Commonly used evaluation metrics include:

[0095] Accuracy: The proportion of samples correctly classified by the model.

[0096]

[0097] Precision: The proportion of samples that the model predicts as positive (burst flow) but are actually positive.

[0098]

[0099] Recall: The proportion of samples that are actually positive that the model predicts to be positive.

[0100]

[0101] F1 score: the harmonic mean of precision and recall.

[0102]

[0103] Example 4:

[0104] This invention addresses the scheduling challenges in scenarios where the endpoint does not support Time-Sensitive Networking (TSN) and where mixed traffic flows occur. It proposes a centralized self-learning TSN technology based on a learnable network model. This method automatically detects various traffic flow types from non-TSN nodes by deploying new traffic flow detection modules in switches or routers, and optimizes traffic awareness and management on a centralized network controller (CNC), thereby improving the availability and adaptability of TSN in complex network environments.

[0105] Specifically, this invention is based on a learnable network framework with two main layers: (1) a service flow detection and feature extraction layer, which uses self-learning algorithms such as CNN to automatically identify and classify the diverse service flows (including time-triggered TT flows and unpredictable burst flows) generated by terminal devices. It can accurately perceive and manage the service flows in the network without requiring full support of the TSN protocol on the terminal side. The controller collects real-time network status information of each service flow, including queue length, packet loss rate and priority level, and organizes them into an N×3 state matrix S, where N is the number of currently active service flows. This state matrix serves as the input environment state space of the deep reinforcement learning DRL model and is used for policy learning and action decision-making. Its construction form is as follows:

[0106]

[0107] Among them, Q iL represents the queue length (in packets) of the i-th service flow at the current moment. i

[0108] P_i represents the packet loss rate of the i-th service flow (in %), and P_i represents the priority label of the i-th service flow (the smaller the value, the higher the priority). The controller updates this matrix once per cycle and feeds it back to the DRL model to support online optimization and dynamic adjustment of the scheduling strategy. This structured state representation not only helps the model quickly capture the trend of network state changes, but also provides a clear input basis for the generation of subsequent scheduling strategies.

[0109] (2) Intelligent Dynamic Resource Scheduling Layer: This layer proposes an enhanced queue scheduling method, Pro-CQF. This method extends the original CQF model by introducing a three-queue structure and combining deep reinforcement learning (DRL) for queue priority allocation. This strategy can provide differentiated services for different traffic types under the constraints of end-to-end low latency and high reliability requirements, thereby effectively reducing queuing delays and frame loss rates in bursty flows while maintaining the deterministic transmission characteristics of TT flows. The method includes the following steps.

[0110] Step 1: Deploy flow-aware modules in network devices (such as switches or routers) to detect and classify various service flow types. These modules can automatically identify service flow types without requiring full TSN protocol support on the endpoints. The flow-aware modules extract service flow characteristics by monitoring network traffic in real time and send these characteristics to a centralized network controller (CNC) for further processing.

[0111] Step 2: The CNC utilizes a CNN self-learning algorithm to automatically identify and classify traffic flows. The CNN algorithm analyzes the characteristics of traffic flows, classifying them into time-triggered flows (TT flows) or unpredictable burst flows. This classification process does not require end-side devices to support the TSN protocol, thereby improving network flexibility and adaptability.

[0112] Step 3: Based on the classification results, the CNC employs an enhanced queue scheduling mechanism, Pro-CQF, to schedule the mixed streams. Pro-CQF introduces a three-queue structure and combines deep reinforcement learning (DRL) for queue priority allocation, ensuring differentiated services for different types of traffic. This scheduling method effectively reduces queuing latency and frame drop rates for bursty streams while maintaining the deterministic transmission characteristics of TT streams.

[0113] Step 4: Based on traffic classification information, the CNC calculates the optimized network configuration and sends these configurations to other devices in the network via the CUC (Cooperative User Controller). These configurations include differentiated services allocated to different types of traffic to ensure optimal network performance. The CUC is responsible for translating user requirements into network configuration information and feeding this information back to the CNC for further optimization.

[0114] Step 5: The network device schedules and forwards service flows based on the received optimized configuration. For known service flows from TSN nodes, the network device prioritizes processing them to ensure quality of service. In this way, the network achieves efficient traffic management and optimization, ensuring end-to-end quality of service.

[0115] Through the above steps, the network can achieve efficient traffic classification, optimization, and scheduling, ensuring the quality of service (QoS) for different types of traffic. Ultimately, the network can provide bounded and low end-to-end latency, meeting users' network performance requirements. This invention significantly improves traffic awareness and scheduling efficiency in network environments with multiple service flows and diverse end-side device functions, and significantly enhances the reliability and scalability of TSN in industrial IoT, edge computing, and other high real-time application scenarios. Compared with existing technologies, this invention can accurately identify complex and diverse service flow types without relying on complete end-side support for the TSN protocol, and intelligently schedules queue resources through DRL technology, balancing network load balancing and end-to-end latency. Experimental results show that it achieves superior performance in key indicators such as burst flow frame drop rate and transmission latency.

Claims

1. A TSN scheduling optimization method based on flow-aware autonomous learning, characterized in that, Includes the following steps: Step 1: Deploy a lightweight traffic detection module in the switch or router. After the controller receives a data request, it uses the lightweight traffic detection module to automatically identify the newly arrived unknown service flow and determine whether it is a periodic TT flow or an unpredictable burst flow. Step 2: The controller classifies and manages the identified service flow types, collects the topology information and traffic requirements of the entire network, and distributes them to the terminal nodes through the network management interface. Step 3: The controller constructs an intelligent queue scheduling task based on the collected network topology information and traffic demand, and transforms it into a Markov Decision Process (MDP). The state space is network resources, queue status and priority, and the action space is scheduling policy. The reward function is designed by combining throughput and latency indicators. The dynamic environment is driven by real-time data and the DRL model is trained. Step 4: The enhanced queue scheduling mechanism Pro-CQF is adopted, and different priority labels are configured according to the classified traffic types before mixed flow scheduling is performed. Step 5: The controller periodically collects latency, packet loss rate, and end-to-end transmission delay metrics and feeds them back to the DRL model to update the scheduling strategy online.

2. The TSN scheduling optimization method based on flow-aware autonomous learning according to claim 1, characterized in that, The specific implementation of step 1 is as follows: When servers need to exchange data, the sending end initiates a data forwarding request to the controller of the control layer through the API protocol; after receiving the data request, the controller uses the lightweight traffic detection module deployed in the switch or router to automatically identify the newly arrived service flow, determine whether it is a periodic time-triggered TT flow or an unpredictable burst flow, and save the identification result and relevant feature parameters together.

3. The TSN scheduling optimization method based on flow-aware autonomous learning according to claim 2, characterized in that, The specific implementation of determining whether a newly arrived service flow is a periodic time-triggered (TT) flow or an unpredictable burst flow is as follows: A lightweight traffic detection module based on a convolutional neural network (CNN) is constructed to extract the time interval sequence T = {t1, t2, ..., t} of the traffic flow. n }, Data packet length distribution L={l1,l2,...,l n } and periodic characteristic index C; After preprocessing the traffic, it is classified according to a preset classification function: Where T represents transmission time, L is data packet length, and C is burst index. Combined with the θc threshold, the traffic type is determined to be either TT flow or unpredictable burst flow.

4. The TSN scheduling optimization method based on flow-aware autonomous learning according to claim 1, characterized in that, The specific implementation of step 2 is as follows: While periodically sampling network information, the controller marks TT flows as high priority and controls their latency and packet loss; burst flows, on the other hand, record their potential bandwidth requirements and flow type. Subsequently, the controller initiates an intelligent queue scheduling process based on the current network congestion status and historical traffic statistics to ensure that different types of service flows receive appropriate Quality of Service (QoS) guarantees.

5. The TSN scheduling optimization method based on flow-aware autonomous learning according to claim 4, characterized in that, The controller's specific implementation for classifying and managing the identified service flow types is as follows: If it is determined to be a TT flow, then it is marked as a high-priority category P. TT Set the maximum allowable delay threshold and zero packet loss target If it is a burst of traffic, record its maximum bandwidth requirement B. peak and priority label P burst ; Subsequently, the controller adjusts the current network congestion index I. cong Based on the values ​​∈[0,1] and historical traffic statistics, the queue scheduling process is initiated to ensure that different types of business flows receive differentiated quality of service guarantees.

6. The TSN scheduling optimization method based on flow-aware autonomous learning according to claim 1, characterized in that: In step 3, when intelligent queue scheduling is required for the current flow, the controller collects real-time queue length, priority mapping and packet loss rate information on the switch or router, and constructs the corresponding state matrix as the environment input for the deep reinforcement learning DRL model. Specifically, when intelligent queue scheduling of the current business flow is required, the controller obtains the real-time queue length Q from the router. i Priority Mapping Table M p and packet loss rate L i Information is collected, and the following state matrix is ​​constructed: This serves as the environment input for the deep reinforcement learning (DRL) model; each row represents the queue length, packet loss rate, and priority level of each service flow at the current moment; the DRL model learns policies and makes action decisions based on this state space, outputting the scheduling action 'a' for the next moment. t .

7. The TSN scheduling optimization method based on flow-aware autonomous learning according to claim 1, characterized in that, Step 4 is implemented as follows: After considering the priority requirements of both TT and burst flows, the DRL outputs the optimal queue allocation and priority adjustment scheme based on constraints. An enhanced queue scheduling mechanism, Pro-CQF, is adopted, with a three-queue structure at the switching node and a three-queue structure {Q0, Q1, 2} at the network node. The controller dynamically configures the gating time windows w0, w1, w2 and priority weights α0, α1, α2 for each queue using scheduling instructions generated by the DRL, and sends these configurations to the network devices via interfaces, driving them to adjust buffer sizes and priority mapping update rules in real time. Then, the DRL dynamically allocates queue time gating and priorities. The controller sends the DRL decision results to the network layer in the form of scheduling instructions, and the network devices adjust the gating time, priority, and buffer space of the corresponding queues according to the instructions, ensuring differentiated resource scheduling for different service flows at the queue level.

8. A TSN scheduling optimization device based on flow-aware autonomous learning, characterized in that, include: Lightweight flow detection module; Deployed in switches or routers, it automatically identifies periodic time-triggered TT flows and unpredictable burst flows using a lightweight detection module. Classification Management Module: The controller classifies and manages the identified service flow types, collects the topology information and traffic requirements of the entire network, and distributes them to the terminal nodes through the network management interface; Construct a transformation module; the controller constructs an intelligent queue scheduling task based on the collected network topology information and traffic demand, and transforms it into a Markov Decision Process (MDP). The state space is based on network resources, queue status and priority, and the action space is based on the scheduling strategy. The reward function is designed by combining throughput and latency indicators. The dynamic environment is driven by real-time data and the DRL model is trained. Scheduling module; The enhanced queue scheduling mechanism Pro-CQF is adopted, which configures different priority labels according to the classified traffic types before performing mixed flow scheduling; Update module: The controller periodically collects latency, packet loss rate, and end-to-end transmission delay metrics and feeds them back to the DRL model to update the scheduling strategy online.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images