Deterministic network time delay scheduling method and device based on TSN and storage medium

Abstract

The invention relates to the technical field of network time delay scheduling, and discloses a deterministic network time delay scheduling method and device based on a TSN and a storage medium. The method comprises the following steps: carrying out graph theory analysis on a TSN network topology structure to obtain a network node connection relationship and link bandwidth information; extracting time delay requirements and bandwidth requirement characteristics of the data streams, and generating a data stream priority queue and a resource requirement index matrix; inputting a deep Q network, establishing a state-action mapping relation including a network load state and a time slice distribution action, and generating an initial time delay scheduling strategy model; calculating queuing time delay, transmission time delay and processing time delay of each data stream, and generating a time slice resource redistribution scheme; generating a global resource allocation matrix based on the time slice resource reallocation scheme; and parameter feedback updating is carried out on the initial time delay scheduling strategy model to obtain a target time delay scheduling strategy model, and the utilization efficiency of network resources is remarkably improved through bandwidth borrowing and time slice reallocation.

Description

Technical Field

[0001] The present invention relates to the technical field of network delay scheduling, and in particular to a deterministic network delay scheduling method, device and storage medium based on TSN. Background Art

[0002] As a new generation of deterministic Ethernet technology, Time Sensitive Networking (TSN) provides deterministic communication guarantee for industrial control systems through mechanisms such as time synchronization and traffic scheduling. However, in practical applications, the delay scheduling method of traditional TSN networks often adopts a static configuration method, which is difficult to cope with dynamically changing network environments and diverse business needs.

[0003] The main challenge facing TSN networks today is how to achieve intelligent latency scheduling under complex network topologies. Especially in ultra-dense node deployment scenarios, where the network topology is complex and data flows are diverse, traditional scheduling algorithms are difficult to simultaneously meet the latency requirements of service flows of different priorities, which can easily cause network congestion and performance degradation. In addition, most existing scheduling methods focus on latency optimization of a single data flow, ignoring the global coordination of network resources, resulting in low resource utilization efficiency. Summary of the invention

[0004] The present invention provides a TSN-based deterministic network delay scheduling method, device and storage medium. The present invention significantly improves the efficiency of network resource utilization through bandwidth borrowing and time slice reallocation.

[0005] In a first aspect, the present invention provides a deterministic network delay scheduling method based on TSN, and the deterministic network delay scheduling method based on TSN includes: Perform graph theory analysis on the TSN network topology to obtain network node connection relationships and link bandwidth information; Inputting the network node connection relationship and the link bandwidth information into a multi-level priority-aware network, extracting the delay requirement and bandwidth requirement characteristics of the data flow, and generating a data flow priority queue and a resource requirement index matrix; Input the data flow priority queue and the resource demand index matrix into the deep Q network, establish a state-action mapping relationship including network load state and time slice allocation action, and generate an initial delay scheduling strategy model; According to the initial delay scheduling strategy model, the queuing delay, transmission delay and processing delay of each data flow are calculated to generate a time slice resource reallocation plan; Generate a global resource allocation matrix based on the time slice resource reallocation scheme; A performance evaluation is performed on the global resource allocation matrix to generate a network performance score, and parameter feedback is performed on the initial delay scheduling strategy model according to the network performance score to obtain a target delay scheduling strategy model.

[0006] In a second aspect, the present invention provides a deterministic network delay scheduling device based on TSN, and the deterministic network delay scheduling device based on TSN includes: Graph analysis module, used to perform graph analysis on the TSN network topology to obtain network node connection relationships and link bandwidth information; An extraction module, used to input the network node connection relationship and the link bandwidth information into a multi-level priority-aware network, extract the delay requirement and bandwidth requirement characteristics of the data flow, and generate a data flow priority queue and a resource requirement index matrix; Establish a module for inputting the data flow priority queue and the resource demand index matrix into a deep Q network, establishing a state-action mapping relationship including network load state and time slice allocation action, and generating an initial delay scheduling strategy model; A calculation module, used to calculate the queuing delay, transmission delay and processing delay of each data flow according to the initial delay scheduling strategy model, and generate a time slice resource reallocation plan; A generating module, configured to generate a global resource allocation matrix based on the time slice resource reallocation scheme; The updating module is used to perform performance evaluation on the global resource allocation matrix, generate a network performance score, and perform parameter feedback update on the initial delay scheduling strategy model according to the network performance score to obtain a target delay scheduling strategy model.

[0007] The third aspect of the present invention provides a TSN-based deterministic network delay scheduling device, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the TSN-based deterministic network delay scheduling device executes the above-mentioned TSN-based deterministic network delay scheduling method.

[0008] A fourth aspect of the present invention provides a computer-readable storage medium, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the above-mentioned TSN-based deterministic network delay scheduling method.

[0009] In the technical solution provided by the present invention, the TSN network topology is modeled by introducing a graph theory analysis method, and the sub-network is divided in combination with the minimum spanning tree algorithm, which effectively reduces the network complexity and improves the accuracy of resource allocation; a multi-level priority-aware network is used to intelligently classify data streams, and the delay demand characteristics of different data streams are accurately identified through three levels of feature extraction, traffic classification and priority mapping. A state-action mapping relationship is established based on a deep Q network, and the scheduling strategy is dynamically optimized through a reinforcement learning method, overcoming the limitations of traditional static scheduling methods; an end-to-end delay evaluation mechanism is designed, which comprehensively considers queuing delay, transmission delay and processing delay, and realizes accurate evaluation and dynamic adjustment of network performance; a bandwidth resource reuse mechanism across sub-networks is proposed, and the utilization efficiency of network resources is significantly improved through bandwidth borrowing and time slice reallocation; a performance evaluation system based on multiple indicators is constructed, and the scheduling strategy can be continuously optimized through a parameter feedback update mechanism to maintain the stability of network performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0011] Figure 1 A schematic diagram of a process flow of a TSN-based deterministic network delay scheduling method provided in an embodiment of the present application; Figure 2 A schematic block diagram of the structure of a TSN-based deterministic network delay scheduling device provided in an embodiment of the present application; Figure 3 A schematic block diagram of the structure of a TSN-based deterministic network delay scheduling device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0012] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0013] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change based on actual conditions.

[0014] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0015] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0016] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0017] See also Figure 1 , Figure 1 A flow chart of a TSN-based deterministic network delay scheduling method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the TSN-based deterministic network delay scheduling method provided in the embodiment of the present application includes steps S100 to S600.

[0018] Step S100: Graph analysis is performed on the TSN network topology structure to obtain network node connection relationships and link bandwidth information; It is understandable that the execution subject of the present invention may be a deterministic network delay scheduling device based on TSN, or a terminal or a server, which is not limited here. The embodiment of the present invention is described by taking a server as the execution subject as an example.

[0019] Specifically, the TSN network is topologically scanned to obtain the number of nodes and node identification information in the network. Through topological scanning, all switching nodes and terminal nodes in the network are identified, and unique identifiers are assigned to these nodes, and these identifiers are organized into an initial node set. The connection relationship of the nodes in the initial node set is analyzed to establish an adjacency matrix between the nodes. The adjacency matrix is ​​a standard graph theory structure used to represent the direct connection relationship between nodes in the network. Its rows and columns represent node identifiers, and the elements in the matrix indicate whether there is a direct link between the corresponding nodes. The basic topology of the TSN network is described through the adjacency matrix, and the network is abstracted into a network connection topology diagram in the form of an undirected graph. For the network connection topology diagram, the minimum spanning tree algorithm is used to calculate and obtain the backbone link distribution diagram of the network. The purpose of the minimum spanning tree is to identify the key links in the network. These links form a set of minimum cost paths to ensure network connectivity, and provide a backbone structure basis for subsequent sub-network division. Based on the generated backbone link distribution diagram, the entire network is sub-divided into several independent sub-network units for easy management and resource scheduling. After the sub-network division is completed, the boundary nodes of each sub-network unit are identified. Boundary nodes are key nodes for subnetworks to communicate with other subnetworks. Identifying boundary nodes can clarify the interconnection relationship between subnetworks, thereby generating an interconnection relationship matrix between subnetworks, describing the topological structure of interconnection between different subnetworks through boundary nodes, and providing accurate connection information for link bandwidth measurement and resource scheduling strategies. According to the interconnection relationship matrix between subnetworks, the bandwidth capacity of each link is measured to obtain a link bandwidth distribution map. Bandwidth capacity measurement includes evaluating the physical and logical bandwidth of the link, and recording the maximum bandwidth and actual available bandwidth of each link. Through this step, a bandwidth distribution map covering the entire network is formed to provide necessary bandwidth constraints for resource allocation during scheduling. The reference time period is calculated based on the link bandwidth distribution map. The reference time period is the basis of periodic scheduling in TSN, which determines the scheduling period and time synchronization of time-sensitive traffic. Based on the calculation of the reference time period, the period is divided into a preset number of time slice units to obtain a time slice division scheme. The time slice division scheme provides a global time resource framework and defines the transmission time window of each traffic in the network. After the time slice division is completed, the initial time slice resources are allocated to each sub-network unit according to the division scheme, a specific scheduling plan is formed, and finally the network node connection relationship and link bandwidth information are generated.

[0020] Step S200: input the network node connection relationship and link bandwidth information into the multi-level priority-aware network, extract the delay requirement and bandwidth requirement characteristics of the data flow, and generate the data flow priority queue and resource requirement index matrix; Specifically, the connection relationship of network nodes is transformed into a matrix. By structurally processing the node adjacency matrix, the mutual relationship and connection mode between nodes in the network are extracted to form a network topology feature matrix. At the same time, the bandwidth information of the link is normalized, and the bandwidth value of each link is mapped to the standardized range of [0,1] so that the subsequent feature extraction module can consistently process data of different scales and obtain the bandwidth feature vector. The network topology feature matrix and bandwidth feature vector are input into the feature extraction layer of the multi-level priority perception network. The feature extraction layer adopts a three-layer convolutional network structure. Each layer of the convolutional network uses the ReLU activation function to enhance the nonlinear expression ability of the network. At the same time, the batch normalization technology is used to improve the training stability and convergence speed. The basic feature map of the data stream output by this process is a preliminary description of the key characteristics of the input network, such as delay and bandwidth. The basic feature map of the data stream is input into the traffic classification layer of the multi-level priority perception network. The traffic classification layer includes two fully connected layers and one Softmax layer. The data stream is classified by this deep structure, which can effectively identify the service characteristics of the data stream and divide it into key service flow, periodic service flow and burst service flow. The classification results are encoded into a traffic category identification matrix, which describes the category and characteristics of each data flow. The feature vector of the traffic category identification matrix is ​​constructed to extract the parameters of the four dimensions of end-to-end delay requirement, bandwidth requirement, burst characteristics and periodic characteristics of the data flow. These parameters are obtained by jointly analyzing the classification results of the traffic and the initial network characteristics, and finally a set of traffic feature vectors is generated, which can fully express the key attributes that need to be considered in the resource scheduling process of each data flow. The set of traffic feature vectors is input into the priority mapping layer of the multi-level priority-aware network. The priority mapping layer consists of a fully connected layer and a Sigmoid activation layer. Through this network structure, the traffic characteristics are refined, and the delay sensitivity weight, bandwidth requirement weight and priority level weight of each data flow are calculated. These weight parameters reflect the different sensitivities of data flows to delay, bandwidth and priority. According to the priority quantification parameters, the resource demand index of each data flow is calculated. The resource demand index is calculated by weighted summation, and the delay sensitivity weight, bandwidth requirement weight and priority level weight are proportionally combined to quantify the overall demand level of the data flow for network resources. The resource demand index is sorted in descending order according to the numerical value to construct a data stream priority queue. In the data stream priority queue, high-demand data streams are scheduled first, while low-demand data streams are processed in subsequent time slices. The priority queues are grouped according to the preset time window size to form a grouping matrix to facilitate the reasonable resource allocation of data streams with different priorities in time slice scheduling. Each matrix element in the grouping matrix is ​​normalized to standardize the resource demand index to the [0,1] interval to eliminate the differences in the magnitude of resource demand between different data streams, and finally generate a resource demand index matrix.

[0021] Step S300: input the data flow priority queue and resource demand index matrix into the deep Q network, establish a state-action mapping relationship including network load state and time slice allocation action, and generate an initial delay scheduling strategy model; Specifically, the data stream priority queue and resource demand index matrix are feature fused. By fusing the priority information of the data stream and its corresponding resource demand index, a high-dimensional feature representation is formed to construct the state space of the deep Q network. The fused features are represented as a network state feature matrix, which contains the priority weights, bandwidth requirements and corresponding resource occupancy information of each data stream in the current network, and comprehensively describes the current network load state. In order to realize the discrete scheduling of time slices, the network state feature matrix is ​​encoded with the time slice allocation action. The time slice allocation action is quantized into several discrete gears, which represent the resource allocation ratio of each time slice. Through this encoding mechanism, an action space matrix is ​​formed, in which each row corresponds to a specific time slice allocation action, and all possible time slice allocation schemes constitute a complete action space to support the action selection and optimization of the deep Q network. The constructed network state feature matrix is ​​input into the value evaluation module of the deep Q network. In the value evaluation module, the data passes through three fully connected layers in sequence. Each fully connected layer uses the ReLU activation function to enhance the nonlinear expression ability and ensure the stability of gradient propagation. After the value evaluation module, the state value evaluation result is obtained, which characterizes the potential scheduling benefits of the current network state under different time slice allocation schemes. Based on the state value evaluation result and the action space matrix, the action value function is established, where the action value function is composed of immediate reward items, discount factors and future reward items. The immediate reward item reflects the direct contribution of the current time slice allocation action to latency optimization; the discount factor is used to weigh the impact of current rewards and future rewards, so that the model can consider the long-term scheduling effect; the future reward item evaluates the cumulative benefits that may be obtained in subsequent time steps. By comprehensively considering these three parts, the action value function can accurately evaluate the value of different time slice allocation actions. After completing the construction of the action value function, the deep Q network is trained based on the policy iteration mechanism. During the training process, the ε-greedy strategy is used for action selection, that is, the time slice allocation action with the highest value is selected with a certain probability, while exploring other actions with a smaller probability to avoid falling into the local optimal solution. Through the ε-greedy strategy, the selection probability distribution of the time slice allocation action is generated, which guides the parameter optimization of the network. According to the generated action selection probability distribution, the weight parameters of the deep Q network are optimized using the stochastic gradient descent method. During the optimization process, the action value function of each sample is used as part of the loss function. The network parameters are continuously adjusted through the back propagation algorithm to better fit the relationship between state and action value, and the state-action mapping model of the deep Q network is obtained. In order to improve the generalization ability of the model, the output results of the state-action mapping model are stored in the experience replay buffer. The experience replay buffer records the state transition samples of each time step. These samples include the current state, the action taken, the immediate reward, and the next state, thus forming a complete transition probability matrix.Based on the transition probability matrix, the deep Q network is learned online. By continuously resampling the data in the buffer and updating the network parameters, the model can adapt to the dynamically changing network load. After multiple rounds of online learning and optimization, the state-action mapping model obtained can effectively reflect the relationship between the network load state and the time slice allocation action, and form an initial delay scheduling strategy model.

[0022] Step S400: Calculate the queuing delay, transmission delay and processing delay of each data flow according to the initial delay scheduling strategy model, and generate a time slice resource reallocation plan; Specifically, the key parameters related to delay are extracted from the initial delay scheduling strategy model, including the number of path hops, link bandwidth and node processing capacity of the data flow. By integrating this information, a delay calculation parameter set is constructed, which is used to describe the performance constraints and resource characteristics involved in the entire transmission path of the data flow from the source node to the destination node. The queue cache data in the delay calculation parameter set is analyzed, focusing on the cache queue length and service processing rate of each node in the network. By modeling the traffic queuing situation of each node, the queuing delay distribution matrix of the data flow is obtained. Queuing delay is an important part of the delay composition. Its value depends on the queue backlog and service capacity of the node in a specific time period, and the queuing delay distribution matrix comprehensively reflects the traffic queuing characteristics of all nodes in the network. After obtaining the queuing delay distribution matrix, the transmission process of the data packet on each link is modeled to calculate the transmission delay. The transmission delay is determined by the link length and bandwidth. Its calculation model is based on the physical transmission rate of the link and the size of the data flow. The link transmission delay matrix is ​​obtained by analyzing each link. This matrix is ​​used to describe the transmission time consumed by the data flow in each link on the path. According to the link transmission delay matrix, the processing capacity of the network nodes is load analyzed to calculate the processing delay. The processing delay is directly related to the processing capacity and current load of the node. By analyzing the performance of each node in processing traffic, the node processing delay matrix is ​​obtained. This matrix reflects the time overhead of each node in the network when processing data flow. Combined with the queuing delay distribution matrix and the link transmission delay matrix, the delay performance of the data flow in the entire network is more accurately evaluated. The queuing delay distribution matrix, the link transmission delay matrix and the node processing delay matrix are superimposed to obtain the end-to-end total delay matrix, which reflects the total delay information of all data flows from source to destination in the network, and marks the data flows with a total delay exceeding the threshold to generate a delay-exceeding traffic set. Based on the delay-exceeding traffic set, the time slice reallocation coefficient of each subnetwork is calculated. The design principle of the time slice reallocation coefficient is inversely proportional to the end-to-end total delay, that is, the traffic with a larger total delay is given priority to obtain more resources in the time slice resource allocation, while the traffic with a smaller delay has less resource allocation, thereby achieving dynamic optimization of global resources. According to the calculation results of the reallocation coefficients, a time slice adjustment factor matrix is ​​generated, which is used to guide the update of the time slice allocation plan. Based on the time slice adjustment factor matrix, the initial time slice allocation plan is optimized and adjusted to obtain a time slice resource reallocation plan.

[0023] Step S500: Generate a global resource allocation matrix based on the time slice resource reallocation solution; Specifically, the resource utilization rate of each sub-network unit in the time slice resource reallocation scheme is statistically calculated, and the time slice allocation in each sub-network unit and the corresponding bandwidth resource occupancy ratio are analyzed to form a sub-network resource utilization status matrix. Each row of the matrix corresponds to a sub-network unit, and each column describes specific resource utilization characteristics, such as bandwidth occupancy, time slice distribution ratio, etc. Based on the sub-network resource utilization status matrix, a traffic load balancing index is constructed. The traffic load balancing index is an important indicator to measure the balance of resource distribution between sub-networks. It is calculated by comprehensive analysis of the resource utilization rate and traffic demand of the sub-network. Through this index, the imbalance of resource allocation in the network is identified, and a sub-network load distribution table is generated to record the load pressure and resource utilization status of each sub-network. After the sub-network load distribution table is generated, the sub-network units whose traffic load balancing index exceeds the preset threshold are selected for analysis. These sub-network units face the problem of resource shortage or unreasonable distribution, and their specific needs are quantified by calculating their bandwidth resource gap. The bandwidth resource gap analysis is modeled based on the difference between the current traffic demand and the existing bandwidth resources. The results are presented in the form of a bandwidth resource demand matrix, which describes the amount of additional bandwidth required by each subnetwork to meet the traffic demand. The bandwidth resource demand matrix is ​​subjected to multi-objective optimization calculation. The bandwidth borrowing cost weight, resource utilization weight, and network risk weight are input into the optimization function, and the optimal bandwidth borrowing scheme is generated by comprehensively considering the allocation cost of bandwidth resources, the overall network resource utilization efficiency, and the potential network risks. The goal of the optimization function is to minimize the cost of resource borrowing while balancing the load pressure of each subnetwork, so as to achieve a reasonable allocation of global resources. The resource allocation parameters in the bandwidth borrowing optimization scheme are input into the resource allocator to reallocate bandwidth resources across subnetworks and generate a bandwidth reuse matrix, which records the specific relationship between resource borrowing and reuse between subnetworks. In order to adapt to the scheduling mechanism of the time-sensitive network, the bandwidth reuse matrix is ​​converted into a time slice mapping, and the allocation relationship of bandwidth resources is mapped to the time slice resources to form a time slice mapping table. The time slice mapping table describes the distribution of bandwidth resources in the time dimension, so that resource allocation not only meets the bandwidth demand, but also meets the periodic scheduling requirements of TSN. Based on the time slice mapping table, the spatial arrangement of resources of each sub-network is calculated, and a three-dimensional resource allocation tensor is constructed to obtain a resource space distribution map, which integrates the information of the time dimension, space dimension and resource dimension to describe the distribution of resources in the entire network. The resource space distribution map is subjected to dimensionality reduction. During the dimensionality reduction process, the information of the time dimension, space dimension and resource dimension is integrated to eliminate redundant data while retaining key characteristics to form a global resource allocation matrix.

[0024] Step S600: Perform performance evaluation on the global resource allocation matrix to generate a network performance score, and perform parameter feedback update on the initial delay scheduling strategy model according to the network performance score to obtain a target delay scheduling strategy model.

[0025] Specifically, the performance index calculation of the global resource allocation matrix is ​​performed. By analyzing the resource utilization and scheduling effect of the network, the three core indicators of end-to-end delay compliance rate, link utilization rate and priority satisfaction are calculated respectively. These indicators comprehensively describe the performance of the network in supporting low-latency transmission, effectively utilizing link bandwidth and meeting the priority requirements of different data flows, forming a performance data set. The performance data set is input into the multi-indicator comprehensive evaluation module. In this module, by setting the weight coefficient α for the end-to-end delay compliance rate, setting the weight coefficient β for the link utilization rate, and setting the weight coefficient γ for the priority satisfaction, a comprehensive evaluation is performed according to the weighted summation calculation rule to obtain the network performance score. The network performance score is a normalized scalar value that reflects the performance of the current scheduling strategy in various dimensions. The network performance score is threshold tested, and the calculated score is compared with the preset performance benchmark threshold to generate a performance difference matrix, which records the deviation between the actual performance and the target performance. Each item in the matrix reflects the optimization space of a specific indicator. By analyzing the performance difference matrix, the shortcomings of the current scheduling strategy in terms of delay control, resource utilization and priority satisfaction are located. Based on the performance difference matrix, the key parameters in the initial delay scheduling strategy model are adjusted by feedback. The reverse error is calculated by the immediate reward item, discount factor and future reward item in the deep Q network. The performance difference is used as the target value and compared with the prediction result of the model to generate a parameter gradient matrix to describe the direction and magnitude that need to be optimized. According to the optimization rule of the stochastic gradient descent method, the parameter gradient matrix is ​​used to update the weight of the deep Q network. The introduction of the stochastic gradient descent method ensures that the model can gradually approach the global optimal solution in the multi-dimensional parameter space, thereby effectively improving the prediction ability and decision-making efficiency of the model. The optimized deep Q network weights more accurately describe the state-action mapping relationship and generate a better strategy in the next scheduling calculation. According to the updated network weights, the delay scheduling calculation of the data flow is re-performed to generate a new state-action mapping table. The state-action mapping table records the optimal time slice allocation action corresponding to different network states, providing an execution plan for actual scheduling. The new mapping table is verified online. By applying it to the verification sample, the average reward value of the sample is calculated to evaluate whether the optimized strategy can significantly improve the network performance. During the online verification process, the improvement effect of the strategy is judged by comparing the average reward value before and after optimization. If the verification results show that the optimized strategy can meet the preset performance goals, the optimized scheduling strategy parameters are solidified into the deep Q network to form the final target delay scheduling strategy model. The solidified model can achieve more efficient scheduling decisions and adapt to the dynamically changing network environment, ensuring consistent delay guarantees and resource utilization efficiency under different load conditions.

[0026] In the embodiment of the present invention, the TSN network topology is modeled by introducing a graph theory analysis method, and the sub-network is divided in combination with the minimum spanning tree algorithm, which effectively reduces the network complexity and improves the accuracy of resource allocation; a multi-level priority-aware network is used to intelligently classify data streams, and the delay demand characteristics of different data streams are accurately identified through three levels of feature extraction, traffic classification and priority mapping. A state-action mapping relationship is established based on a deep Q network, and the scheduling strategy is dynamically optimized through a reinforcement learning method, overcoming the limitations of traditional static scheduling methods; an end-to-end delay evaluation mechanism is designed, which comprehensively considers queuing delay, transmission delay and processing delay, and realizes accurate evaluation and dynamic adjustment of network performance; a bandwidth resource reuse mechanism across sub-networks is proposed, and the utilization efficiency of network resources is significantly improved through bandwidth borrowing and time slice reallocation; a performance evaluation system based on multiple indicators is constructed, and the scheduling strategy can be continuously optimized through a parameter feedback update mechanism to maintain the stability of network performance.

[0027] In a specific embodiment, the process of executing step S100 may specifically include the following steps: Perform topological scan on the TSN network, obtain the number of nodes and node identification information in the network, and obtain the initial node set; Analyze the connection relationship of the nodes in the initial node set, establish the node adjacency matrix, obtain the network connection topology map, and calculate the minimum spanning tree of the network connection topology map to obtain the network backbone link distribution map; Based on the network backbone link distribution map, the sub-network is divided to obtain a sub-network unit set, and the boundary node of each sub-network unit in the sub-network unit set is identified to obtain the interconnection relationship matrix between sub-networks; According to the interconnection relationship matrix between sub-networks, the bandwidth capacity of each link is measured to obtain the link bandwidth distribution map; The reference time period is calculated based on the link bandwidth distribution diagram, and the reference time period is divided into a preset number of time slice units to obtain a time slice division scheme. According to the time slice division scheme, initial time slice resources are allocated to each sub-network unit to obtain the network node connection relationship and link bandwidth information.

[0028] Specifically, the TSN network is topologically scanned to identify the number of nodes and node identification information in the network. The network devices are gradually queried through network protocols (such as LLDP, Link Layer Discovery Protocol) to obtain the identification information of all nodes (such as MAC address or IP address), and these identification information are organized into an initial node set, recorded as set ,in Indicates nodes, Represents the total number of nodes. Perform connection relationship analysis on the nodes in the initial node set to establish a node adjacency matrix The adjacency matrix is ​​a The two-dimensional matrix, matrix elements Representation Node and The connection status between nodes. and If there is a direct physical link connection, then otherwise For example, suppose there are 5 nodes in the network and the connection relationship between the nodes is as follows: connect and ;node connect and ;node connect ;node connect and ;node Connect only . Then the adjacency matrix It is expressed as: ; The network connection topology diagram constructed based on the adjacency matrix clearly describes the network structure. In order to extract the key link structure of the network, its minimum spanning tree is calculated. The minimum spanning tree is a connected subgraph containing all nodes, and the sum of the edge weights of the graph is the smallest. Assume that the weight of the link is the inverse of its bandwidth ,in Is a node and If the link bandwidth between the two nodes is large, the minimum spanning tree algorithm (such as Prim's algorithm or Kruskal's algorithm) is used to find the network backbone link distribution map with the lowest cost. After obtaining the network backbone link distribution map, sub-network division is performed based on the backbone links. By analyzing the key nodes (i.e., highly connected central nodes) in the minimum spanning tree, multiple sub-network unit sets are divided. ,in Indicates Sub-network units. Analyze each sub-network unit in the sub-network unit set, and identify the boundary node set by detecting its connection nodes with other sub-networks. The boundary nodes are bridges connecting different sub-networks, and the interconnection matrix between sub-networks is constructed through these nodes. ,in Represents a subnetwork and The connection strength is the number of links between the boundary nodes. , measure the bandwidth capacity of each link and obtain the link bandwidth distribution diagram Bandwidth capacity The measurement is to query the bandwidth parameters of the device through the network management protocol (such as SNMP), or to obtain the link utilization through real-time monitoring After calculation: ; in, Yes Link and Current utilization of the link bandwidth. Calculate the base time period The benchmark time period is the basis for global network scheduling, and its value is determined by the minimum bandwidth link: ; in, is the maximum transmission unit of a data stream. is the minimum link bandwidth in the network. Divide into time slice units, and obtain the time slice division scheme: ; in, is the length of each time slice. According to the time slice division scheme, the initial time slice resources are allocated to each sub-network. Assume that the sub-network The total demand time slice is , then the time slice resource ratio is: ; Through the above allocation, the network node connection relationship and link bandwidth information are obtained.

[0029] In a specific embodiment, the process of executing step S200 may specifically include the following steps: Perform matrix transformation on the connection relationship of network nodes to generate the network topology feature matrix, and normalize the link bandwidth information to obtain the bandwidth feature vector; The network topology feature matrix and bandwidth feature vector are input into the feature extraction layer of the multi-level priority perception network. The feature extraction layer contains three convolutional layers. Each convolutional layer uses the ReLU activation function and batch normalization to obtain the basic feature map of the data stream. The basic feature graph of the data stream is input into the traffic classification layer of the multi-level priority-aware network. The traffic classification layer includes two fully connected layers and one Softmax layer. The data stream is classified into key business flows, periodic business flows, and burst business flows to obtain a traffic category identification matrix. The traffic category identification matrix is ​​constructed with feature vectors, and the parameters of the four dimensions of end-to-end delay requirement, bandwidth requirement, burst characteristics, and periodic characteristics are extracted to obtain a set of traffic feature vectors. The traffic feature vector set is input into the priority mapping layer of the multi-level priority-aware network. The priority mapping layer includes a fully connected layer and a Sigmoid layer. The delay sensitivity weight, bandwidth requirement weight and priority level weight are calculated to obtain the priority quantization parameter. According to the priority quantization parameter, the resource demand index is calculated for each data flow. The resource demand index is obtained by weighted summation of the delay sensitivity weight, bandwidth demand weight and priority level weight. The resource demand indexes are sorted in descending order according to their numerical values, a data stream priority queue is constructed, and the data stream priority queues are grouped according to the preset time window size to obtain a grouping matrix; Each matrix element in the grouping matrix is ​​normalized to obtain a resource demand index matrix.

[0030] Specifically, the network node connection relationship is transformed into a matrix to generate a network topology feature matrix. Assume that the nodes in the network are represented by the set Indicates that the connection relationship is represented by the adjacency matrix Indicates that Representation Node and The connection status of the node (1 means connected, 0 means disconnected). In order to reflect the network topology characteristics, the degree centrality of the node is calculated by matrix transformation. , the formula is: ; By combining the degree centrality of the node with the adjacency matrix, the network topology feature matrix is ​​constructed , where each row represents the local topological characteristics of a node, such as degree centrality, link weight, etc. The link bandwidth information is normalized, assuming that the bandwidth set is ,in Representation Node and The bandwidth of the link between the two nodes is the normalized bandwidth feature vector Calculated by the following formula: ; in and are the minimum and maximum bandwidth values ​​in the network respectively. Mapped to the interval [0,1] for feature extraction. and bandwidth eigenvector Input the feature extraction layer of the multi-level priority perception network. The feature extraction layer contains three convolutional layers, each of which uses the ReLU activation function. Improve the nonlinear expression ability of the model and standardize the intermediate output through batch normalization technology to accelerate model training and improve stability. Assume that the convolution kernel size is , the input matrix is , the calculation process of the first layer of convolution is: ; in, is the weight matrix of the first convolution layer, is the bias term, and the symbol * indicates the convolution operation. The basic feature map of the output data stream after the three layers of convolution are superimposed , whose shape is ,in represents the number of samples of the feature map, Represents the feature dimension. The basic feature map of the data stream Input the traffic classification layer of the multi-level priority-aware network, which contains two fully connected layers and one Softmax layer. The first fully connected layer is connected through the weight matrix Mapping to the intermediate feature space: ; The second fully connected layer further compresses the feature dimension, and the Softmax layer normalizes the output results to classification probabilities. : ; in Yes Category The activation value of is the number of classification categories (critical traffic, periodic traffic, and burst traffic). The classification results are organized into a traffic category identification matrix , where each row represents the category distribution of data flow. Construct feature vectors and generate a set of traffic feature vectors by analyzing parameters in four dimensions: end-to-end delay requirements, bandwidth requirements, burst characteristics, and periodic characteristics For example, the end-to-end delay is required to be calculated by the formula: ; in is the number of path hops, is the packet size, is the link bandwidth. The traffic feature vector set is input into the priority mapping layer, which contains a fully connected layer and a Sigmoid activation function to calculate the delay sensitivity weight. , Bandwidth demand weight and priority level weights : ; in is the activation value of the corresponding dimension. Based on the priority quantization parameter , calculate the resource requirement index of each data flow : ; in is the weight coefficient, which indicates the relative importance of latency, bandwidth and priority. The resource demand index of all data flows is sorted in descending order according to the numerical value, and a data flow priority queue is constructed. Group the data stream to form a grouping matrix . Grouping matrix Each element in is normalized, and the normalization formula is: ; The normalized matrix It is the resource demand index matrix, which fully describes the resource demand of each data flow in the network in different time windows.

[0031] In a specific embodiment, the process of executing step S300 may specifically include the following steps: The data stream priority queue and resource demand index matrix are integrated to construct the state space of the deep Q network and obtain the network state feature matrix. The network state feature matrix is ​​encoded with time slice allocation action, and the allocation ratio of each time slice is divided into discrete gears to obtain the action space matrix; The network state feature matrix is ​​input into the value assessment module of the deep Q network. The value assessment module contains three fully connected layers. Each fully connected layer uses the ReLU activation function to obtain the state value assessment result. According to the state value evaluation results and the action space matrix, an action value function is constructed. The action value function includes an immediate reward item, a discount factor, and a future reward item. Perform strategy iteration calculation on the action value function, select time slices to allocate actions based on the ε-greedy strategy, and obtain the action selection probability distribution of the deep Q network; According to the probability distribution of action selection, the parameters of the deep Q network are optimized, and the network weights are updated using the stochastic gradient descent method to obtain the state-action mapping model; The state-action mapping model is input into the experience replay buffer, the state transition samples of each time step are stored, and the transition probability matrix is ​​obtained. The state-action mapping model is learned online based on the transition probability matrix to obtain the initial delay scheduling strategy model.

[0032] Specifically, the data stream priority queue and the resource demand index matrix Perform feature fusion to generate state space features of the deep Q network. Data flow priority Describe the scheduling priority of data flow and resource demand index Indicates The data stream is in By combining the two into a state vector , generate the state feature matrix ,in , is the number of data streams, is the number of time slices. For the state feature matrix The time slice allocation action is coded. The time slice allocation ratio is divided into Discrete gears , each gear represents a different resource allocation strategy. For example, suppose the time slice allocation ratio is divided into , each gear corresponds to a specific bandwidth resource allocation. Action space matrix Then all possible action combinations of each data stream are recorded, and the shape is , represents the possible allocation strategy of each data stream in each time slice. Input to the value assessment module of the deep Q network, the value assessment module contains three fully connected layers, each layer uses the ReLU activation function To enhance the nonlinear expression ability and avoid the gradient vanishing problem. The calculation formula of the first fully connected layer is: ; in, is the weight matrix of the first layer, is the bias term, is the output of the first layer. After three layers of full connection calculation, the output state value evaluation result , whose value indicates the current state Based on the state value evaluation results and the action space matrix , construct the action value function The action value function consists of the immediate reward term , Discount Factor and future rewards Composition, its formula is: ; in, Indicates that at the current time step Execute an action Immediate rewards after the event, such as reduced latency or improved resource utilization; is a discount factor that weighs the impact of current rewards against future rewards; is the next state The maximum expected value of . Calculated through policy iteration, using - Greedy strategy selects time slice allocation action. In each round of iteration, The probability of randomly selecting actions to explore is The probability of selecting the action with the greatest current value is used, thereby avoiding falling into the local optimum while searching for the global optimal solution. It is described by the following formula: ; According to the action selection probability distribution, the parameters of the deep Q network are optimized and the network weights are updated using the stochastic gradient descent method. Assume that the target value is , the current value is , the optimization goal is to minimize the loss function: ; in, is the parameter of the deep Q network. The gradient is calculated by the back propagation algorithm , and adjust the parameters according to the following update rules: ; in, is the learning rate, which determines the step size of parameter update. At each time step, the state-action mapping model The samples are stored in the experience playback buffer to record the current state ,action , Instant Rewards and the next state Based on experience replay, sample state transition samples to calculate the transition probability matrix , which is used to further optimize the model. Through iterative optimization, the state-action mapping model generated by the deep Q network effectively reflects the mapping relationship between the network state and the time slice allocation, thereby obtaining an optimized initial delay scheduling strategy model.

[0033] In a specific embodiment, the process of executing step S400 may specifically include the following steps: According to the initial delay scheduling strategy model, the path hop count, link bandwidth and node processing capability parameters of the data flow are extracted to obtain a delay calculation parameter set; Analyze the queue cache data in the delay calculation parameter set, and obtain the data flow queue delay distribution matrix according to the cache queue length and service processing rate of each node; Based on the data flow queuing delay distribution matrix, the transmission process of the data packet on each link is modeled, the transmission delay is calculated, and the link transmission delay matrix is ​​obtained; According to the link transmission delay matrix, the data processing capacity of the network node is load analyzed, the processing delay is calculated, and the node processing delay matrix is ​​obtained; The data flow queuing delay distribution matrix, link transmission delay matrix and node processing delay matrix are superimposed to obtain the end-to-end total delay matrix, and the data flows exceeding the delay threshold in the end-to-end total delay matrix are marked to generate a delay-exceeding traffic set. According to the delay-exceeding traffic set, the time slice reallocation coefficient of each subnetwork is calculated. The time slice reallocation coefficient is inversely proportional to the end-to-end total delay. The time slice adjustment factor matrix is ​​obtained, and the time slice allocation plan is updated based on the time slice adjustment factor matrix to obtain the time slice resource reallocation plan.

[0034] Specifically, the key network parameters of the data flow, including the number of path hops, are extracted based on the initial delay scheduling strategy model. , link bandwidth and node processing power . Path hop count Indicates the number of network nodes that the data flow passes through from the source node to the destination node, and the link bandwidth Representation Node and The maximum transmission rate of the link between nodes, and the node processing capacity The reflected node The amount of data that can be processed per unit time. By organizing these parameters into a delay calculation parameter set , describing the transmission path characteristics and resource allocation of each data flow. Analyze the queue cache data in the delay calculation parameter set, combined with the cache queue length of each node and service processing rate , calculate the queuing delay of the data flow at each node. The queue delay Using the M / M / 1 queue model for estimation, the formula is: ; in Is a node The service rate, Is a node The arrival traffic rate. By calculating the queuing delay of each node, the queuing delay distribution matrix of the data flow is generated. , each element in the matrix Indicates that the data flow is The node The queuing delay on the queue. Based on the queuing delay distribution matrix , model the transmission process of data packets on each link to calculate the transmission delay , the transmission delay of the link is determined by the packet size and link bandwidth Determine, the formula is: ; in is the size of the packet; Yes Link By calculating the transmission delay of each link, we can get the link transmission delay matrix , where each element Indicates that the data packet is on the link After obtaining the link transmission delay matrix, the data processing capacity of the network node is load analyzed to calculate the processing delay The processing delay reflects the time required for a node to process a data packet, and the formula is: ; in Is a node By calculating the processing delay of all nodes, a node processing delay matrix is ​​formed. . The queue delay distribution matrix , link transmission delay matrix and node processing delay matrix Perform superposition operation to obtain the end-to-end total delay matrix , the formula is: ; in is the number of path hops, Representing data flow The total delay from source to destination. The data flow that exceeds the preset delay threshold is marked to generate a delay-exceeding traffic set , where each element represents the timed-out data flow and its delay exceeding limit. , calculate the time slice reallocation coefficient of each sub-network. Time slice reallocation coefficient It is inversely proportional to the total end-to-end delay and is calculated as follows: ; in It is a subnetwork The total delay set of the data flow in the network. The time slice adjustment factor matrix is ​​generated by calculating the time slice reallocation coefficients of all sub-networks , which records the time slice resource adjustment ratio of each sub-network. Based on the time slice adjustment factor matrix , update the time slice allocation plan. New time slice resource allocation Calculated by the following formula: ; in It is a subnetwork The initial time slice allocation, is the corresponding adjustment factor. By updating the time slice allocation schemes of all sub-networks, the final time slice resource reallocation scheme is obtained.

[0035] In a specific embodiment, the process of executing step S500 may specifically include the following steps: Perform resource utilization statistics calculation on each sub-network unit in the time slice resource reallocation scheme to obtain a sub-network resource utilization state matrix; Based on the sub-network resource utilization state matrix, a traffic load balancing index is constructed, and the bandwidth resource distribution of each sub-network unit is analyzed to obtain a sub-network load distribution table; According to the sub-network load distribution table, a sub-network unit whose traffic load balancing index exceeds a preset threshold is selected, and a bandwidth resource gap analysis is performed on the sub-network unit to obtain a bandwidth resource demand matrix; Perform multi-objective optimization calculation on the bandwidth resource demand matrix, input bandwidth borrowing cost weight, resource utilization weight and network risk weight into the optimization function, and obtain the bandwidth borrowing optimization plan; The resource allocation parameters in the bandwidth borrowing optimization scheme are input into the resource allocator to reallocate the bandwidth resources across sub-networks to obtain a bandwidth reuse matrix, and the bandwidth reuse matrix is ​​converted into a time slice mapping to convert the bandwidth resources into a time slice resource allocation relationship to obtain a time slice mapping table; According to the time slice mapping table, the spatial arrangement of resources of each sub-network is calculated, and a three-dimensional resource allocation tensor is constructed to obtain a resource space distribution map. The resource space distribution map is then subjected to dimensionality reduction processing, and the information of the time dimension, space dimension, and resource dimension is integrated to obtain a global resource allocation matrix.

[0036] Specifically, the resource utilization rate of each sub-network unit in the time slice resource reallocation scheme is statistically calculated. Suppose the sub-network set is , the resource utilization of each sub-network is defined as: ; in It is a subnetwork Resource utilization, Indicates the data flow assigned The amount of resources, Represents a subnetwork The total time slice resources are calculated by Value, generate sub-network resource utilization state matrix , which describes the resource usage of each sub-network. Based on the sub-network resource utilization status matrix , build traffic load balancing index , which is used to measure the balance of resource loads between sub-networks. The calculation formula of the load balancing index is: ; in is the average resource utilization of all subnetworks and is defined as: ; By calculation Value, generate sub-network load distribution table , record the load deviation degree of each sub-network. According to the sub-network load distribution table, select the traffic load balancing index Exceeding the preset threshold The sub-network units are marked as overloaded sub-networks, and bandwidth resource gap analysis is performed on them. Suppose the actual required bandwidth of the overloaded sub-network is , the allocated bandwidth is , then the bandwidth resource gap for: ; like , then the subnetwork There is a shortage of resources. By analyzing the bandwidth resource gap of all overloaded sub-networks, a bandwidth resource demand matrix is ​​generated. The bandwidth resource demand matrix is ​​optimized by multiple objectives to determine the best allocation scheme for bandwidth borrowing. The optimization goal is to balance the bandwidth borrowing cost, resource utilization efficiency, and network risk. Assume that the bandwidth borrowing cost weight is , resource utilization weight is , the network risk weight is , the optimization objective function is: ; in, represents the bandwidth borrowing cost, and the total cost is , It is a subnetwork The unit bandwidth borrowing cost; Represents the resource utilization difference, ; Represents network risk, based on potential link bottlenecks generated during resource allocation. The bandwidth allocation upper limit constraint is solved by an optimization algorithm (such as genetic algorithm or gradient descent method) to generate a bandwidth borrowing optimization plan. The resource allocation parameters in the bandwidth borrowing optimization plan are input into the resource allocator to redistribute bandwidth resources across sub-networks. After bandwidth redistribution, a bandwidth reuse matrix is ​​generated. ,in From the sub-network Borrow to subnetwork In order to map bandwidth resource allocation to time slice resource allocation, a time slice mapping conversion is performed, and the conversion formula is: ; in is bandwidth The corresponding time slice resource amount, is the smallest unit of bandwidth, Is the base time period. According to the time slice mapping table , calculate the spatial arrangement of resources of each sub-network and construct a three-dimensional resource allocation tensor . Tensor The three dimensions are time dimension , spatial dimension and resource dimensions , describes the distribution of resources in the entire network. Perform dimensionality reduction processing to integrate the information of time dimension, space dimension and resource dimension. The dimensionality reduction formula is: ; The global resource allocation matrix is ​​obtained by dimensionality reduction , this matrix is ​​the result of the optimal allocation of global network resources.

[0037] In a specific embodiment, the process of executing step S600 may specifically include the following steps: The performance indicators of the global resource allocation matrix are calculated to obtain the performance data set of three indicators: end-to-end delay compliance rate, link utilization rate and priority satisfaction rate. The performance data set is input into the multi-index comprehensive evaluation module, and the network performance score is obtained by weighted summing the weight coefficient α of the end-to-end delay compliance rate, the weight coefficient β of the link utilization rate, and the weight coefficient γ of the priority satisfaction degree. Perform threshold detection on the network performance score and compare it with the preset performance benchmark threshold to obtain a performance difference matrix; Based on the performance difference matrix, reverse error calculation is performed on the immediate reward item, discount factor and future reward item in the initial delay scheduling strategy model to obtain the parameter gradient matrix. The parameter gradient matrix is ​​updated according to the optimization rule of the stochastic gradient descent method to the deep Q network weights of the initial delay scheduling strategy model to obtain the updated network weights; According to the updated network weights, the delay scheduling calculation of the data flow is re-performed to obtain a new state-action mapping table; The new state-action mapping table is verified online, and the policy optimization result is obtained by calculating the average reward value of the verification sample. The scheduling strategy is confirmed based on the policy optimization result, and the verified scheduling strategy parameters are solidified into the deep Q network to obtain the target delay scheduling strategy model.

[0038] Specifically, from the global resource allocation matrix The network performance-related data is extracted from the network, and three key performance indicators, end-to-end delay compliance rate, link utilization rate, and priority satisfaction rate, are calculated to form a performance data set. Reflects the proportion of data flows that meet the latency requirement. The calculation formula is: ; in, Representing data flow The total end-to-end delay is is the delay threshold, is the total number of data flows, It is an indicator function, which takes the value of 1 when the condition is met, otherwise it takes the value of 0. Link utilization Measures the usage of link bandwidth resources. The calculation formula is: ; in, Yes Link The bandwidth utilization of Yes Link The total bandwidth of the priority. Describes the degree to which the network satisfies the scheduling of data flows of different priorities, defined as: ; in, It is a data stream The weight of is the actual assigned priority, It is a data stream The calculated performance data set Input the multi-index comprehensive evaluation module and use the weight coefficient , and The three performance indicators are weighted and summed to obtain the network performance score. : ; in, Ensure the normalization of weights. The specific values ​​are allocated according to actual needs (for example, delay-sensitive scenarios can increase By calculating , generate comprehensive performance evaluation results. Score network performance Perform threshold detection and compare it with the preset performance benchmark threshold Compare and generate performance difference matrix : ; like , indicating that the current scheduling strategy does not achieve the performance target, and the initial delay scheduling strategy model needs to be further optimized. The optimization process starts from the performance difference matrix Start with the immediate reward item in the model , Discount Factor and future rewards Perform reverse error calculation to obtain the parameter gradient matrix ; in, are the parameters of the deep Q network, is the error loss function. Update the network weights by gradient descent , the formula is: ; in is the learning rate, which determines the step size of parameter update. After updating the network weights, the delay scheduling calculation of the data flow is re-calculated based on the optimized deep Q network to generate a new state-action mapping table The state-action mapping table describes the optimal time slice allocation action under each network state and is a direct reflection of the scheduling strategy. The state-action mapping table is evaluated through online verification, and the average reward value of the verification sample is As an indicator of optimization effect, the calculation formula is: ; in, is the total number of validation samples, It is a sample Instant reward. If it is higher than the previous verification result, the current scheduling strategy parameters are solidified into the deep Q network to form the final target delay scheduling strategy model.

[0039] See also Figure 2 , Figure 2 A schematic block diagram of the structure of a TSN-based deterministic network delay scheduling device 200 provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, a deterministic network delay scheduling device 200 based on TSN includes: A graph analysis module 210 is used to perform graph analysis on the TSN network topology structure to obtain network node connection relationships and link bandwidth information; Extraction module 220, used to input the network node connection relationship and link bandwidth information into the multi-level priority-aware network, extract the delay requirement and bandwidth requirement characteristics of the data flow, and generate the data flow priority queue and resource requirement index matrix; Establishing module 230, for inputting data flow priority queue and resource demand index matrix into deep Q network, establishing state-action mapping relationship including network load state and time slice allocation action, and generating initial delay scheduling strategy model; The calculation module 240 is used to calculate the queuing delay, transmission delay and processing delay of each data flow according to the initial delay scheduling strategy model, and generate a time slice resource reallocation plan; A generating module 250, configured to generate a global resource allocation matrix based on a time slice resource reallocation scheme; The updating module 260 is used to perform performance evaluation on the global resource allocation matrix, generate a network performance score, and perform parameter feedback update on the initial delay scheduling strategy model according to the network performance score to obtain a target delay scheduling strategy model.

[0040] Through the collaborative cooperation of the above components, the TSN network topology is modeled by introducing graph theory analysis methods, and the sub-network is divided by combining the minimum spanning tree algorithm, which effectively reduces the network complexity and improves the accuracy of resource allocation; a multi-level priority-aware network is used to intelligently classify data streams, and the delay demand characteristics of different data streams are accurately identified through three levels of feature extraction, traffic classification and priority mapping. A state-action mapping relationship is established based on a deep Q network, and the scheduling strategy is dynamically optimized through reinforcement learning methods, overcoming the limitations of traditional static scheduling methods; an end-to-end delay evaluation mechanism is designed, which comprehensively considers queuing delay, transmission delay and processing delay, and realizes accurate evaluation and dynamic adjustment of network performance; a bandwidth resource reuse mechanism across sub-networks is proposed, which significantly improves the efficiency of network resource utilization through bandwidth borrowing and time slice reallocation; a performance evaluation system based on multiple indicators is constructed, and the scheduling strategy can be continuously optimized through the parameter feedback update mechanism to maintain the stability of network performance.

[0041] See also Figure 3 , Figure 3A schematic block diagram of the structure of a TSN-based deterministic network delay scheduling device 300 provided in an embodiment of the present application, wherein the TSN-based deterministic network delay scheduling device 300 includes a processor 301 and a memory 302, wherein the processor 301 and the memory 302 are connected via a device bus 303, wherein the memory 302 may include a non-volatile storage medium and an internal memory.

[0042] The non-volatile storage medium can store a computer program. The computer program includes program instructions, and when the program instructions are executed by the processor 301, the processor 301 can execute any of the above-mentioned deterministic network delay scheduling methods based on TSN.

[0043] The processor 301 is used to provide computing and control capabilities to support the operation of the entire TSN-based deterministic network delay scheduling device 300.

[0044] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor 301, the processor 301 can execute any of the above-mentioned deterministic network delay scheduling methods based on TSN.

[0045] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the TSN-based deterministic network delay scheduling device 300 involved in the solution of the present application. The specific TSN-based deterministic network delay scheduling device 300 may include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0046] It should be understood that the processor 301 may be a central processing unit (CPU), and the processor 301 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0047] It should be noted that technical personnel in the relevant field can clearly understand that, for the convenience and conciseness of description, the specific working process of the TSN-based deterministic network delay scheduling device 300 described above can refer to the corresponding process of the aforementioned TSN-based deterministic network delay scheduling method, and will not be repeated here.

[0048] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by one or more processors, the one or more processors implement a deterministic network delay scheduling method based on TSN as provided in an embodiment of the present application.

[0049] The computer-readable storage medium may be an internal storage unit of the TSN-based deterministic network delay scheduling device 300 in the aforementioned embodiment, such as a hard disk or memory of the TSN-based deterministic network delay scheduling device 300. The computer-readable storage medium may also be an external storage device of the TSN-based deterministic network delay scheduling device 300, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped with the TSN-based deterministic network delay scheduling device 300.

[0050] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0051] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0052] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A deterministic network delay scheduling method based on TSN, characterized in that: include: Perform graph analysis on the TSN network topology to obtain network node connection relationships and link bandwidth information; Inputting the network node connection relationship and the link bandwidth information into a multi-level priority-aware network, extracting the delay requirement and bandwidth requirement characteristics of the data flow, and generating a data flow priority queue and a resource requirement index matrix; Input the data flow priority queue and the resource demand index matrix into the deep Q network, establish a state-action mapping relationship including network load state and time slice allocation action, and generate an initial delay scheduling strategy model; According to the initial delay scheduling strategy model, the queuing delay, transmission delay and processing delay of each data flow are calculated to generate a time slice resource reallocation plan; Generate a global resource allocation matrix based on the time slice resource reallocation scheme; A performance evaluation is performed on the global resource allocation matrix to generate a network performance score, and parameter feedback is performed on the initial delay scheduling strategy model according to the network performance score to obtain a target delay scheduling strategy model.

2. The TSN-based deterministic network delay scheduling method according to claim 1, characterized in that: The graph theory analysis of the TSN network topology structure is performed to obtain the network node connection relationship and link bandwidth information, including: Perform topological scan on the TSN network, obtain the number of nodes and node identification information in the network, and obtain the initial node set; Performing connection relationship analysis on the nodes in the initial node set, establishing a node adjacency matrix, obtaining a network connection topology map, and performing minimum spanning tree calculation on the network connection topology map to obtain a network backbone link distribution map; Based on the network backbone link distribution map, sub-network division is performed to obtain a sub-network unit set, and boundary node identification is performed on each sub-network unit in the sub-network unit set to obtain an interconnection relationship matrix between sub-networks; According to the interconnection relationship matrix between the sub-networks, bandwidth capacity of each link is measured to obtain a link bandwidth distribution map; Based on the link bandwidth distribution diagram, a reference time period is calculated, the reference time period is divided into a preset number of time slice units, a time slice division scheme is obtained, and according to the time slice division scheme, initial time slice resources are allocated to each sub-network unit to obtain network node connection relations and link bandwidth information.

3. The TSN-based deterministic network delay scheduling method according to claim 2, characterized in that: The step of inputting the network node connection relationship and the link bandwidth information into a multi-level priority-aware network, extracting the delay requirement and bandwidth requirement characteristics of the data flow, and generating a data flow priority queue and a resource requirement index matrix includes: Performing matrix transformation on the network node connection relationship to generate a network topology feature matrix, and normalizing the link bandwidth information to obtain a bandwidth feature vector; Inputting the network topology feature matrix and the bandwidth feature vector into a feature extraction layer of a multi-level priority perception network, wherein the feature extraction layer comprises three convolutional layers, each convolutional layer uses a ReLU activation function and batch normalization processing to obtain a data stream basic feature map; Inputting the data flow basic feature graph into the traffic classification layer of the multi-level priority perception network, the traffic classification layer includes two fully connected layers and one Softmax layer, classifying the data flow into key business flow, periodic business flow and burst business flow, and obtaining a traffic category identification matrix; Constructing a feature vector for the traffic category identification matrix, extracting parameters of four dimensions, namely, end-to-end delay requirement, bandwidth requirement, burst characteristic and periodic characteristic, to obtain a set of traffic feature vectors; Inputting the traffic feature vector set into the priority mapping layer of the multi-level priority-aware network, the priority mapping layer includes a fully connected layer and a Sigmoid layer, calculating the delay sensitivity weight, the bandwidth requirement weight and the priority level weight, and obtaining the priority quantization parameter; According to the priority quantization parameter, a resource requirement index is calculated for each data flow, where the resource requirement index is obtained by weighted summation of a delay sensitivity weight, a bandwidth requirement weight, and a priority level weight; The resource demand indexes are sorted in descending order according to their numerical values, a data stream priority queue is constructed, and the data stream priority queues are grouped according to a preset time window size to obtain a grouping matrix; Each matrix element in the grouping matrix is ​​normalized to obtain a resource demand index matrix.

4. The TSN-based deterministic network delay scheduling method according to claim 3, characterized in that: The step of inputting the data stream priority queue and the resource demand index matrix into a deep Q network, establishing a state-action mapping relationship including a network load state and a time slice allocation action, and generating an initial delay scheduling strategy model includes: The data stream priority queue and the resource demand index matrix are subjected to feature fusion to construct a state space of a deep Q network and obtain a network state feature matrix; Performing time slice allocation action encoding on the network state feature matrix, dividing the allocation ratio of each time slice into discrete gears, and obtaining an action space matrix; Input the network state feature matrix into the value assessment module of the deep Q network, wherein the value assessment module comprises three fully connected layers, each of which uses a ReLU activation function to obtain a state value assessment result; Constructing an action value function according to the state value evaluation result and the action space matrix, wherein the action value function includes an immediate reward item, a discount factor, and a future reward item; Performing strategy iteration calculation on the action value function, selecting time slice allocation actions based on the ε-greedy strategy, and obtaining the action selection probability distribution of the deep Q network; According to the action selection probability distribution, the parameters of the deep Q network are optimized, and the network weights are updated using the stochastic gradient descent method to obtain a state-action mapping model; The state-action mapping model is input into an experience playback buffer, a state transition sample of each time step is stored, a transition probability matrix is ​​obtained, and the state-action mapping model is online learned based on the transition probability matrix to obtain an initial delay scheduling strategy model.

5. The TSN-based deterministic network delay scheduling method according to claim 4, characterized in that: The step of calculating the queuing delay, transmission delay and processing delay of each data flow according to the initial delay scheduling strategy model and generating a time slice resource reallocation scheme includes: Extract the path hop count, link bandwidth and node processing capability parameters of the data flow according to the initial delay scheduling strategy model to obtain a delay calculation parameter set; Analyze the queue cache data in the delay calculation parameter set, and obtain a data flow queuing delay distribution matrix according to the cache queue length and service processing rate of each node; Based on the data flow queuing delay distribution matrix, the transmission process of the data packet on each link is modeled, the transmission delay is calculated, and the link transmission delay matrix is ​​obtained; According to the link transmission delay matrix, load analysis is performed on the data processing capability of the network node, processing delay is calculated, and a node processing delay matrix is ​​obtained; The data flow queuing delay distribution matrix, the link transmission delay matrix and the node processing delay matrix are superimposed to obtain an end-to-end total delay matrix, and the data flows exceeding the delay threshold in the end-to-end total delay matrix are marked to generate a delay-exceeding traffic set; According to the delay-exceeding traffic set, the time slice reallocation coefficient of each subnetwork is calculated, and the time slice reallocation coefficient is inversely proportional to the end-to-end total delay. A time slice adjustment factor matrix is ​​obtained, and the time slice allocation plan is updated based on the time slice adjustment factor matrix to obtain a time slice resource reallocation plan.

6. The TSN-based deterministic network delay scheduling method according to claim 5, characterized in that: The generating a global resource allocation matrix based on the time slice resource reallocation scheme includes: Performing resource utilization statistics calculation on each sub-network unit in the time slice resource reallocation scheme to obtain a sub-network resource utilization state matrix; Constructing a traffic load balancing index based on the sub-network resource utilization state matrix, analyzing the bandwidth resource distribution of each sub-network unit, and obtaining a sub-network load distribution table; According to the sub-network load distribution table, a sub-network unit whose traffic load balancing index exceeds a preset threshold is selected, and a bandwidth resource gap analysis is performed on the sub-network unit to obtain a bandwidth resource demand matrix; Performing multi-objective optimization calculation on the bandwidth resource demand matrix, inputting bandwidth borrowing cost weight, resource utilization weight and network risk weight into the optimization function, and obtaining a bandwidth borrowing optimization solution; Input the resource allocation parameters in the bandwidth borrowing optimization scheme into a resource allocator, perform bandwidth resource reallocation across sub-networks, obtain a bandwidth reuse matrix, and perform time slice mapping conversion on the bandwidth reuse matrix, convert the bandwidth resources into a time slice resource allocation relationship, and obtain a time slice mapping table; The spatial arrangement of resources of each sub-network is calculated according to the time slice mapping table, a three-dimensional resource allocation tensor is constructed, a resource space distribution map is obtained, and the resource space distribution map is subjected to dimensionality reduction processing, the information of the time dimension, space dimension and resource dimension is integrated, and a global resource allocation matrix is ​​obtained.

7. The TSN-based deterministic network delay scheduling method according to claim 6, characterized in that: The performing performance evaluation on the global resource allocation matrix to generate a network performance score, and performing parameter feedback update on the initial delay scheduling strategy model according to the network performance score to obtain a target delay scheduling strategy model, including: Calculating the performance index of the global resource allocation matrix to obtain a performance data set of three indicators: end-to-end delay compliance rate, link utilization rate and priority satisfaction rate; The performance data set is input into a multi-index comprehensive evaluation module, and a weighted sum is calculated by a weight coefficient α of the end-to-end delay compliance rate, a weight coefficient β of the link utilization rate, and a weight coefficient γ of the priority satisfaction degree to obtain a network performance score; Performing a threshold detection on the network performance score, comparing it with a preset performance benchmark threshold, and obtaining a performance difference matrix; Based on the performance difference matrix, reverse error calculation is performed on the immediate reward item, discount factor and future reward item in the initial delay scheduling strategy model to obtain a parameter gradient matrix; Update the deep Q network weights of the initial delay scheduling strategy model according to the optimization rule of the stochastic gradient descent method using the parameter gradient matrix to obtain updated network weights; Recalculate the delay scheduling of the data flow according to the updated network weight to obtain a new state-action mapping table; The new state-action mapping table is verified online, and the policy optimization result is obtained by calculating the average reward value of the verification sample. The scheduling strategy is confirmed based on the policy optimization result, and the verified scheduling strategy parameters are solidified into the deep Q network to obtain the target delay scheduling strategy model.

8. A deterministic network delay scheduling device based on TSN, characterized in that: The method for executing a TSN-based deterministic network delay scheduling method as claimed in any one of claims 1 to 7 comprises: Graph analysis module, used to perform graph analysis on the TSN network topology to obtain network node connection relationships and link bandwidth information; An extraction module, used to input the network node connection relationship and the link bandwidth information into a multi-level priority-aware network, extract the delay requirement and bandwidth requirement characteristics of the data flow, and generate a data flow priority queue and a resource requirement index matrix; Establish a module for inputting the data flow priority queue and the resource demand index matrix into a deep Q network, establishing a state-action mapping relationship including network load state and time slice allocation action, and generating an initial delay scheduling strategy model; A calculation module, used to calculate the queuing delay, transmission delay and processing delay of each data flow according to the initial delay scheduling strategy model, and generate a time slice resource reallocation plan; A generating module, configured to generate a global resource allocation matrix based on the time slice resource reallocation scheme; The updating module is used to perform performance evaluation on the global resource allocation matrix, generate a network performance score, and perform parameter feedback update on the initial delay scheduling strategy model according to the network performance score to obtain a target delay scheduling strategy model.

9. A deterministic network delay scheduling device based on TSN, characterized in that: The TSN-based deterministic network delay scheduling device includes: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instruction in the memory so that the TSN-based deterministic network delay scheduling device executes the TSN-based deterministic network delay scheduling method as described in any one of claims 1-7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instruction is executed by the processor, a deterministic network delay scheduling method based on TSN as described in any one of claims 1 to 7 is implemented.

Images