Time-sensitive service flow resource adaptation method for large-bandwidth multicast flow perception

Abstract

The invention relates to the technical field of industrial networks, in particular to a time-sensitive service flow resource adaptation method for large-bandwidth multicast flow awareness, which comprises the following steps: S1, a controller collects registered service flow information and real-time network state information; s2, after service flow priorities are divided according to the information, a mapping relation between service flows and multiple queues is established, and a forwarding rule is established according to the mapping relation; s3, establishing an optimization target and a resource constraint according to the mapping relation and the forwarding rule; and S4, designing a Markov decision according to the optimization target and the resource constraint, training the neural network model according to the Markov decision and the double-delay depth deterministic strategy gradient so as to output a scheduling decision, and obtaining a multi-dimensional resource adaptation result by a solver according to the scheduling decision. According to the invention, a basic queue model support is provided for end-to-end deterministic transmission of the unicast stream and the multicast stream through the mapping relation and the forwarding rule, the utilization rate of network resources and the number of service streams capable of being subjected to incremental scheduling are improved, and the worst end-to-end delay of the large-bandwidth multicast stream is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial networks, and particularly to a method, device, equipment, and storage medium for resource adaptation of time-sensitive service flows with large-bandwidth multicast traffic awareness. Background Art

[0002] As a technology that ensures deterministic end-to-end transmission delay, jitter, and high reliability of service flows, time-sensitive networking has been widely applied in many fields such as aerospace, intelligent driving, and smart healthcare. The queue cyclic forwarding mechanism proposed by the IEEE 802.1 TSN working group for time-sensitive networking can effectively achieve deterministic guarantee for hop-by-hop forwarding of time-sensitive service flows. Considering that the cyclic forwarding mechanism only defines the queue model and traffic categories and does not stipulate the in-network scheduling method of traffic, a large amount of existing work further studies the scheduling methods for time-sensitive unicast flows, and proposes algorithms such as satisfiability modulo theory, heuristic queues, and slot search to improve the scheduling performance of service flows. However, in actual industrial scenarios, there are not only time-sensitive unicast flows but also large-bandwidth multicast flows carrying critical data. The existing methods that only target the scheduling of time-sensitive unicast flows cannot meet the requirements of the actual scenario for the joint scheduling of mixed service flows (time-sensitive unicast flows and large-bandwidth multicast flows). Therefore, it is urgent to study the single-multicast mixed service flow scheduling method based on the queue cyclic forwarding mechanism.

[0003] Regarding the research on service flow scheduling algorithms, traditional scheduling algorithms, such as satisfiability modulo theory and heuristic algorithms, are difficult to meet the requirements of online scheduling scenarios for fast and dynamic resource adaptation of service flows. Compared with traditional algorithms, scheduling algorithms based on artificial intelligence have innate advantages in fast reasoning and dynamic decision-making, enabling the network to actively adapt to the deterministic requirements of services. Therefore, it is also necessary to further study the single-multicast mixed service flow scheduling algorithm based on artificial intelligence under the queue cyclic forwarding mechanism to achieve traffic awareness and intelligent network resource adaptation. Summary of the Invention

[0004] The purpose of the present invention is to solve the deficiencies in the prior art, and provides a method for resource adaptation of time-sensitive service flows with large-bandwidth multicast traffic awareness, including the following steps: S1: The controller collects registered service flow information including service flow ID, source address, destination terminal or multicast group address, flow period, packet size, number of packets, service flow end-to-end delay requirement, jitter requirement, and reliability requirement, and collects real-time network status information including network topology structure, link bandwidth resources, and interface queue information; S2: Divide the service flow priorities according to the registered service flow information and the real-time network status information, establish a mapping relationship between the service flows and multiple queues according to the service flow priorities, and establish a forwarding rule according to the mapping relationship; S3: Establish the optimization objective for unicast and multicast traffic co-network scheduling and resource constraints including network scheduling period constraints, transmission time slot constraints, loop-free constraints for the forwarding path and multicast tree, credit value constraints, end-to-end delay, jitter, and reliability constraints of the service flow according to the mapping relationship and the forwarding rules; S4: Design a Markov decision according to the optimization objective and the resource constraints, train a neural network model according to the Markov decision and the twin-delayed deep deterministic policy gradient, obtain a scheduling decision according to the trained neural network output, and the solver obtains a multi-dimensional resource adaptation result according to the scheduling decision.

[0005] Preferably, in step S2, further including establishing the forwarding rules according to the mapping relationship: Map the queue of high-priority service flows to execute a loop forwarding mechanism, that is, within each transmission time slot, check the queue responsible for sending high-priority service flows. If there are high-priority service flows waiting to be sent, send the high-priority service flows to the link and receive the high-priority service flows through the queue responsible for receiving the high-priority service flows, so as to realize the alternating switching of the sending and receiving queues in different time slots, where the high-priority service flow is a unicast time-sensitive service flow; Map the queue of low-priority service flows to execute a credit-based shaping mechanism to avoid the starvation phenomenon of multicast traffic, and transmit the service flow according to the credit-based shaping mechanism, where the low-priority service flow is a large-bandwidth multicast service flow.

[0006] Preferably, in step S2, further including the queue of the low-priority service flow executing the credit-based shaping mechanism: Set the initial credit values of the queue for sending low-priority service flows and the queue for receiving the low-priority service flows to 0, and set the sending rate parameter sending slope and the idle rate parameter idle slope; Detect the queue of the low-priority service flow, and judge whether there are low-priority service flows waiting to be sent in the queue of the low-priority service flow. When the queue of the low-priority service flow sends the low-priority service flow, consume the credit value at the rate of sending slope, that is, credit value = credit value - sending slope * sending time. When the low-priority service flow is waiting in the queue of the low-priority service flow, accumulate the credit value at the rate of idle slope, that is, credit value = credit value + idle slope * waiting time; Among them, when the gating of the queue of the low-priority service flow is closed, the credit value is frozen, and only when the credit value is non-negative is the queue of the low-priority service flow allowed to send the low-priority service flow.

[0007] Preferably, in step S3, establishing an optimization objective for unicast and multicast traffic co-network scheduling according to the mapping relationship and the forwarding rule further includes: The optimization objective includes maximizing the incrementally schedulable number of hybrid flows and minimizing the worst end-to-end transmission delay of low-priority traffic flows. Based on the mapping relationship and the forwarding rule, the optimization objective is obtained through the set of successfully scheduled traffic flows and the worst end-to-end transmission delay. The calculation formula is as follows: maxα×|Fsuc|-β×WCD(fi) Where, is a weight factor, is the set of all successfully scheduled traffic flows, and WCD(fi) is used to count the worst end-to-end transmission delay of low-priority traffic flows.

[0008] Preferably, in step S3, establishing resource constraints including network scheduling period constraints, transmission time slot constraints, loop-free constraints for forwarding paths and multicast trees, credit value constraints, traffic flow end-to-end delay, jitter, and reliability constraints further includes: The network scheduling period constraint is to obtain the length of a scheduling period by calculating the least common multiple of the periods of all traffic flows , so as to execute the same resource adaptation mechanism within each scheduling period. The calculation formula is as follows: Where, LCM(F.period) is the least common multiple of the periods of all traffic flows; The transmission time slot constraint requires that each transmission time slot should be able to accommodate at least all the traffic flows sent in a queue at a minimum, and the maximum transmission time slot length should not exceed the greatest common divisor of the periods of all traffic flows. The calculation formula is as follows: Where, represents the maximum number of bytes of traffic flows that can be sent in a queue, is the link bandwidth, is the transmission time slot length, is used to calculate the greatest common divisor of traffic flow periods; The loop-free constraint for the forwarding path and the multicast tree requires that the forwarding path selected by high-priority traffic flows is loop-free, and the multicast tree path used by low-priority traffic flows is also loop-free, that is , where represents a link loop, is the resource set composed of the selected paths; The credit value constraint requires that the idle slope rate is positively correlated with the link bandwidth and does not exceed the bandwidth size, that is Where, , the sending slope rate is equal to the absolute value of the difference between the idle slope and the link bandwidth, that is, ; The end-to-end delay, jitter and reliability constraints of the service flow require that each high-priority service flow transmission has bounded delay, jitter and zero packet loss characteristics, and the worst end-to-end delay of the low-priority service flow does not exceed the set threshold. The calculation formula is as follows: in, , They represent the transmission delay of high priority traffic on each node and each link respectively. is the theoretical upper limit of end-to-end delay, is the jitter of the high-priority service flow, is the theoretical upper limit of jitter, is the low priority end-to-end delay threshold, is the packet loss rate passing through each node, The calculation formula is as follows: in, and Respectively represent the number of data packets sent by the outbound interface of the switch node and the number of data packets received by the inbound interface.

[0009] Preferably, in step S4, designing a Markov decision according to the optimization objective and the resource constraint further includes: According to the link bandwidth resource utilization rate in the current transmission time slot And the business flow attributes of online incremental access Get the current status , according to the current state of each transmission slot Get the state space , the calculation formula is as follows: in, Indicates the current state. It represents the link bandwidth resource utilization rate corresponding to the jth interface on the kth switch node in the nth time slot; The transmission time slot size decision of each step is based on satisfying the target optimization and the resource constraints , Multicast Tree Decision , Unicast Stream Forwarding Path Decision , Mixed flow sending time slot decision Obtain the current action , and based on the current action of each step Obtain the action space , and the calculation formula is as follows: Obtain the scheduling success rate feedback based on each step and the end-to-end worst-case delay feedback Obtain the reward value of the Markov decision process , and the calculation formula is as follows: wherein, is the weight coefficient; The said scheduling success rate feedback and the said end-to-end worst-case delay feedback The calculation formula is as follows: wherein, is the discount factor required for different decision steps.

[0010] Preferably, further including according to the Markov decision and the double-delay deep deterministic policy gradient to train the neural network model: The action decision neural network in the neural network model makes a decision on the service flow scheduling action based on the current state , and at the same time superimposes random noise with a mean of 0 and a variance of on the action, that is , , ; The controller executes the action to obtain the immediate reward of the environmental feedback and the next state , and stores the trajectory sample obtained by each step of transfer into the experience pool; After each step of service flow scheduling exploration and sampling, the controller batch-collects a number of trajectory samples from the experience pool, inputs the trajectory samples into the evaluation neural network in the neural network model to calculate the Q value, and constructs the target value of the state-action value using the minimum value in the Q value , and the calculation formula is as follows: wherein, the clipping function controls the noise value within to avoid excessive training jitter, and updates the parameters of the evaluation neural network according to the constructed label value , and its principle is to calculate the mean square error between the target value of the state-action value and the estimated Q value , and then use the gradient descent method to optimize the parameters; After the evaluation neural network in the neural network model is stable, update the parameters of the action decision neural network by maximizing the estimated Q value through the gradient ascent method, and optimize the parameters of the target neural network by soft update; The formula for updating the parameters of the action decision neural network by maximizing the estimated Q value through the gradient ascent method is as follows: where, are the parameters of the action decision neural network, are the parameters of the evaluation neural network; Optimize the parameters of the target neural network by soft update, and the calculation formula is as follows: where, is a hyperparameter, are the parameters of the evaluation neural network before update, are the new parameters of the evaluation neural network, are the parameters of the action decision neural network before update, are the new parameters of the action decision neural network.

[0011] Based on the same concept, the present invention also provides a method and device for resource adaptation of time-sensitive service flows with large-bandwidth multicast traffic awareness, including: A collection and detection module, used for the controller to collect registration service flow information including service flow ID, source address, destination terminal or multicast group address, flow period, packet size, number of packets, service flow end-to-end delay requirement, jitter requirement and reliability requirement, and collect real-time network status information including network topology structure, link bandwidth resources, interface queue information; A multi-queue single-multicast traffic shaping module, used to divide service flow priorities according to the registration service flow information and the real-time network status information, establish a mapping relationship between service flows and multi-queues according to the service flow priorities, and establish forwarding rules according to the mapping relationship to achieve multi-flow differentiated deterministic transmission at the bottom layer; A service flow multi-dimensional resource adaptation module, used to establish an optimization goal for co-network scheduling of single-multicast traffic and resource constraints including network scheduling period constraints, transmission time slot constraints, forwarding path and multicast tree acyclic constraints, credit value constraints, service flow end-to-end delay, jitter and reliability constraints according to the mapping relationship and the forwarding rules, so as to achieve constraint optimization of multi-dimensional resource adaptation of single-multicast service flows with network routing, queues and time slots; A Markov decision module, used to design a Markov decision according to the optimization goal and the resource constraints, and provide a theoretical support for the design of intelligent scheduling algorithms; The time-sensitive service flow scheduling module is used to train a neural network model according to the Markov decision and the twin-delayed deep deterministic policy gradient, and output a scheduling decision according to the trained neural network, so as to realize the intelligent scheduling of the co-network transmission of single unicast and multicast service flows; The solution module is used to obtain a multi-dimensional resource adaptation result according to the scheduling decision, so as to realize the efficient adaptation problem between the service flow and the multi-dimensional resources.

[0012] Based on the same concept, the present invention also provides a computer device, including a memory and a processor. Computer-readable instructions are stored in the memory. When the computer-readable instructions are executed by the processor, the processor is caused to execute the steps of the method for adapting time-sensitive service flow resources with large-bandwidth multicast traffic perception as described in any one of the embodiments.

[0013] Based on the same concept, the present invention also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, one or more processors are caused to execute the steps of the method for adapting time-sensitive service flow resources with large-bandwidth multicast traffic perception as described in any one of the embodiments.

[0014] Compared with the prior art, the beneficial effects of the present invention are: The present invention establishes an optimization objective and resource constraints, and reasonably allocates service flows to links with idle bandwidth, thereby improving the resource utilization rate of the entire network, reducing the idle and waste of resources, avoiding the problems of over-allocation or under-allocation that may occur in traditional network resource allocation, making the network resources more fully utilized, and improving the overall performance of the network.

[0015] By establishing a multi-queue single multicast traffic shaping module, the present invention provides a basic queue model support for the end-to-end deterministic transmission of single multicast flows, which can effectively improve the network resource utilization rate and the number of service flows that can be incrementally scheduled, and at the same time reduce the worst end-to-end delay of large-bandwidth multicast flows. Description of the Drawings

[0016] By reading the detailed description of the preferred embodiments below, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present invention.

[0017] Figure 1 It is a flowchart of the method for adapting time-sensitive service flow resources with large-bandwidth multicast traffic perception of the present invention; Figure 2 It is an application scenario diagram of the method for adapting time-sensitive service flow resources with large-bandwidth multicast traffic perception of the present invention; Figure 3This is the structural block diagram of the large-bandwidth multicast traffic-aware time-sensitive service flow resource adaptation device of the present invention. Detailed implementation mode

[0018] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the scope of protection of the present application.

[0019] Those skilled in the art of the present technology can understand that, unless specifically stated otherwise, the singular forms "a", "an", and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present invention means the presence of the described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0020] First embodiment Please refer to Figure 1 As shown, in order to improve the efficiency of online resource adaptation of single multicast hybrid service flows in a time-sensitive network, this embodiment provides a time-sensitive service flow resource adaptation method for large-bandwidth multicast traffic awareness, including the following steps: S1: The controller collects registration service flow information including service flow ID, source address, destination terminal or multicast group address, flow period, packet size, number of packets, service flow end-to-end delay requirement, jitter requirement, and reliability requirement, and collects real-time network status information including network topology structure, link bandwidth resources, and interface queue information. Specifically, in this embodiment, the controller sends specific probe messages (such as LLDP packets, ICMP packets, etc.) to each node (such as routers, switches, etc.) in the network. After receiving the probe messages, the network nodes return response messages containing their own information (such as device identification, port information, etc.) and adjacent node information according to their own configurations and capabilities. The controller collects the response messages of all nodes, constructs the network topology structure by analyzing the connection relationships between the nodes, and represents it as a set of acyclic paths and acyclic multicast tree resources. Graph theory algorithms (such as depth-first search, breadth-first search) can be used to detect and eliminate loops to ensure the acyclicity of the topology structure; After collecting the registration service flow information and real-time network status information, establishing a display network resource representation according to the registration service flow information and real-time network status information further includes: Establish an acyclic path and an acyclic multicast tree according to the network topology, and establish a resource set of the acyclic path and the acyclic multicast tree based on the acyclic path and the acyclic multicast tree; Establish a bandwidth resource utilization set according to the link bandwidth resources; Map according to the number of interface queues in the interface queue information and the traffic priority in the interface queue information to establish a resource mapping set of the queue number and the priority.

[0021] Set a certain time interval (such as every minute, every hour), and regularly detect and update the network status to ensure the real-time nature of the network resource representation.

[0022] S2: Divide the traffic priorities according to the registered traffic information and the real-time network status information, establish the mapping relationship between the traffic and the multi-queues according to the traffic priorities, and establish the forwarding rules according to the mapping relationship. Specifically, in this embodiment, the time-sensitive unicast traffic is set as the high priority, and the large-bandwidth multicast traffic is set as the low priority to divide the traffic priorities, so as to preferentially ensure the highly reliable transmission of the time-sensitive unicast traffic. Establish the mapping relationship between the unicast and multicast hybrid traffic and the multi-queues according to the traffic priorities, that is, the high-priority time-sensitive unicast traffic is mapped to the first two queues, and the low-priority large-bandwidth multicast traffic is mapped to the last two queues based on different categories.

[0023] Preferably, in step S2, establishing the forwarding rules according to the mapping relationship further includes: The queues mapping the high-priority traffic execute a cyclic forwarding mechanism, that is, within each transmission time slot, check the queue responsible for sending the high-priority traffic. If there is high-priority traffic waiting to be sent, send the high-priority traffic to the link, and receive the high-priority traffic through the queue responsible for receiving the high-priority traffic, so as to achieve the alternating switching of the sending and receiving queues in different time slots. Specifically, in this embodiment, the two queues processing the time-sensitive unicast traffic execute a cyclic forwarding strategy, that is, within each transmission time slot, one queue is responsible for sending the traffic, and the other queue is responsible for receiving the traffic, and the sending and receiving queues alternate in different time slots; The queues mapping the low-priority traffic execute a credit-based shaping mechanism to avoid the starvation phenomenon of multicast traffic, and transmit the traffic according to the credit-based shaping mechanism. Specifically, in this embodiment, the two queues processing the large-bandwidth multicast traffic execute a credit-based shaping strategy.

[0024] Preferably, in step S2, the queues mapping the low-priority traffic executing the credit-based shaping mechanism further includes: Set the initial credit values of the queue for sending low-priority traffic flows and the queue for receiving low-priority traffic flows to 0, and set the sending rate parameter sending slope and the idle rate parameter idle slope. Detect the queue of low-priority traffic flows, and determine whether there are low-priority traffic flows waiting to be sent in the queue of low-priority traffic flows. When the queue of low-priority traffic flows sends low-priority traffic flows, consume the credit value at the rate of sending slope, that is, credit value = credit value - sending slope * sending time. When the low-priority traffic flow is waiting in the queue of low-priority traffic flows, accumulate the credit value at the rate of idle slope, that is, credit value = credit value + idle slope * waiting time. Among them, when the gating of the queue of low-priority traffic flows is closed, the credit value is frozen. Only when the credit value is non-negative is the queue of low-priority traffic flows allowed to send low-priority traffic flows. To ensure the determinism of the time-sensitive unicast traffic flow, the gating of other queues needs to be closed during the transmission of the time-sensitive unicast traffic flow to avoid multicast flows preempting link bandwidth resources.

[0025] S3: Establish the optimization objective of co-network scheduling for single multicast traffic flows and resource constraints including network scheduling period constraints, transmission time slot constraints, acyclic constraints of forwarding paths and multicast trees, credit value constraints, end-to-end delay, jitter, and reliability constraints of traffic flows according to the mapping relationship and forwarding rules. Specifically, in this embodiment, by establishing the optimization objective and resource constraints, maximize the accommodation of traffic flows in the industrial scenario, that is, calculate the traffic flows on each loan to prevent the traffic flows converging on each link from exceeding the bandwidth of that link.

[0026] Preferably, in step S3, establishing the optimization objective of co-network scheduling for single multicast traffic flows according to the mapping relationship and forwarding rules further includes: The optimization objective includes maximizing the incremental schedulable quantity of mixed traffic flows and minimizing the worst end-to-end transmission delay of low-priority traffic flows. Based on the mapping relationship and forwarding rules, obtain the optimization objective through the set of successfully scheduled traffic flows and the worst end-to-end transmission delay. The calculation formula is as follows: maxα×|Fsuc|-β×WCD(fi) Among them, is the weight factor, is the set of all successfully scheduled traffic flows, is used to statistically calculate the worst end-to-end transmission delay of low-priority traffic flows. Specifically, in this embodiment, the optimization objective realizes minimizing the worst end-to-end transmission delay of multicast flows.

[0027] Preferably, in step S3, resource constraints including network scheduling period constraint, transmission time slot constraint, loop-free constraint of forwarding path and multicast tree, credit value constraint, end-to-end delay, jitter and reliability constraints of traffic flow are established according to the mapping relationship and forwarding rules, further including: The network scheduling period constraint is to obtain the length of a scheduling period by calculating the least common multiple of all traffic flow periods , so as to execute the same resource adaptation mechanism within each scheduling period. The calculation formula is as follows: where LCM(F.period) is the least common multiple of all traffic flow periods; The transmission time slot constraint requires that each transmission time slot should be able to accommodate at least all the traffic flows in a queue for sending, and the maximum transmission time slot length does not exceed the greatest common divisor of all traffic flow periods. The calculation formula is as follows: where represents the maximum number of bytes of traffic flow that a queue can send, is the link bandwidth, is the transmission time slot length, is used to calculate the greatest common divisor of traffic flow periods; The loop-free constraint of the forwarding path and the multicast tree requires that the forwarding path selected by high-priority traffic flows (time-sensitive unicast flows) is loop-free, and the multicast tree path adopted by low-priority traffic flows (multicast flows) is also loop-free, that is where represents the link loop, is the resource set composed of the selected path; The credit value constraint requires that the idle slope rate is positively correlated with the link bandwidth and does not exceed the bandwidth size, that is where , and the sending slope rate is equal to the absolute value of the difference between the idle slope and the link bandwidth, that is ; The end-to-end delay, jitter and reliability constraints of traffic flow require that each high-priority traffic flow (time-sensitive unicast flow) has bounded delay, jitter and zero packet loss characteristics during transmission, and the worst end-to-end delay of low-priority traffic flows (large-bandwidth multicast flows) does not exceed the set threshold. The calculation formula is as follows: where , respectively represent the delay of high-priority traffic flows (time-sensitive unicast flows) during transmission at each node and each link, is the theoretical upper limit value of the end-to-end delay, is the jitter of high-priority service flows (time-sensitive unicast flows), is the theoretical upper limit of jitter, is the low priority end (large bandwidth multicast stream) to end delay threshold. is the packet loss rate passing through each node, The calculation formula is as follows: in, and Respectively represent the number of data packets sent by the outbound interface of the switch node and the number of data packets received by the inbound interface.

[0028] S4: Design a Markov decision according to the optimization goal and resource constraints, train the neural network model according to the Markov decision and double-delay deep deterministic policy gradient, output the scheduling decision based on the trained neural network, and the solver obtains the multi-dimensional resource adaptation result according to the scheduling decision. Specifically, in this embodiment, the problem of non-uniform overestimation of traditional deep reinforcement learning is avoided, and intelligent single-multicast hybrid service flow scheduling is implemented based on double-delay deep deterministic policy gradient.

[0029] Preferably, in step S4, designing a Markov decision according to the optimization objective and resource constraints further includes: According to the link bandwidth resource utilization rate in the current transmission time slot And the business flow attributes of online incremental access Get the current status , according to the current state of each transmission slot Get the state space , that is, the action that satisfies the target optimization and resource constraints, the calculation formula is as follows: in, Indicates the current state. Indicates the link bandwidth resource utilization rate corresponding to the jth interface on the kth switch node in the nth time slot. Specifically, in this embodiment, the elements of the service flow state set correspond to the service flow ID, source address, destination terminal or multicast group address, flow period, data packet size, number of data packets, end-to-end delay, jitter and reliability deterministic indicator requirements of the service flow added to the network online incrementally at each step; The transmission slot size decision is made at each step according to the optimization goal and resource constraints. , Multicast Tree Decision , Unicast Stream Forwarding Path Decision 、Mixed-flow transmission time slot decision Obtain the current action According to the current action of each step Obtain the action space The calculation formula is as follows: Obtain the scheduling success rate feedback according to each step And the end-to-end worst delay feedback Obtain the reward value of the Markov decision process The calculation formula is as follows: Among them, is the weight coefficient; Scheduling success rate feedback And the end-to-end worst delay feedback The calculation formula is as follows: Among them, is the discount factor required for different decision steps.

[0030] The execution steps of unicast and multicast hybrid traffic flow scheduling include algorithm initialization, traffic flow scheduling exploration and sampling, neural network parameter update, and resource adaptation online decision-making.

[0031] During the algorithm initialization process, establish an action decision neural network , a judgment neural network And , a target action decision neural network , a target judgment neural network And A total of 6 neural networks, for the neural network parameters , , Perform random initialization, and assign the initialization values to the corresponding target neural networks in sequence, that is , , , and initialize the traffic flow information set and the network resource status matrix.

[0032] Preferably, further including training the neural network model according to Markov decision and double-delay deep deterministic policy gradient: During the traffic flow scheduling exploration and sampling process, the action decision neural network in the neural network model makes traffic flow scheduling actions based on the current state , and at the same time superimpose random noise with a mean of 0 and a variance of for the action, that is ​, ; The controller executes an action to obtain an immediate reward for the environmental feedback and the next state , and stores the trajectory samples obtained from each step of the transfer into the experience pool; After each step of business flow scheduling exploration and sampling, a neural network parameter update process is required. The controller batch-collects several (M) trajectory samples from the experience pool, inputs the trajectory samples into the evaluation neural network (two evaluation neural networks) in the neural network model to calculate Q values (two Q values), and constructs the target value of the state-action value using the minimum value in the Q values (i.e., the smaller value of the two Q values) , and the calculation formula is as follows: Among them, the clipping function controls the noise value within to avoid excessive training jitter, and updates the parameters of the evaluation neural network according to the constructed label value , and its principle is to calculate the mean square error between the target value of the state-action value and the estimated Q value , and then uses the gradient descent method to optimize the parameters; When the evaluation neural network in the neural network model is stable, the parameters of the action decision neural network are updated by maximizing the estimated Q value through the gradient ascent method, and the parameters of the target neural network are optimized by the soft update method; The formula for updating the parameters of the action decision neural network by maximizing the estimated Q value through the gradient ascent method is as follows: Among them, is the parameter of the action decision neural network, is the parameter of the evaluation neural network; The parameters of the target neural network are optimized by the soft update method, and the calculation formula is as follows: Among them, is the hyperparameter, is the parameter of the evaluation neural network before update, is the parameter of the new evaluation neural network, is the parameter of the action decision neural network before update, is the parameter of the new action decision neural network.

[0033] Based on the trained neural network parameters, the controller makes an online scheduling decision for the time-sensitive service flow of multicast traffic awareness, and issues a signaling for single multicast service flow resource adaptation to the network data plane according to the scheduling decision, completing fast and efficient resource adaptation.

[0034] Please refer to Figure 2 As shown, an application scenario of an embodiment of the present invention is described. This scenario is based on a fully centralized control mode, and the global controller uniformly schedules and manages the service flow information registered on the user side and the network side resources. The user registers the single multicast hybrid flow information with the CUC centralized user configuration component in the controller. The packet size of the time-sensitive unicast flow is between 50B and 1KB, and the packet size of the large-bandwidth multicast flow is between 1KB and 1.5KB. The service flow period includes 400us, 1600us, and 3200us. The network link bandwidth is set to 1Gbps. Both the terminal and the switch support the multi-queue single multicast traffic shaping strategy. The CNC component of the controller collects the network side status information in real time through protocols such as link detection. According to the collected service flow and network status information, the CNC uses the time-sensitive service flow scheduling algorithm with multicast traffic awareness deployed on it to decide the hybrid service flow scheduling process, and sends the scheduling decision signaling to the network side to execute the corresponding service flow sending, gating switch, and resource adaptation tasks. The scheduling algorithm updates the neural network using the Adam optimizer, the learning rate is set to 1×10 -4 , the experience pool size is set to 1024, 32 trajectory samples are sampled each time for training, and the discount factor is set to 0.95. The action decision neural network and the evaluation neural network have the same structure as their respective target action decision neural network and target evaluation neural network. In this embodiment, the time-sensitive network topology and the neural network structure of the algorithm in the online resource adaptation scenario of the time-sensitive service flow with large-bandwidth multicast traffic awareness are not specifically limited.

[0035] Second Embodiment Please refer to Figure 3 As shown, based on the same concept, the present invention also provides a time-sensitive service flow resource adaptation method and device with large-bandwidth multicast traffic awareness, including: A collection and detection module, configured to collect, by the controller, the registered service flow information including service flow ID, source address, destination terminal or multicast group address, flow period, packet size, number of packets, service flow end-to-end delay requirement, jitter requirement, and reliability requirement, and collect the real-time network status information including network topology structure, link bandwidth resources, and interface queue information; A multi-queue single multicast traffic shaping module, configured to divide the service flow priorities according to the registered service flow information and the real-time network status information, establish a mapping relationship between the service flow and the multi-queue according to the service flow priorities, and establish a forwarding rule according to the mapping relationship to implement underlying support for multi-flow differentiated deterministic transmission; The service flow multi-dimensional resource adaptation module is used to establish the optimization objective of unicast / multicast traffic co-network scheduling and resource constraints including network scheduling period constraints, transmission time slot constraints, loop-free constraints of forwarding paths and multicast trees, credit value constraints, end-to-end delay, jitter, and reliability constraints of service flows according to the mapping relationship and the forwarding rules, so as to realize the constraint optimization of the adaptation of unicast / multicast service flows to multi-dimensional resources including network routing, queues, and time slots. The Markov decision module is used to design a Markov decision according to the optimization objective and the resource constraints, providing a theoretical support for the design of intelligent scheduling algorithms. The time-sensitive service flow scheduling module is used to train a neural network model according to the Markov decision and the twin-delayed deep deterministic policy gradient, and output a scheduling decision according to the trained neural network to realize the intelligent scheduling of the fusion co-network transmission of unicast / multicast service flows. The solution module is used to obtain the multi-dimensional resource adaptation result according to the scheduling decision, so as to realize the efficient adaptation problem between service flows and multi-dimensional resources.

[0036] The third embodiment In some embodiments of the present application, a computer device is further provided, including a memory and a processor. Computer-readable instructions are stored in the memory. When the computer-readable instructions are executed by the processor, the processor is caused to execute the steps of the time-sensitive service flow resource adaptation method for large-bandwidth multicast traffic perception as described in any one of the embodiments.

[0037] The present invention also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors are caused to execute the steps of the time-sensitive service flow resource adaptation method for large-bandwidth multicast traffic perception as described in any one of Embodiment 1.

[0038] Based on the same concept, the present invention also provides a computer device, including a memory and a processor. Computer-readable instructions are stored in the memory. When the computer-readable instructions are executed by the processor, the processor is caused to execute the steps of the time-sensitive service flow resource adaptation method for large-bandwidth multicast traffic perception as described in any one of the embodiments.

[0039] It can be understood that for the foregoing method for adapting resources of time-sensitive service flows with large-bandwidth multicast traffic awareness, when it is implemented in the form of software functional modules and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer server, a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The foregoing storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, optical disks, and other various media that can store program codes.

[0040] A computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium can also be any readable medium other than the readable storage medium, and this readable medium can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination of the above.

[0041] The above are only the preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions falling within the concept of the present invention belong to the protection scope of the present invention. It should be noted that for those of ordinary skill in the art, several improvements and refinements made without departing from the principle of the present invention should also be regarded as within the protection scope of the present invention.

Claims

1. A large-bandwidth multicast traffic-aware time-sensitive service flow resource adaptation method, characterized in that: The following steps are involved: S1: The controller collects registered service flow information including service flow ID, source address, destination terminal or multicast group address, flow period, data packet size, number of data packets, service flow end-to-end delay requirement, jitter requirement and reliability requirement, as well as real-time network status information including network topology, link bandwidth resources and interface queue information; S2: Divide the service flow priority according to the registered service flow information and the real-time network status information, establish a mapping relationship between the service flow and the multiple queues according to the service flow priority, and establish forwarding rules according to the mapping relationship; S3: establishing the optimization target of single-multicast traffic co-network scheduling and resource constraints including network scheduling cycle constraints, transmission time slot constraints, forwarding path and multicast tree acyclic constraints, credit value constraints, service flow end-to-end delay, jitter and reliability constraints according to the mapping relationship and the forwarding rule; S4: Design a Markov decision according to the optimization objective and the resource constraint, train a neural network model according to the Markov decision and double-delay deep deterministic policy gradient, output a scheduling decision according to the trained neural network, and the solver obtains a multi-dimensional resource adaptation result according to the scheduling decision.

2. The method for adapting time-sensitive service flow resources based on large bandwidth multicast traffic perception according to claim 1 is characterized in that: In step S2, establishing a forwarding rule according to the mapping relationship further includes: The queue mapped to the high-priority service flow executes a cyclic forwarding mechanism, that is, in each transmission time slot, the queue responsible for sending the high-priority service flow is checked. If there is the high-priority service flow waiting to be sent, the high-priority service flow is sent to the link, and the high-priority service flow is received by the queue responsible for receiving the high-priority service flow, so as to realize the alternating switching of the sending and receiving queues in different time slots, wherein the high-priority service flow is a unicast time-sensitive service flow; The queue mapped to the low-priority service flow executes a credit-based shaping mechanism to avoid multicast traffic starvation, and transmits the service flow according to the credit-based shaping mechanism, wherein the low-priority service flow is a large-bandwidth multicast service flow.

3. The method for adapting time-sensitive service flow resources based on large bandwidth multicast traffic perception according to claim 2 is characterized in that: In step S2, the queue mapped to the low priority service flow executes a credit-based shaping mechanism further comprising: Set the initial credit value of the queue for sending low priority service flow and the queue for receiving the low priority service flow to 0, and set the sending rate parameter sending slope and the idle rate parameter idle slope; Detect the queue of the low-priority service flow, determine whether the queue of the low-priority service flow has a low-priority service flow waiting to be sent, when the queue of the low-priority service flow sends the low-priority service flow, consume credit value at the rate of sendingslope, that is, credit value = credit value - sending slope * sending time, when the low-priority service flow waits in the queue of the low-priority service flow, accumulate credit value at the rate of idle slope, that is, credit value = credit value + idle slope * waiting time; When the gating of the queue of the low-priority service flow is closed, the credit value is frozen, and the queue of the low-priority service flow is allowed to send the low-priority service flow only when the credit value is non-negative.

4. The method for adapting time-sensitive service flow resources based on large bandwidth multicast traffic perception according to claim 3 is characterized in that: In step S3, establishing an optimization target for co-network scheduling of unicast and multicast traffic according to the mapping relationship and the forwarding rule further includes: The optimization objectives include maximizing the number of hybrid flows that can be incrementally scheduled and minimizing the worst end-to-end transmission delay of low-priority service flows. Based on the mapping relationship and the forwarding rule, the optimization objectives are obtained by the set of successfully scheduled service flows and the worst end-to-end transmission delay. The calculation formula is as follows: maxα×|F suc |-β×WCD(f i ) Among them, α, β∈[0,1] are weight factors, F suc is the set of all successfully scheduled business flows, WCD(f i ) is used to calculate the worst end-to-end transmission delay of low-priority service flows.

5. The method for adapting time-sensitive service flow resources based on large bandwidth multicast traffic perception according to claim 4 is characterized in that: In step S3, resource constraints including network scheduling cycle constraints, transmission time slot constraints, forwarding path and multicast tree loop constraints, credit value constraints, service flow end-to-end delay, jitter and reliability constraints are established according to the mapping relationship and the forwarding rule, further including: The network scheduling cycle constraint is to obtain the length P of a scheduling cycle by calculating the common multiples of all service flow cycles, so as to execute the same resource adaptation mechanism in each scheduling cycle. The calculation formula is as follows: P=LCM(F.period) Among them, LCM (F.period) is the common multiple of all service flow periods; The transmission time slot constraint requires that each transmission time slot should be able to accommodate all service flows in a queue at least, and the maximum transmission time slot length should not exceed the greatest common divisor of all service flow cycles. The calculation formula is as follows: Among them, Que f Indicates the maximum amount of business flow bytes that a queue can send, B l is the link bandwidth, T is the transmission time slot length, and GCD(·) is used to calculate the greatest common divisor of the service flow period; The forwarding path and multicast tree loop-free constraint requires that the forwarding path selected by the high-priority service flow is loop-free, and the multicast tree path used by the low-priority service flow is also loop-free, that is, Where loop represents the link loop, and R is the resource set consisting of the selected path; The credit value constraint requirement is that the idle slope rate is positively correlated with the link bandwidth and does not exceed the bandwidth size, that is, idle slope = δ × B l , where δ∈(0,1], the sending slope rate is equal to the absolute value of the difference between the idle slope and the link bandwidth, that is, sending slope=|idle slope-B l |; The end-to-end delay, jitter and reliability constraints of the service flow require that each high-priority service flow transmission has bounded delay, jitter and zero packet loss characteristics, and the worst end-to-end delay of the low-priority service flow does not exceed the set threshold. The calculation formula is as follows: in, Denote the transmission delay of high priority traffic flow on each node and each link, D i is the theoretical upper limit of end-to-end delay, jitter (i) is the jitter of the high-priority service flow, S i is the theoretical upper limit of jitter, Θ is the low priority end-to-end delay threshold, is the packet loss rate passing through each node, The calculation formula is as follows: in, and Respectively represent the number of data packets sent by the outbound interface of the switch node and the number of data packets received by the inbound interface.

6. The method for adapting time-sensitive service flow resources based on large bandwidth multicast traffic perception according to claim 5, characterized in that: In step S4, designing a Markov decision according to the optimization objective and the resource constraint further includes: According to the link bandwidth resource utilization rate in the current transmission time slot And the business flow attributes of online incremental access Get the current status , according to the current state of each transmission slot Get the state space , the calculation formula is as follows: in, Indicates the current state. It represents the link bandwidth resource utilization rate corresponding to the jth interface on the kth switch node in the nth time slot; The transmission time slot size decision of each step is based on satisfying the target optimization and the resource constraints , Multicast Tree Decision , Unicast Stream Forwarding Path Decision , Mixed flow sending time slot decision Get the current action , according to the current action of each step Get the action space , the calculation formula is as follows: Get scheduling success rate feedback based on each step and end-to-end worst-case delay feedback Get the reward value of the Markov decision process , the calculation formula is as follows: in, is the weight coefficient; The scheduling success rate feedback and the end-to-end worst-case delay feedback The calculation formula is as follows: in, is the discount factor required for different decision steps.

7. The method for adapting time-sensitive service flow resources based on large bandwidth multicast traffic perception according to claim 6 is characterized in that: The neural network model trained according to the Markov decision and double-delayed deep deterministic policy gradient further includes: The action decision neural network in the neural network model is based on the current state Decision-making business flow scheduling action , and the mean value of the action superposition is 0, and the variance is The random noise is , ; The controller performs actions and receives immediate rewards from the environment and the next state , and transfer the trajectory samples obtained at each step Stored in the experience pool; After each step of business flow scheduling exploration and sampling, the controller collects a number of trajectory samples from the experience pool in batches, inputs the trajectory samples into the judgment neural network in the neural network model to calculate the Q value, and uses the minimum value of the Q value to construct the target value of the state action value. , the calculation formula is as follows: Among them, the clipping function Keep the noise level within To avoid excessive training jitter, update the parameters of the judgment neural network according to the constructed label value , the principle is to calculate the mean square error between the target value of the state action value and the estimated Q value , and then use the gradient descent method to optimize the parameters; When the judgment neural network in the neural network model is stable, the action decision neural network parameters are updated by maximizing the estimated Q value through the gradient ascent method, and the target neural network parameters are optimized through soft updating; The formula for updating the action decision neural network parameters by maximizing the estimated Q value through the gradient ascent method is as follows: in, are the action decision neural network parameters, To judge the neural network parameters; The target neural network parameters are optimized by soft updating. The calculation formula is as follows: in, is a super parameter, To judge the neural network parameters before updating, is the new judgment neural network parameter, To update the parameters of the previous action decision neural network, Decide on neural network parameters for new actions.

8. A method and device for adapting time-sensitive service flow resources based on large bandwidth multicast traffic perception, characterized in that: include: The collection and detection module is used by the controller to collect registered service flow information including service flow ID, source address, destination terminal or multicast group address, flow period, data packet size, number of data packets, service flow end-to-end delay requirement, jitter requirement and reliability requirement, as well as real-time network status information including network topology, link bandwidth resources and interface queue information; A multi-queue single multicast traffic shaping module, used to divide the service flow priority according to the registered service flow information and the real-time network status information, establish a mapping relationship between the service flow and the multiple queues according to the service flow priority, and establish a forwarding rule according to the mapping relationship to achieve the bottom layer supporting multi-flow differentiated deterministic transmission; A service flow multi-dimensional resource adaptation module is used to establish the optimization target of single-multicast traffic co-network scheduling and resource constraints including network scheduling cycle constraints, transmission time slot constraints, forwarding path and multicast tree loop constraints, credit value constraints, service flow end-to-end delay, jitter and reliability constraints according to the mapping relationship and the forwarding rules, so as to realize the constraint optimization of multi-dimensional resource adaptation including single-multicast service flow and network routing, queue and time slot; A Markov decision module, used to design a Markov decision according to the optimization goal and the resource constraint, and provide theoretical support for the design of an intelligent scheduling algorithm; A time-sensitive service flow scheduling module is used to train a neural network model based on the Markov decision and dual-delay deep deterministic policy gradient, output a scheduling decision based on the trained neural network, and realize intelligent scheduling of single-multicast service flow fusion and shared network transmission; The solution module is used to obtain the multi-dimensional resource adaptation result according to the scheduling decision to achieve efficient adaptation between the business flow and the multi-dimensional resources.

9. A computer device, characterized in that: It includes a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the large-bandwidth multicast traffic-aware time-sensitive service flow resource adaptation method as described in any one of claims 1 to 7.

10. A storage medium storing computer-readable instructions, characterized in that: When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the large-bandwidth multicast traffic-aware time-sensitive service flow resource adaptation method as described in any one of claims 1 to 7.

Images