Deterministic network traffic scheduling method, apparatus and device, medium and program product

Abstract

The invention provides a deterministic network traffic scheduling method, apparatus and device, a medium and a program product. The method comprises the steps of obtaining a to-be-scheduled traffic and a network state; the network state is a current state of a deterministic network; inputting the network state into a strategy network to obtain a forwarding action and a forwarding period of the to-be-scheduled traffic; the policy network is constructed based on historical scheduling traffic and the deterministic network; determining a forwarding link of the traffic to be scheduled based on the network bandwidth state corresponding to the forwarding period; under the condition that the state of the to-be-scheduled traffic meets a plurality of constraint conditions, scheduling the to-be-scheduled traffic based on the forwarding link; a plurality of constraints are determined based on the policy network. According to the invention, by considering the bandwidth state of the traffic forwarding link and the period problem of traffic scheduling, an optimized traffic scheduling strategy is provided.

Description

Technical Field

[0001] The present invention relates to the field of network communication technologies, and in particular, to a method, device, equipment, medium, and program product for deterministic network traffic scheduling. Background Art

[0002] The power system needs to transmit its service information through a dedicated wide - area communication network to ensure the reliable transmission of various signals. Timely and accurate implementation of load control can improve the frequency stability level of the power system, enhance the fault protection ability of the power grid, and ensure the safe and stable operation of the power grid. To achieve the above goals, the power system must rely on an efficient and reliable data communication network to ensure the rapid and accurate transmission of instructions.

[0003] Existing deterministic bearer technologies are based on dedicated equipment and dedicated networks. When facing a large number of terminal devices, problems such as poor networking ability, high resource configuration cost, weak service - bearing ability, and insufficient security will occur. In addition, with the rapid increase of devices and traffic, the existing network architecture cannot effectively track and match changing traffic, and cannot provide the best path selection and allocate reasonable bandwidth resources for specific service requirements. Summary of the Invention

[0004] The present invention provides a method, device, equipment, medium, and program product for deterministic network traffic scheduling, aiming to solve the defects that the existing traffic scheduling scheme of the deterministic network cannot provide the best traffic scheduling path and cannot allocate reasonable bandwidth resources.

[0005] The present invention provides a method for deterministic network traffic scheduling, including the following steps: Obtain the traffic to be scheduled and the network status; the network status is the current status of the deterministic network; Input the network status into the policy network to obtain the forwarding action and forwarding period of the traffic to be scheduled; the policy network is constructed based on historical scheduled traffic and the deterministic network; Determine the forwarding link of the traffic to be scheduled based on the network bandwidth status corresponding to the forwarding period; Schedule the traffic to be scheduled based on the forwarding link when the status of the traffic to be scheduled meets multiple constraint conditions; the multiple constraint conditions are determined based on the policy network.

[0006] According to the method for deterministic network traffic scheduling provided by the present invention, the method for deterministic network traffic scheduling further includes: Determine the information tuple of historical scheduled traffic; the information tuple includes the sending period, deterministic bounded delay, and traffic bandwidth; Determine the network topology information of the deterministic network; the network topology information includes link propagation delay and link bandwidth capacity; Determine the period constraint condition among multiple constraint conditions based on the sending period, the deterministic bounded delay, and the link propagation delay; Determine the bandwidth constraint condition among multiple constraint conditions based on the traffic bandwidth and the link bandwidth capacity.

[0007] According to a deterministic network traffic scheduling method provided by the present invention, the scheduling of the traffic to be scheduled based on the forwarding link includes: Determine a traffic target based on the historical scheduled traffic, the network topology information, and the deterministic bounded delay; Determine a resource target based on the sending period and the bandwidth constraint condition; Schedule the traffic to be scheduled based on the forwarding link and the scheduling target; the scheduling target is determined based on the traffic target and the resource target.

[0008] According to a deterministic network traffic scheduling method provided by the present invention, the determining of the resource target based on the sending period and the bandwidth constraint condition includes: Determine the bandwidth occupancy ratio based on the traffic bandwidth and the link bandwidth capacity; Determine a super period based on the sending period; the super period is the least common multiple of multiple sending periods; Determine a resource target based on the inter-node link, the bandwidth occupancy ratio, and the super period; the inter-node link belongs to the network topology information.

[0009] According to a deterministic network traffic scheduling method provided by the present invention, the scheduling of the traffic to be scheduled based on the forwarding link further includes: Determine the state of the traffic to be scheduled based on the information tuple of the traffic to be scheduled; Determine a forwarding coefficient based on the forwarding information of the traffic to be scheduled; Schedule the traffic to be scheduled based on a traffic scheduling policy and the forwarding link; the traffic scheduling policy is determined based on the forwarding coefficient.

[0010] According to a deterministic network traffic scheduling method provided by the present invention, the deterministic network traffic scheduling method further includes: Determine the scheduling policy of the traffic to be scheduled based on the forwarding action and the forwarding period of the traffic to be scheduled; Determine a policy ratio and a policy advantage estimation value based on the scheduling policy; Optimize the parameters of the policy network based on the policy ratio and the policy advantage estimation value.

[0011] The present invention also provides a deterministic network traffic scheduling device, including the following modules: An acquisition module, configured to acquire the traffic to be scheduled and the network status; the network status is the current status of the deterministic network; A traffic forwarding information determination module, configured to input the network status into a policy network to obtain the forwarding action and forwarding period of the traffic to be scheduled; the policy network is constructed based on historical scheduled traffic and the deterministic network; A forwarding link determination module, configured to determine the forwarding link of the traffic to be scheduled based on the network bandwidth status corresponding to the forwarding period; A scheduled traffic scheduling module, configured to schedule the traffic to be scheduled based on the forwarding link when the status of the traffic to be scheduled meets multiple constraint conditions; the multiple constraint conditions are determined based on the policy network.

[0012] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and running on the processor. When the processor executes the computer program, the deterministic network traffic scheduling method described in any of the above is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the deterministic network traffic scheduling method described in any of the above is implemented.

[0014] The present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, the deterministic network traffic scheduling method described in any of the above is implemented.

[0015] The deterministic network traffic scheduling method, device, equipment, medium, and program product provided by the present invention analyze the network status of the deterministic network through a policy network constructed based on historical scheduled traffic and the deterministic network, and obtain the scheduling policy of the traffic to be scheduled, including the forwarding action and forwarding period. The forwarding link of the traffic to be scheduled is determined through the network bandwidth status corresponding to the forwarding period, and finally, when the status of the traffic to be scheduled meets multiple constraint conditions, the traffic to be scheduled is scheduled through the forwarding link. The present application provides an optimized traffic scheduling policy by considering the bandwidth status of the traffic forwarding link and the period problem of traffic scheduling. Description of the Drawings

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0017] Figure 1 is one of the flow schematic diagrams of the deterministic network traffic scheduling method provided by the present invention.

[0018] Figure 2 is the second of the flow schematic diagrams of the deterministic network traffic scheduling method provided by the present invention.

[0019] Figure 3 is the structural schematic diagram of the deterministic network traffic scheduling device provided by the present invention.

[0020] Figure 4 is the structural schematic diagram of the electronic device provided by the present invention. Detailed implementation manners

[0021] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Apparently, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without making creative efforts shall fall within the protection scope of the present invention.

[0022] The following combines Figures 1 - 4 to describe the deterministic network traffic scheduling method, device, equipment, medium and program product of the present invention.

[0023] Figure 1 is one of the flow schematic diagrams of the deterministic network traffic scheduling method provided by the present invention. As Figure 1 shown, the method includes the following: Step 100: Obtain the traffic to be scheduled and the network status; the network status is the current status of the deterministic network; Specifically, the present application provides a traffic scheduling method for a deterministic network, which can be implemented based on a policy network and is used for services such as precise load control in a power wide-area scenario. The present application realizes global traffic management through centralized control, and explicitly transmits data packets through the Segment Routing (SR) technology to ensure the stability and reliability of services. The present application decouples the cycle selection and path selection strategies, reduces the training complexity of the model (policy network), and optimizes the path selection through the weighted shortest path set strategy.

[0024] After the policy network is completed, solve the deterministic routing scheduling model in the precise load control service, and the training steps are as follows: Step 1: At the beginning of each round of training, first, according to the known network topology information and scheduling tasks, for the network information and the set of traffic to be scheduled Initialize, set the bandwidth matrix according to the information in the network, set the number of scheduled traffic to 0, and initialize the reward.

[0025] Step 2: Extract a piece of traffic from the set of traffic to be scheduled as the traffic to be scheduled. Before scheduling, also obtain the current network state, that is, the current state of the deterministic network.

[0026] Step 200: Input the network state into the policy network to obtain the forwarding action and forwarding period of the traffic to be scheduled; the policy network is constructed based on historical scheduled traffic and the deterministic network; Specifically, the above training steps further include: Step 3: By inputting the network state of the deterministic network into the policy network, obtain the next forwarding action of the traffic to be scheduled . The training process of the policy network is as shown below.

[0027] Step 300: Based on the network bandwidth state corresponding to the forwarding period, determine the forwarding link of the traffic to be scheduled; Specifically, the above training steps further include: Step 4: According to the cycle of the network bandwidth state, calculate the weighted shortest path set from the current node to the target node. The weight of the path is the sum of all link distances on the path multiplied by , so that both the physical distance of the path and the utilization rate of the bandwidth are considered, and the next forwarding link is selected according to the first path after sorting by weight , to obtain the action .

[0028] Step 400: When the state of the traffic to be scheduled satisfies multiple constraint conditions, schedule the traffic to be scheduled based on the forwarding link; the multiple constraint conditions are determined based on the policy network.

[0029] Specifically, the above training steps further include: Step 5: Adjust the above bandwidth matrix according to the action , and at the same time update the state of the currently scheduled traffic. After this step of forwarding is completed, judge the current state to determine whether it satisfies the relevant constraints (that is, the multiple constraint conditions in this embodiment). If not, all network resources occupied by this traffic need to be returned, and then obtain the reward for this step of forwarding and accumulate it.

[0030] Step 6: If a certain deterministic traffic scheduling fails or reaches the target node, switch to the next traffic to be scheduled and re-obtain the current network state , and enter the scheduling of the next traffic.

[0031] Step 7: After all deterministic traffic scheduling in a single training is completed, optimize the parameters of the policy network and the value network according to the results of this scheduling until the target training round is completed.

[0032] In this embodiment, through a policy network constructed based on historical scheduling traffic and a deterministic network, analyze the network state of the deterministic network to obtain a scheduling policy for the traffic to be scheduled, including a forwarding action and a forwarding period. Determine the forwarding link of the traffic to be scheduled through the network bandwidth status corresponding to the forwarding period. Finally, when the status of the traffic to be scheduled meets multiple constraint conditions, schedule the traffic to be scheduled through the forwarding link. This application provides an optimized traffic scheduling policy by considering the bandwidth status of the traffic forwarding link and the period problem of traffic scheduling.

[0033] Figure 2 is the second flowchart of the deterministic network traffic scheduling method provided by the present invention. As Figure 2 shown, the method may further include: Step 10: Determine the information tuple of the historical scheduling traffic; the information tuple includes a sending period, a deterministic bounded delay, and a traffic bandwidth; Step 20: Determine the network topology information of the deterministic network; the network topology information includes a link propagation delay and a link bandwidth capacity; Step 30: Based on the sending period, the deterministic bounded delay, and the link propagation delay, determine the period constraint condition among multiple constraint conditions; Step 40: Based on the traffic bandwidth and the link bandwidth capacity, determine the bandwidth constraint condition among multiple constraint conditions.

[0034] Specifically, for a deterministic network, its network topology diagram can be expressed as: , where represents the set of transmission nodes of the deterministic network; represents the set of links between nodes. The definition of deterministic network traffic is: periodic unicast traffic from a source node to a destination node. Name the set of DetNet (deterministic network) traffic that needs to be scheduled in the service as , DetNet traffic , can be defined by a tuple { , , , }. Among them, represents 's source node; represents The target node; Represents The period, that is, how often it will be re - issued; Is The bandwidth to be occupied; Is The deterministic bounded delay of It needs to reach the target node within this bounded delay.

[0035] For each traffic flow to be scheduled It has a forwarding path sequence And a forwarding period sequence , where Represents which step of the scheduling, Represents which traffic flow, Represents the selected edge, Represents the selected period.

[0036] Based on the above, it can be seen that for each traffic flow to be scheduled , there are period constraints and bandwidth constraints. Among them, the period constraint is shown in Formula 1.

[0037] ; (1) Is the propagation delay of link , which is determined by the distance between nodes in the network topology graph. That is, the first forwarding needs to be carried out within the generation period of , and at the same time, the result of each traffic flow scheduling needs to meet the Deterministic bounded delay of

[0038] Represents the current period; Is the total number of forwarding queues of the specified cyclic sequential queue forwarding mechanism (CSQF). Because for each port of the router node, one of the queues is the forwarding queue in the current period, and the rest are receiving queues. When the traffic determines which port to forward according to the weighted shortest path set, it needs to select one from the receiving queues of this port for forwarding transmission. Due to the characteristics of CSQF, it also determines the period of the forwarding traffic.

[0039] ; (2) The bandwidth constraint is shown in Formula 2, where Represents the bandwidth capacity of link At the same moment; Represents within The period occupies the link bandwidth, that is, within any period, the sum of the bandwidths of the deterministic traffic occupying the link is less than the bandwidth capacity of the link.

[0040] In this embodiment, by using the information tuple of the traffic and the network topology information of the deterministic network, the period constraint condition and the bandwidth constraint condition in multiple constraint conditions are determined, considering the bandwidth problem of the traffic forwarding link and the period problem of the traffic scheduling, and an optimization method for the traffic scheduling strategy is provided.

[0041] In one embodiment, the deterministic network traffic scheduling method provided by the embodiments of the present application may further include: Step 410: Determine a traffic target based on the historical scheduled traffic, the network topology information, and the deterministic bounded delay; Step 420: Determine a resource target based on the sending period and the bandwidth constraint condition; Step 430: Schedule the traffic to be scheduled based on the forwarding link and the scheduling target; the scheduling target is determined based on the traffic target and the resource target.

[0042] Specifically, the traffic scheduling problem can also be divided into multiple implementation goals, such as the quantity target and the resource target of traffic scheduling. Taking the quantity target of traffic scheduling as an example, the deterministic traffic scheduling problem can be described as: Given a network topology graph and a batch of scheduling tasks for deterministic network traffic, find an effective scheduling scheme (routing assignment and period assignment) for the deterministic traffic, and schedule as much traffic as possible without exceeding the deterministic bounded delay of these traffic to be scheduled.

[0043] ; (3) Its traffic target is achieved as shown in Formula 3, where represents whether it is successfully scheduled, 1 for success and 0 for failure.

[0044] This embodiment realizes the flexibility of traffic scheduling by setting some traffic scheduling goals.

[0045] In one embodiment, the deterministic network traffic scheduling method provided by the embodiments of the present application may further include: Step 421: Determine a bandwidth occupancy ratio based on the traffic bandwidth and the link bandwidth capacity; Step 422: Determine a super period based on the sending period; the super period is the least common multiple of multiple sending periods; Step 423: Determine the resource target based on the inter-node link, the bandwidth ratio, and the hypercycle; the inter-node link belongs to the network topology information.

[0046] Since deterministic network traffic is periodic and has different generation periods , therefore, a hypercycle is defined . Within each hypercycle, all deterministic network traffic has the same scheduling characteristics, which is convenient for scheduling calculation. The hypercycle of all traffic can be calculated using the least common multiple (LCM) of all traffic periods, that is . Therefore, discussing the traffic scheduling problem in a hypercycle loop can handle the overall situation of this batch of traffic. For simplicity and ease of calculation, the clock deviation existing between nodes can be ignored so that the hypercycle can be synchronously applied among all nodes.

[0047] Specifically, taking the realization of the resource target as an example, it is necessary to analyze the overall utilization of the deterministic network to more effectively utilize network resources and avoid waste of some resources. The realization of its resource target is shown in Formulas 4 and 5. Among them, represents the bandwidth ratio of link e in the t period; represents the number of links; represents the size of the hypercycle.

[0048] ; (4) ; (5) ; (6) The above two objective functions can also be combined into a weighted objective function for easy optimization and solution. As shown in Formula 6, where is a weight coefficient that can be adjusted according to needs. For example, , .

[0049] This embodiment improves the flexibility of traffic scheduling by formulating a multi-objective traffic scheduling problem.

[0050] In one embodiment, the deterministic network traffic scheduling method provided by the embodiments of the present application may further include: Step 440: Determine the status of the traffic to be scheduled based on the information tuple of the traffic to be scheduled; Step 450: Determine the forwarding coefficient based on the forwarding information of the traffic to be scheduled; Step 460: Schedule the traffic to be scheduled based on the traffic scheduling policy and the forwarding link; the traffic scheduling policy is determined based on the forwarding coefficient.

[0051] Specifically, a problem with a specific network scenario and corresponding deterministic traffic scheduling can be converted into a Markov decision process to facilitate processing using deep reinforcement learning. In this application, the state, action, and reward can be set as follows. The state represents the overall situation of the scheduled traffic and the network determinism. According to the scheduling situation of the traffic each time, the state will also change. The environment includes the generation period of the traffic to be scheduled itself, the required bandwidth size, the deterministic bounded delay, the current period, the current location, and the bandwidth situation of each link in the entire supercycle, etc.

[0052] For the state of a certain scheduling, it is defined as follows: , where represents which step of the scheduling; represents which traffic; represents the available bandwidth situation of each period of the link in the current scheduling state; indicates the node where the current traffic is located.

[0053] The action is that for each scheduling of the traffic, according to the current traffic state select the next behavior. The scheduling of each traffic is divided into various actions, which are manifested as the specific period selection and node selection for a certain step of the currently scheduled traffic. For each action, it is defined as follows: .

[0054] After the forwarding node and period selection are completed, it is necessary to update the period and node of the current flow as the period and node of the next state, which are defined as follows: , and the node is replaced by the other end of the selected link .

[0055] ; (7) The definition of the reward is shown in formula 7, where is the minimum value of the available bandwidth ratio in the path link under the period and path selected by the action, representing the bandwidth situation of the link under this path; is the penalty coefficient for the successful forward forwarding of this scheduling, because it is encouraged to schedule the flow with fewer forwarding times; represents the reward coefficient for this scheduling to reach the destination; is the penalty coefficient when this scheduling fails; ={1,0} indicates whether this scheduling is successful or not; represents whether this scheduling reaches the end state.

[0056] In this embodiment, the traffic to be scheduled is scheduled through a traffic scheduling policy and a forwarding link, providing an optimized traffic scheduling policy.

[0057] In one embodiment, the deterministic network traffic scheduling method provided by the embodiments of the present application may further include: Step 500: Determine the scheduling policy of the traffic to be scheduled based on the forwarding action and forwarding period of the traffic to be scheduled; Step 600: Determine a policy ratio and a policy advantage estimate value based on the scheduling policy; Step 700: Optimize the parameters of the policy network based on the policy ratio and the policy advantage estimate value.

[0058] Specifically, the purpose of the present application is to obtain a better traffic scheduling policy. The policy is a parameterized policy, representing the probability of performing an action in state , with parameters , which can be continuously iterated during the training of the neural network. In the present application, the optimized policy will give the best period for traffic scheduling according to the current network state.

[0059] ; (8) ; (9) ; (10) ; (11) In the improved Proximal Policy Optimization (PPO) algorithm, the function of formula 8 is used to optimize the policy parameter . Among them, is the policy ratio; is the advantage estimate under policy , representing how good or bad the action is relative to the baseline policy in state ; is a small integer used to limit the amplitude of policy update; clips to be within the range of , and the clip function can be used to control the parameter gap between the old and new policies within a certain range.

[0060] The Generalized Advantage Estimation (GAE) method can be used for estimation. For the The GAE definition during a step is shown in Equation 9. Among them, represents the discount factor; , is a hyperparameter introduced in GAE; , represents the temporal difference error; represents the state value function.

[0061] To optimize the policy, a loss function for the value function is proposed, defined as in Equation 10, and its gradient is as shown in Equation 11. Then, Adam with Weight Decay (AdamW) is used to update the network parameters; AdamW adds a weight decay mechanism to Adaptive Moment Estimation (Adam), can dynamically adjust the learning rate according to the gradient history of the parameters, can prevent overfitting, and can accelerate the convergence speed and improve performance.

[0062] In this embodiment, the parameters of the traffic scheduling policy are optimized by an improved PPO algorithm.

[0063] Next, the deterministic network traffic scheduling device provided by the present invention will be described. The deterministic network traffic scheduling device described below can be mutually corresponding and referred to the deterministic network traffic scheduling method described above.

[0064] Please refer to Figure 3 , the present invention also provides a deterministic network traffic scheduling device, including: An acquisition module 301, configured to acquire the traffic to be scheduled and the network state; the network state is the current state of the deterministic network; A traffic forwarding information determination module 302, configured to input the network state into the policy network to obtain the forwarding action and forwarding period of the traffic to be scheduled; the policy network is constructed based on historical scheduled traffic and the deterministic network; A forwarding link determination module 303, configured to determine the forwarding link of the traffic to be scheduled based on the network bandwidth state corresponding to the forwarding period; A scheduled traffic scheduling module 304, configured to schedule the traffic to be scheduled based on the forwarding link when the state of the traffic to be scheduled meets multiple constraint conditions; the multiple constraint conditions are determined based on the policy network.

[0065] Optionally, the deterministic network traffic scheduling device further includes: An information tuple determination module, configured to determine the information tuple of historical scheduled traffic; the information tuple includes the sending period, deterministic bounded delay, and traffic bandwidth; A network topology information determination module, configured to determine the network topology information of the deterministic network; the network topology information includes link propagation delay and link bandwidth capacity; A period constraint condition determination module, configured to determine a period constraint condition among multiple constraint conditions based on the sending period, the deterministic bounded delay, and the link propagation delay; A bandwidth constraint condition determination module, configured to determine a bandwidth constraint condition among multiple constraint conditions based on the traffic bandwidth and the link bandwidth capacity.

[0066] Optionally, the scheduled traffic scheduling module includes: A traffic target determination unit, configured to determine a traffic target based on the historical scheduled traffic, the network topology information, and the deterministic bounded delay; A resource target determination unit, configured to determine a resource target based on the sending period and the bandwidth constraint condition; A to-be-scheduled traffic scheduling unit, configured to schedule the to-be-scheduled traffic based on the forwarding link and the scheduling target; the scheduling target is determined based on the traffic target and the resource target.

[0067] Optionally, the resource target determination unit includes: A bandwidth ratio determination unit, configured to determine a bandwidth ratio based on the traffic bandwidth and the link bandwidth capacity; A super-period determination unit, configured to determine a super-period based on the sending period; the super-period is the least common multiple of multiple sending periods; A resource target determination unit, configured to determine a resource target based on the inter-node link, the bandwidth ratio, and the super-period; the inter-node link belongs to the network topology information.

[0068] Optionally, the scheduled traffic scheduling module further includes: A traffic state determination unit, configured to determine the state of the to-be-scheduled traffic based on the information tuple of the to-be-scheduled traffic; A forwarding coefficient determination unit, configured to determine a forwarding coefficient based on the forwarding information of the to-be-scheduled traffic; A traffic scheduling unit, configured to schedule the to-be-scheduled traffic based on a traffic scheduling policy and the forwarding link; the traffic scheduling policy is determined based on the forwarding coefficient.

[0069] Optionally, the deterministic network traffic scheduling device further includes: A scheduling policy determination module, configured to determine the scheduling policy of the to-be-scheduled traffic based on the forwarding action and the forwarding period of the to-be-scheduled traffic; A policy parameter determination module, configured to determine a policy ratio and a policy advantage estimation value based on the scheduling policy; A policy network parameter determination module, configured to optimize the parameters of the policy network based on the policy ratio and the policy advantage estimation value.

[0070] Figure 4 An example of a schematic physical structure diagram of an electronic device is shown as Figure 4 shown. The electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440. Among them, the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call the logical instructions in the memory 430 to execute a deterministic network traffic scheduling method, which includes: obtaining traffic to be scheduled and a network state; the network state is the current state of a deterministic network; inputting the network state into a policy network to obtain a forwarding action and a forwarding period of the traffic to be scheduled; the policy network is constructed based on historical scheduled traffic and the deterministic network; determining a forwarding link of the traffic to be scheduled based on the network bandwidth state corresponding to the forwarding period; scheduling the traffic to be scheduled based on the forwarding link when the state of the traffic to be scheduled satisfies multiple constraint conditions; the multiple constraint conditions are determined based on the policy network.

[0071] In addition, when the logical instructions in the above-mentioned memory 430 are implemented in the form of software functional units and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk, or an optical disc that can store program codes.

[0072] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the deterministic network traffic scheduling method provided by each of the above methods. The method includes: obtaining the traffic to be scheduled and the network state; the network state is the current state of the deterministic network; inputting the network state into a policy network to obtain the forwarding action and forwarding period of the traffic to be scheduled; the policy network is constructed based on historical scheduled traffic and the deterministic network; determining the forwarding link of the traffic to be scheduled based on the network bandwidth state corresponding to the forwarding period; scheduling the traffic to be scheduled based on the forwarding link when the state of the traffic to be scheduled meets multiple constraint conditions; the multiple constraint conditions are determined based on the policy network.

[0073] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it realizes the deterministic network traffic scheduling method provided by each of the above methods. The method includes: obtaining the traffic to be scheduled and the network state; the network state is the current state of the deterministic network; inputting the network state into a policy network to obtain the forwarding action and forwarding period of the traffic to be scheduled; the policy network is constructed based on historical scheduled traffic and the deterministic network; determining the forwarding link of the traffic to be scheduled based on the network bandwidth state corresponding to the forwarding period; scheduling the traffic to be scheduled based on the forwarding link when the state of the traffic to be scheduled meets multiple constraint conditions; the multiple constraint conditions are determined based on the policy network.

[0074] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0075] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. And these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A deterministic network traffic scheduling method, characterized in that: include: Obtain the traffic to be scheduled and the network status; The network state is the current state of the deterministic network; Inputting the network state into a policy network to obtain a forwarding action and a forwarding period of the traffic to be scheduled; the policy network is constructed based on historical scheduling traffic and the deterministic network; Determining a forwarding link for the traffic to be scheduled based on a network bandwidth state corresponding to the forwarding period; When the state of the traffic to be scheduled satisfies a plurality of constraint conditions, scheduling the traffic to be scheduled based on the forwarding link; A plurality of constraints are determined based on the policy network.

2. The deterministic network traffic scheduling method according to claim 1, characterized in that: The network traffic scheduling method further includes: Determine an information tuple of historical scheduling traffic; the information tuple includes a sending period, a deterministic bounded delay, and a traffic bandwidth; Determining network topology information of the deterministic network; the network topology information includes link propagation delay and link bandwidth capacity; Determine a period constraint condition among a plurality of constraint conditions based on the sending period, the deterministic bounded delay, and the link propagation delay; Based on the traffic bandwidth and the link bandwidth capacity, a bandwidth constraint condition among a plurality of constraint conditions is determined.

3. The deterministic network traffic scheduling method according to claim 2, characterized in that: The scheduling of the to-be-scheduled traffic based on the forwarding link includes: Determining a traffic target based on the historical scheduling traffic, the network topology information, and the deterministic bounded delay; Determining a resource target based on the sending period and the bandwidth constraint; The to-be-scheduled traffic is scheduled based on the forwarding link and the scheduling target; the scheduling target is determined based on the traffic target and the resource target.

4. The deterministic network traffic scheduling method according to claim 3, characterized in that: The determining of the resource target based on the sending period and the bandwidth constraint condition comprises: Determining a bandwidth share based on the traffic bandwidth and the link bandwidth capacity; Determining a super period based on the sending period; the super period is the least common multiple of multiple sending periods; A resource target is determined based on inter-node links, the bandwidth proportion and the super period; the inter-node links belong to the network topology information.

5. The deterministic network traffic scheduling method according to claim 1, characterized in that: The scheduling of the to-be-scheduled traffic based on the forwarding link further includes: Determining the state of the traffic to be scheduled based on the information tuple of the traffic to be scheduled; Determining a forwarding coefficient based on the forwarding information of the traffic to be scheduled; The to-be-scheduled traffic is scheduled based on a traffic scheduling strategy and the forwarding link; the traffic scheduling strategy is determined based on the forwarding coefficient.

6. The deterministic network traffic scheduling method according to claim 1, characterized in that: The deterministic network traffic scheduling method further includes: Determine a scheduling strategy for the traffic to be scheduled based on the forwarding action and forwarding period of the traffic to be scheduled; Determining a strategy ratio and a strategy advantage estimate based on the scheduling strategy; Parameters of the policy network are optimized based on the policy proportions and the policy advantage estimates.

7. A deterministic network traffic scheduling device, characterized in that: include: The acquisition module is used to obtain the traffic to be scheduled and the network status; The network state is the current state of the deterministic network; A traffic forwarding information determination module, used to input the network state into a policy network to obtain a forwarding action and a forwarding period of the traffic to be scheduled; the policy network is constructed based on historical scheduled traffic and the deterministic network; A forwarding link determination module, used to determine the forwarding link of the traffic to be scheduled based on the network bandwidth status corresponding to the forwarding period; A scheduling traffic scheduling module, used for scheduling the traffic to be scheduled based on the forwarding link when the state of the traffic to be scheduled meets multiple constraints; A plurality of constraints are determined based on the policy network.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the deterministic network traffic scheduling method as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the deterministic network traffic scheduling method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the deterministic network traffic scheduling method according to any one of claims 1 to 6 is implemented.

Images