5G Scheduling with Time-Dependent Network Information

Abstract

The techniques disclosed in this disclosure relate to a communication network and method for supporting time-dependent deterministic communication based on a time-dependent network (TSN) mechanism. The communication network may include a plurality of network domains in the communication network connected to each other, and a plurality of schedulers in the plurality of network domains. Each of the plurality of schedulers may calculate a local schedule for each of the plurality of network domains, and a first scheduler and a second scheduler of the plurality of schedulers may repeatedly exchange a first local schedule and a second local schedule. The first local schedule and the second local schedule may include different cycle times. The first network domain may transmit inter-domain Ethernet link information related to a link connecting the first network domain and the second network domain, and intra-domain schedule information related to the first local schedule.

Description

Priority

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 318,431, filed March 10, 2022, and U.S. Provisional Patent Application No. 63 / 318,896, filed March 11, 2022, both of which are incorporated by reference in their entireties herein. [Technical field]

[0002] This specification relates generally to the integration of wireless communication systems and time-sensitive networks (TSN), and more specifically, for example, to integrating scheduling of TSN with scheduling of 5G systems to improve scheduling of both systems via distributed scheduling. [Background technology]

[0003] Media Access Control (MAC) is a sublayer of the Data Link Layer of telecommunications networks following the Open Systems Interconnection (OSI) model. The Data Link Layer is Layer 2 of the 7-layer framework. Together with the Local Link Control (LLC), the MAC provides flow control and multiplexing of the transmission medium. In cellular networks such as 5G systems (5GS) and their predecessors, the MAC is designed to maximize the utilization of the over-the-air spectrum. 5GS seeks to provide media access control using a combination of time and frequency isolation.

[0004] Some applications, such as industrial automation and manufacturing, require ubiquitous and seamless connectivity with strict, deterministic timing requirements for communication between the various devices or components of the application (e.g., industrial controllers, sensors, actuators, etc.). To meet such requirements, TSN systems provide deterministic communication with relatively strict Quality of Service (QoS) parameters such as latency, jitter, and reliability requirements for data traffic. Time-Sensitive Networking (TSN) was originally created because traditional Ethernet networks did not deliver packets with any notion of time. Traditional Ethernet networks delivered packets in a "best effort" framework, with no absolute deadlines for when, for example, a file needed to be transferred from one place to another. The file needed to arrive at its destination as soon as possible, in a contending manner, but not at any specific time either. Certain use cases, such as voice, video, and diagnostic communications, require deadlines, which resulted in the concept of TSN. TSN scheduling uses only temporal separation. Time is synchronized between nodes in a TSN network, which schedules frames to be sent to their destinations through the node's path, with frames being selected based on knowledge of whether the node is busy at that particular time.

[0005] Considering the growing popularity of 5G wireless communication systems, TSN systems may be integrated with 5G systems that provide high reliability services such as Ultra-Reliable and Low Latency Communication (URLLC) services. Since both TSN and 5G systems have schedulers, information from the TSN scheduler can be incorporated into the 5G scheduler to improve the scheduling it performs.

[0006] Particular features of the subject technology are set forth in the appended claims, but for purposes of illustration, certain aspects of the subject technology are illustrated in the following figures. [Brief description of the drawings]

[0007] [Figure 1] 1 illustrates an example of a conventional integrated TSN-5G system 100 according to one or more implementations. [Diagram 2] 1 illustrates a block diagram of an example integrated TSN-5G system 200 architecture in accordance with one or more implementations. [Diagram 3] Illustrates an electronic system in which one or more implementations of the subject technology may be implemented. [Figure 4] 8 illustrates a block diagram of an example integrated TSN-5G system 800 architecture, according to one or more implementations. [Figure 5A] 1 illustrates a block diagram of inputs and outputs of a TSN and 5G scheduler, according to one or more implementations. [Figure 5B] 1 illustrates a block diagram of inputs and outputs of a TSN and 5G scheduler, according to one or more implementations. [Figure 6] 1 illustrates a block diagram of a TSN network including a scheduler, according to one or more implementations. [Figure 7] 1 is a flowchart of a distributed scheduling algorithm according to one aspect of the present disclosure. [Figure 8] FIG. 1 is a block diagram of distributed scheduling in multiple domains according to an aspect of the present disclosure. [Figure 9] FIG. 2 is a block diagram of a distributed scheduler message according to one aspect of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] The detailed description set forth below is intended as a description of various configurations of the subject technology, and is not intended to represent the only configurations in which the subject technology can be implemented. The accompanying drawings are incorporated in this specification and constitute a part of the detailed description. The detailed description includes specific details for the purpose of providing a thorough understanding of the subject technology. However, the subject technology is not limited to the specific details described herein and can be implemented using one or more other implementation forms. In one or more implementation forms, structures and components are shown in block diagrams to avoid obscuring the concepts of the subject technology.

[0009] As mentioned above, in some applications, such as, but not limited to, industrial automation, manufacturing, and aerospace and automotive in-vehicle communications, TSN systems providing deterministic communications may be integrated with fifth-generation (5G) wireless communication systems providing flexible and ultra-reliable, low-latency communications (URLLC) services.

[0010] As mentioned above, TSN was created to satisfy the need in traditional Ethernet networks for frames to be delivered with a maximum tolerable latency from when they were sent. In other words, to be delivered by a specific time. Traditional Ethernet networks did not have a time domain, but 5GS does and also creates delivery schedules for frames that have a timed element. Thus, the TSN scheduler can generate a better schedule if it can have knowledge of the state of the 5G scheduler. If the TSN application scheduler can be informed of the state of the 5G scheduler resource blocks, it can add appropriate constraints to guide its solution. On the other hand, if the 5G scheduler knows that the traffic is coming from a TSN system and is therefore periodic, it can make more informed scheduling decisions.

[0011] TSN leverages the IEEE 802.1Qbv standard, which requires a scheduler to calculate when Ethernet frames of a particular flow are transmitted at each hop along the path from source to destination without colliding with other TSN scheduled Ethernet frames. TSN frames may flow through a 5G system (5GS), which may include 5G base stations (gNBs, also known as gNodeBs) that use 5g New Radio (NR) technology. NRs and gNBs also require schedulers to determine when and on what resources to transmit data across the radio. Thus, it would be advantageous to enable joint TSN / 5G scheduling operations. However, typically in such an integrated TSN-5G system, the entire 5G system is configured to operate as one single TSN component (e.g., TSN bridge), and the integrated system is configured as a fully centralized configuration model, e.g., using one centralized TSN configuration controller. Thus, such an integrated system may not support the flexibility of deploying a 5G system that includes various components provided by different vendors or for a decentralized TSN configuration of the integrated system. Furthermore, the 5G system may support reliable communication by using redundant paths for data transmission. However, when the 5G system is configured as one TSN component in a typical TSN-5G integrated system, redundant paths are configured throughout the entire 5G system through all its components (e.g., from the user equipment (UE) to the user plane function (UPF)). This does not allow the flexibility to configure redundant data transmission paths only for parts of the 5G system, such as the air interface between the UE and the radio access network (RAN) / gNodeB, which are more susceptible to data delays and / or errors.

[0012] To address the issues discussed above in integrated TSN-5G systems, the subject technology provides a novel method of joint scheduling that leverages the predictability of the TSN Qbv schedule, which includes precise, non-overlapping transmission times that allow the 5GS to predict when TSN data from a particular flow will reach maximum radio and message size. Knowledge of information known by the TSN scheduler, including arrival times and message sizes, can enable the gNB scheduler to better allocate resource elements (REs) and resource blocks (RBs). Knowledge of TSN scheduling information can also play a role in optimizing multiple-input / multiple-output (MIMO) antenna operation, which is at a lower physical level than the gNB scheduler.

[0013] Joint TSN-5G scheduling may be achieved, in one aspect of the present disclosure, via distributed scheduling. Accordingly, the present disclosure relates to a 5G bidirectional scheduling status information message. The message may include expected TSN user equipment (UE) configuration from the TSN application scheduler and may return 5G scheduler information such as that found in system information block (SIB) scheduling information. Further information regarding SIB scheduling information may be found in 3GPP TS38.331 Version 15.5.1 Release 15 ("5G, NR, Radio Resource Control (RRC), Protocol Specification"), which is incorporated herein by reference in its entirety.

[0014] FIG. 1 illustrates a non-limiting example of a conventional integrated TSN-5G system 100 in which a 5G system 106 is configured to be emulated as a single TSN component (e.g., a TSN bridge). Overall, the system 100 is configured as a deterministic TSN system that communicates data between end devices, e.g., input / output (I / O) devices 102 and controllers 104, via the 5G system 106 (emulating as a TSN bridge) and one or more (conventional) TSN bridges 108, and using a TSN controller 110. The system 100 is configured based on standard methods for time synchronization and traffic management to enable deterministic communication over a standard Ethernet network between the end devices, e.g., the I / O devices 102 and controllers 104. For example, the system 100 may operate in accordance with the IEEE 802.1Q TSN specification suite, which standardizes layer 2 communication for networking protocols that provide deterministic communication while sharing the same infrastructure. For example, several standards establish various technical paradigms for TSN systems, namely clock synchronization (802.1AS, generalized Precision Time Protocol (gPTP)), frame preemption (802.3br and 802.1Qbu), scheduled traffic (802.1Qbv), and redundancy management (Frame Replication and Elimination for Reliability (FRER) IEEE 802.1CB). These standards need to work together at Ethernet Layer 2 to ensure that critical control and safety functions are performed while meeting their respective deadlines and constraints. As another implementation, a similar integrated system can be configured to implement TSN techniques over a wireless local area network (wireless LAN), such as a Wi-Fi network, for example based on Wi-Fi 6 and other common wireless LANs.

[0015] As used herein, a "TSN schema" may refer to, but is not limited to, a network, component, element, unit, node, hub, switch, control, module, path, data, data frame, traffic, protocol, operation, transmission, and combinations thereof that adhere to, are configured for, or are compliant with one or more of the IEEE 802.1 TSN standards. The 802.1Qbv TSN standard addresses the transmission of critical and non-critical data traffic within a TSN. Critical data traffic is guaranteed to be delivered at a scheduled time, while non-critical data traffic is typically given a lower priority. Various traffic classes have been established according to IEEE 802.1Q that are used to prioritize different types of data traffic.

[0016] Ethernet frame preemption is defined by the IEEE 802.3br and IEEE 802.1Qbu standards, which can interrupt the transmission of non-critical Ethernet frames, and is also useful for reducing the latency and latency variation of critical traffic. The basis of resource management is defined by the TSN configuration model (IEEE 802.1Qcc). A centralized network configuration (CNC) 112 can be applied to network devices (bridges, e.g., 5G system bridges 106, bridges 108), while a centralized user configuration (CUC) 114 can be applied to user devices (end stations, e.g., I / O devices 102). The fully centralized configuration model follows a software-defined networking (SDN) approach. In other words, the CNC 112 and CUC 114 in the controller 110 provide a control plane instead of a distributed protocol. In contrast, a distributed control protocol is applied in a fully distributed model where there is no CNC or CUC.

[0017] As a consequence of ultra-reliability, high availability can be provided for data flows via per-packet level reliability mechanisms by Frame Replication and Elimination (FRER) for Reliability (IEEE 802.1CB), which provides reliability by sending multiple copies of the same data packets over disjoint paths in the network. Per-stream filtering and policing (802.1Qci) improves reliability by protecting against bandwidth violations, malfunctions, and malicious behavior. Furthermore, time synchronization in TSN systems can be defined by the Generic Precision Time Protocol (gPTP) (802.1AS), a profile of the Precision Time Protocol standard (IEEE 1588). gPTP provides reliable time synchronization that can be used with other TSN tools such as scheduled traffic (802.1Qbv).

[0018] To achieve a desired level of reliability, TSN employs time synchronization and time-aware data traffic shaping. Data traffic shaping uses schedules to control the gating of transmissions on network switches and bridges (e.g., nodes). In some aspects, such data traffic schedules in TSN may be determined in advance of the operation of the network. In other aspects, data traffic schedules may be determined during an initial design stage based on system requirements and updated as desired. For example, in addition to defining the TSN topology (including communication paths, bandwidth reservations, and various other parameters), a network-wide synchronization time for data transmission may be defined in advance. Such a plan for data transmission on the communication paths of a network is typically referred to as a "communication schedule" or simply a "schedule." A data traffic schedule on a TSN may be determined for a particular data packet over a particular path, at a particular time, and for a particular duration. Non-limiting examples of techniques for generating TSN data traffic schedules are discussed in U.S. Application No. 17 / 100,356, which is incorporated herein by reference in its entirety.

[0019] Time-critical communications between end devices or nodes (e.g., I / O device 102 and controller 104) in a TSN include "TSN flows," also known as "data flows" or simply "flows." For example, a data flow may include datagrams, such as data packets or data frames. Each data flow is unidirectional, travels from a first source or originating end device (e.g., I / O device 102) to a second destination end device (e.g., controller 104) in the system, and has a unique identity and time requirement. These source and destination devices are generally referred to as "talkers" and "listeners." Specifically, "talkers" and "listeners" are the source and destination of a data flow, respectively, and each data flow is uniquely identified by an end device operating in the system. It will be understood that for a given network topology including multiple interconnected devices, a set of data flows between the interconnected devices or nodes can be defined. For example, the set of data flows can be between the interconnected devices. For a set of data flows, various subsets or permutations of data flows can be additionally defined. Furthermore, time-critical communications between end devices or nodes in a TSN network include "TSN streams" or "streams," and each TSN stream can originate at a particular talker node that is intended to be communicated to one or more listener nodes. Thus, each TSN stream can include one or more data flows, and each data flow is between a talker node (where the TSN stream originates) and a listener node.

[0020] Both end devices (e.g., 102, 104) and switches (commonly referred to as "bridges" or "switching nodes") (e.g., 106, 108) transmit and receive data (in one non-limiting example, Ethernet frames) in a data flow based on a predefined time schedule. The switching nodes and end devices must be time synchronized to ensure that the predefined time schedule of the data flow is properly adhered to throughout the network. For example, in FIG. 1, clock 116 represents that the various switching nodes and end devices in the TSN system 100 (including in the 5G system 106) are time synchronized with reference to a global clock (grand master clock timing). In some other aspects, only the switches may transmit data based on a predefined schedule, while the end devices, e.g., legacy devices, may transmit data in an unscheduled manner.

[0021] Data flows in TSN can be scheduled using a single device (e.g., controller 110) that assumes a fixed, unchanging path through the network between talker / listener devices and switching nodes in the network. Alternatively, data flows can be scheduled using a set of devices or modules. The scheduling device, whether a single device or a set of devices, can be arranged to define a centralized scheduler. In yet other aspects, the scheduler device can constitute a distributed arrangement. TSN can also receive non-time sensitive communications, such as rate-constrained communications. In one non-limiting example, the scheduling device can include an offline scheduling system or module.

[0022] TSN traffic may be tagged using a variety of mechanisms, including VLAN tag Ethernet address IP header information, and a combination of VLAN tag Ethernet address and IP header information. Traffic may be identified and tagged anywhere in the system before protocol data unit (PDU) identification is required. TSN talkers may create multiple TSN flows (streams) with different TSN latency and determinism requirements and may be assigned different paths that meet the requirements. In some implementations of the subject invention, latency and determinism values ​​may be defined and provided to TSN applications as a limited set of static discrete values, rather than providing for an unlimited set of continuous values ​​to be accepted.

[0023] In some implementations, the I / O end device 102 may be, in various aspects, a complex mechanical entity, such as a production line in a factory, a gas-fired power plant, an avionics data bus on an aircraft, a jet engine on an aircraft in an aircraft (e.g., two or more aircraft), a digital backbone in an aircraft, an avionics system, a mission or flight network, a wind farm, a locomotive, etc. In various implementations, the I / O end device 102 may include any number of end devices, such as sensors, actuators, motors, and software applications. The sensors may include any conventional sensor or transducer, such as a camera that generates video or image data, an x-ray detector, an acoustic pickup device, a tachometer, a global positioning system receiver, a wireless device that transmits a wireless signal and detects the reflection of the wireless signal to generate image data, or another device.

[0024] Additionally, actuators (e.g., devices, equipment, or machines that move to perform one or more operations of an I / O device 102) can communicate using the TSN system 100. Non-limiting examples of actuators may include brakes, throttles, robotic devices, medical imaging devices, lights, turbines, etc. An actuator can communicate its status data to one or more other devices (e.g., other I / O devices 102, controller 104 via the TSN system 100). The status data may represent the position, state, or health, etc., of the actuator sending the status data. An actuator can receive command data from one or more other devices (e.g., other I / O devices 102, controller 104) of the TSN system 100. The command data may represent instructions that instruct the actuator how or when to move, act, etc.

[0025] In some implementations, the controller 104 can communicate various data between or among the I / O end devices 102 via the TSN 100. For example, the control system 104 can communicate command data to one or more of the devices 102 or receive data, such as status data or sensor data, from one or more of the devices 102. Thus, the controller 104 can be configured to control the operation of the I / O devices 102 based on data acquired or generated by or communicated among the I / O devices 102, for example, to enable automated control of the I / O devices 102 and to provide information to an operator or user of the I / O devices 102. The controller 104 can define or determine data flows and data flow characteristics within the TSN system 100.

[0026] Referring now to the 5G system 106 in the system 100, the 5G system 106 is a wireless communication system used to carry TSN traffic between various TSN end devices, e.g., between the I / O device 102 and the controller 104. In some implementations, the 5G system 106 is configured to emulate one TSN bridge per user plane function (UPF) (similar to the TSN bridge 108, in accordance with the TSN standards discussed above). The 5G system 106 may be a New Radio (NR) network implemented in accordance with the 3GPP® 23 and 38 series specifications (incorporated herein in their entireties) and integrated into the system 100 in accordance with the 3GPP® Release 17 23.501 standards v17.1.1 and v17.2.0 (incorporated herein in their entireties). As shown, the 5G system 106 may include a user equipment (UE) 118, a RAN (gNB) 120, a user plane function (UPF) 122 in the 5G user plane, and an application function (AF) 124 and a policy control function (PCF) 126 in the 5G control plane, among other components. In some implementations, the 5G system 106 may be configured to provide Ultra-Reliable Low Latency Communication (URLLC) services. The 5G system 106 based on the new over-the-air (NR) interface includes several features to achieve low latency for selected data flows. NR allows for shorter slots in the radio subframe, benefiting low latency applications. NR also introduces mini-slots, where prioritized transmissions can begin without waiting for a slot boundary, further reducing latency. As part of prioritizing URLLC traffic and providing faster radio access, NR introduces preemption, where URLLC data transmissions can preempt ongoing non-URLLC transmissions. In addition, NR applies very fast processing, allowing retransmissions even within low latency ranges.

[0027] In some implementations, 5G defines ultra-robust transmission modes to improve the reliability of both data and control radio channels, which is further improved by various techniques such as multi-antenna transmission based on multiple-input multiple-output (MIMO) techniques, the use of multiple carriers, and packet duplication over independent radio links.

[0028] Time synchronization is built into the operation of 5G cellular radio systems as an important part and was already a common practice in previous cellular network generations. The radio network components themselves are also time synchronized, for example via the Precision Time Protocol Telecommunication Profile. This provides a good basis for providing synchronization for time-critical applications. For URLLC services, the 5G system 106 uses time synchronization for its own operation as well as multiple antennas and radio channels to provide reliability. In addition to 5G RAN functions, the 5G system 106 may also provide solutions within the Core Network (CN) for Ethernet networking and URLLC. The 5G CN supports native Ethernet Protocol Data Unit (PDU) sessions. 5G supports the establishment of redundant user plane paths through the 5GS, including the RAN, CN, and transport networks. The 5GS also enables redundant user planes between RAN and CN nodes and between UE and RAN nodes separately.

[0029] As described above, in the integrated system 100, the 5G system 106, one TSN (virtual) bridge per UPF. The 5G system 106 includes a TSN translator (TT) function for adapting the 5G system 106 to the TSN domain for both the user plane and the control plane, and hides the internal procedures of the 5G system 106 from the TSN bridged network. The 5G system 106 provides TSN bridge ingress and egress port operation through the TT function. For example, the TT supports hold and forward functions for digital ring. Although FIG. 1 illustrates the case where the 5G system 106 connects the end station 102 to the bridged network 108, the 5G system 106 may also interconnect the bridges 108.

[0030] For the 5G system 106 to be integrated into the TSN system 100, the requirements of the TSN stream can only be met when the resource management allocates network resources for each hop along the entire path. This is achieved by the interaction between the 5G system 106 and the CNC 112 along with the TSN configuration (802.1Qcc). The interface between the 5G system 106 and the CNC allows the CNC 112 to learn the characteristics of the 5G virtual bridge and allows the 5G system 106 to establish a connection with specific parameters based on the information received from the CNC 112. Bounded latency requires deterministic delay from the 5G and QoS alignment between the TSN and 5G domains. For example, when the 5G virtual bridge acts as a TSN bridge, the 5G system 106 emulates time-controlled packet transmission along with scheduled traffic (802.1Qbv). For the 5G control plane, the TT in the AF 124 receives the transmission time information of the TSN traffic classes from the CNC 112. In the 5G user plane, the TT at the UE 118 and the TT at the UPF 122 may adjust time-based packet transmission accordingly. Different TSN traffic classes may be mapped to different 5G QoS Indicators (5QIs) in the AF 124 and PCF 126 as part of the QoS alignment between the TSN and 5G domains, and the different 5QIs are processed according to their QoS requirements.

[0031] With respect to time synchronization, the 5G system 106 may implement gPTP for the connected TSN network. The 5G system 106 may function as a virtual gPTP time-aware system and may support the transfer of gPTP time synchronization information between the end station 102 and the bridge 108 through the 5G user plane TT.

[0032] Reference is now made to FIG. 2, which illustrates a block diagram of a system 200 architecture in accordance with some implementations of the subject technology. In general, system 200 may be implemented as an integrated TSN-5G system similar to system 100 described above, and may include similar physical components as system 100 discussed above. However, unlike system 100, system 200 provides a new architecture for an integrated TSN-5G system, in which 5G system 106 is configured as a set of discrete 5G components, with each 5G component configured to be emulated as one discrete TSN block 202. In other words, in system 200, 5G system 106 is configured as a decomposed structure including multiple TSN blocks 202-1 to 202-N, with each TSN block 202 configured in accordance with a TSN specification (e.g., in accordance with IEEE 802.1 and related standards discussed above), for example, as a TSN bridge, a TSN end device, or a combination of the two. Moreover, instead of having a centralized configuration controller 110 for controlling the TSN-5G system, the subject technology provides multiple distributed configuration modules 215 within the distributed controller 210 of the TSN-5G system 200. The configuration modules 215-a-215-f may be interconnected in one or more topologies (mesh, star, tree), and each configuration module 215 may be responsible for communicating with and configuring one or more TSN blocks 202. As used herein, a "topology" may refer to one or more devices of a network, where the devices may include multiple nodes (e.g., sender devices, receiver devices, switches, or bridges) and connection lines (e.g., communication links, including wired or wireless communication links, or "hops") between the nodes in the network. Each link may communicatively couple a corresponding pair of nodes. A set of links may be coupled in turn through their respective nodes to define, for example, a link path between a source node and a destination node. The topologies may include, but are not limited to, one or more of mesh, star, bus, ring, and tree topologies.

[0033] In some implementations, the system 200 is configured to support and manage deterministic TSN data flows between a data source 204 and a data destination 206 through the 5G system 106 according to a TSN configuration including a TSN schedule determined by one or more of the configuration modules 215. The data source 204 and the data destination 206 may include one or more of the I / O devices 102 and the controller 104. Although not shown, the system 200 may also include a TSN bridge 108 and other TSN components.

[0034] In some implementations, in the decomposed structure, each of the multiple TSN blocks 202-1 to 202-N may correspond to one particular component of a 5G system, such as the 5G system 106 shown in FIG. 1 and discussed above. For example, the UE 118 may be configured to emulate as the TSN block 202-1, the RAN 120 may be configured to emulate as the TSN block 202-2, the UPF 122 may be configured to emulate as the TSN block 202-3, and the core network and / or other typical components of the 5G system (e.g., fronthaul, backhaul, multi-access edge computing (MEC) modules) may be configured to emulate as one or more TSN blocks 202-N. Each TSN block 202-1 to 202-N may be configured in accordance with a TSN specification (e.g., in accordance with IEEE 802.1 and related standards discussed above), for example, as a TSN bridge, a TSN end device, or a combination of the two.

[0035] In some implementations, the memory device of a TSN block 202 may store a set of parameters that describe the ability to support and perform data flows (e.g., transport URLLC data traffic) through the corresponding TSN block 202. In some implementations, the set of parameters includes, but is not limited to, identity, link quality, and link bandwidth. The identity parameter(s) may include a device type (i.e., whether the TSN block 202 is a TSN bridge or a TSN end station). The latency parameter(s) may include at least port-to-port (start of the TSN block to end of the TSN block) latency and latency variation (commonly known as "jitter"). The link quality parameter(s) may include at least a packet error rate. The link bandwidth parameter(s) may include at least an available bandwidth (in bits / second).

[0036] In some implementations, the set of parameters of the TSN block 202 may include a subset of parameters specific to 5G RAN, including, for example, short transmission time intervals, TSC assistance information (TSCAI), configured grant (CG) information, semi-persistent scheduling (SPS) allocation, and / or other parameters, as specified in 3GPP TS 28.540. Additionally, in some implementations, the set of parameters of the TSN block 202 may include a subset of parameters specific to TSN, including time synchronization characteristics, scheduled transmission (Qbv) attributes, redundancy attributes including number of connected RANs, number of paths to UPF, path diversity, number of available frequencies, propagation characteristics, available radios, different physical media (e.g., free space optics).

[0037] In some implementations, the set of parameters for the TSN block 202 may define a worst case time synchronization error, a worst case gate operation error, a maximum gate control list size, a maximum cycle time, a maximum gate interval duration, a transmission start delay, etc., or a combination thereof. The set of parameters may include a discrete set of deterministic parameters that may vary depending on the type of traffic processed by the TSN block 202. The set of parameters may be utilized, at least in part, to generate a TSN schedule, or configuration, etc., that is feasible on the hardware of the TSN block 202. In the absence of such a set of parameters, the TSN scheduling module must take a least common denominator approach, where all devices of the system 200, including the TSN block 202, are assumed to have the most restrictive characteristics, resulting in a suboptimal solution. In this sense, the set of parameters enables better scheduling solutions in the TSN system 200, resulting in improved performance metrics including latency, jitter, packet delay variation, and bandwidth utilization.

[0038] Referring back to FIG. 2, the subject technology provides a number of distributed configuration modules 215 in a distributed controller 210 of a TSN-5G system 200. The configuration modules 215-a-215-f may be interconnected in one or more topologies (mesh, star, tree), and each configuration module (CM) 215 may be responsible for communicating with and configuring one or more TSN blocks 202. For example, CM 215-a may be responsible for and operably and communicatively connected to TSN block 202-1 and TSN block 202-2. CM 215-b may be responsible for and operably and communicatively connected to TSN block 202-3. CM 215-c may be responsible for and operably and communicatively connected to TSN blocks 202-3 and 202-N. Thus, TSN block 202-3 may be configured and controlled by both CM 215-b and CM 215-c. For example, a portion of the functionality in TSN block 202-3 (e.g., relating to a first type of TSN application) may be configured and controlled by CM 215-b, and another portion of the functionality in TSN block 202-3 (e.g., relating to a second type of TSN application) may be configured and controlled by CM 215-c.

[0039] In the example shown in FIG. 2, the configuration modules 215 are arranged in a tree structure with CM 215-f forming the highest level of the tree structure, CM 215-a, 215-b, and 215-c forming the lowest level of the tree structure, and CM 215-d and 215-e being between the highest and lowest levels of the tree structure. However, the configuration modules 215 are operatively and communicatively connected to each other via API 230. In some implementations, in a tree structure (e.g., as shown in FIG. 2), a configuration module 215 may communicate with other configuration modules 215 at adjacent tree levels (one tree level above or below). However, in other topologies (e.g., mesh or peer-to-peer structures), any two configuration modules 215 in the distributed controller 210 may be operatively connected to each other and communicate directly with each other.

[0040] In some implementations, each configuration module 215 may be an external utility responsible for configuring one or more corresponding TSN blocks 202 or may be configured as a software module within the TSN block 202. Each configuration module 215 may be configured to determine and provide configuration data including a TSN schedule for one or more data flows transmitted through the TSN block 202 to the TSN block 202 controlled by the configuration module 215. The configuration modules 215 may exchange information with each other using a standardized API 230. The exchanged information may include information regarding the cycle time of the TSN system (e.g., discrete levels of administrative cycle time (ACT) buckets, each corresponding to a particular data flow, supported ACTs including maximum / minimum cycle times), configuration data including transmission schedules (including time offsets / durations / resources for transmission) of one or more TSN blocks 202, and information requesting or responding to requests for resource allocation. In some implementations, one or more of the configuration modules 215 may be configured as the CNC 112 and / or CUC 114 of the TSN controller 110 discussed above.

[0041] In some implementations, each configuration module (CM) 215 receives some or all of the set of parameters (discussed above) of the TSN block 202 from the ICI 406 of the TSN block 202 controlled by the CM 215. The CM 215 also receives information regarding the data flows to be configured through the TSN block 202 from another entity in the system 200 via the API 230. In a non-limiting example, the data source 204 and / or the data destination 206 provide the CM 215 with the requirements for the data flows between the data source 204 and the data destination 206, either directly or via an intermediary such as the CUC 114. In some implementations, the CM itself may allow a user to define a set of data flows to be configured via a user interface. As used herein, information regarding the data flows may include or define a set of data flows, data streams, transmission paths (predefined or otherwise adapted), etc., to define a desired TSN communication path between the data source 202 and the data destination 206. A non-limiting example set of information for a data flow may include maximum tolerable latency, data rate, data frame size ("payload"), data frame destination, band allocation gap, and the like, or combinations thereof.

[0042] Based at least on the received data flow information and a set of parameters of the TSN block 202, the CM 215 determines a "solution" (or configuration data), which will indicate how to process each data flow through the TSN block 202. This solution may include, among other things, time-aware schedules, policing rules, as discussed in U.S. Application No. 17 / 100,356, which is incorporated herein by reference in its entirety. The CM 215 may then send this solution or configuration data to the TSN block 202 via the ICI 406. The TSN block 202 may then execute this solution and transmit data for each flow according to its configuration. In some implementations, as an exemplary process for having the distributed CM 215 compute a solution, the same system modulo theory (SMT) solver may be used at each level of the CM 215 tree structure, where the data flows and their requirements are expressed as constraints, and linear programming is used to solve a feasible solution. Solutions from lower levels of the CM tree are entered as used resources (again represented by constraints) at the next higher level of the CM tree. This process is repeated until the top level of the CM tree is reached and a global solution is determined.

[0043] In some implementations, different data sources (e.g., data source 204) and their applications operate with different cycle times or intervals. Relatedly, different data sources and destinations (and their applications) require different levels of time determinism for the many data flows between them. In conventional TSN systems, a convergence cycle time (commonly known as a "management cycle time") is determined that works for all data flows in the network. However, in some implementations of the subject disclosure, the integrated TSN-5G system may use a set of discrete / quantized cycle times in the network. Each data flow selects one of the available quantized cycle times to operate on. Scheduling in the TSN-5G system 100 for scheduled transmissions may be based on a quantized / discrete set of cycle times. As an example, the integrated TSN-5G system 100 may limit the available stream intervals, and therefore the corresponding cycle times, to a set of discrete values, including but not limited to 1, 10, 100, 1000 milliseconds in nature. Similarly, stream or data flow requirements may be limited, for example, jitter (packet delay variation) requirements may be essentially limited to a set of predetermined discrete values, including but not limited to 1, 10, 100, 1000 microseconds. In some implementations, different sets of discrete values ​​may be used depending on the applications and use cases supported by the integrated TSN-5G system. For example, a geographically distributed system may use discrete cycle times on the order of milliseconds. In yet another example, a system limited to a local factory may use discrete cycle times on the order of microseconds. The members of this discrete set may be regularly or irregularly spaced or follow other statistical distributions (including but not limited to logarithmic, linear, Gaussian) without defaulting to a continuous set of cycle times. In another implementation, the set of cycle times is standardized in a manner that all TSN blocks have cycle times that are products of elements selected from a small common set of prime numbers.This ensures that all composite cycle times can be easily calculated by the TSN scheduler to result in one common network cycle time.

[0044] Each TSN block in the integrated TSN-5G system may support a set of cycle times (the set may include one or more). The CM 215 configures the TSN-5G system 106 or 200 to enable scheduled transmission of data flows across TSN blocks 202 operating with different cycle times. In some implementations, the TSN blocks 202 may be required to operate with corresponding cycle times, where corresponding means that the cycle times are integer multiples of each other. If an application requires an interval that does not directly map to the set of discrete cycle times available across the set of TSN blocks 202 that the flow traverses, the CM 215 may adapt to the closest available cycle time. The closest available cycle time would be an integer multiple or integer submultiple of the available cycle time. The CM 215 may exchange quantized / discrete sets of supported cycle time information with each other during the configuration process. In this example, the subject disclosure enables a decomposed TSN-5G system to create feasible configurations for multiple data streams / flows. Without quantized cycles / intervals, the construction typically requires large computation times and may even prevent the discovery of a feasible solution.

[0045] In some instances, each integrated TSN-5G network slice in the 5G system 106 may have a predefined set of supported cycle times and jitter limits. In some instances, the network slices may be more granular than typical 5G network slices, in accordance with 3GPP® Specification 23.501, which is incorporated herein by reference. In some implementations, the TSN-5G system 106 or 200 may be sliced ​​based on the TSN cycle time. For example, a 5G network supporting multiple critical services may have URLLC slices dedicated to the applications and the periodicity of their streams. For example, a service with an application that operates with a periodicity of about 1 millisecond may have a dedicated slice that operates with a cycle time of 1 millisecond. Similarly, a service and application that operates with a period (or interval) of 100 milliseconds may have a dedicated slice in the integrated TSN-5G system that operates with a cycle time of 100 milliseconds. Such cycle time slicing improves both the speed of configuration and the overall network performance. In some implementations, a TSN block 202 exposes the supported cycle time(s) of a given slice to its respective CM 215 via its ICI 406. The CMs 215 may exchange the supported cycle times of the TSN blocks under their control with each other via the inter-configuration module API 230 to create a configuration solution for a sliced ​​TSN-5G system.

[0046] In some implementations, the solution or configuration data determined by the CM 215 may include, but is not limited to, a collection or set of configurations, timing, commands, controls, instructions, and the like, or combinations thereof, for operating each TSN block 202 according to the characteristics (e.g., defined by a set of parameters) of the TSN block 202. In some aspects, the configuration data may include specific transmission information for individual or collective (e.g., “global”) data frame transmissions for one or more respective TSN blocks 202. The transmission information may include time information for the transmission of the data frames. In one or more aspects, the configuration data of the data frames may include a transmission start time. For example, the transmission start time may be a time when the transmission of the data frames from the respective TSN block 202 begins. In one aspect, the transmission of the data frames may be initiated by selective opening of a gate of the respective TSN block 202 to transmit the data frames as the data flows to a destination node (e.g., another TSN block 202). Conversely, the transmission of the data frames may be stopped or prevented by selective closing of a gate of the respective TSN block 202 to transmit the data frames. The configuration data may also define or allocate specific paths or links communicatively coupling each TSN block 202 to another node over which data flows are transmitted. In addition, the configuration data may define the duration of transmission of each data flow from each TSN block 202. In one aspect, the duration of data flow transmission may be defined by the period between selective opening of a gate (i.e., transmitting data frames) and selective closing of a gate of the respective node (i.e., ceasing transmission of data frames to the destination node).

[0047] In traditional TSN systems, the TSN schedule is expressed as an absolute time offset in a periodic cycle at which the TSN block is instructed to transmit its data. However, that may be too restrictive for a 5G system 106 that is composed of components from multiple vendors. In some implementations, a deadline-based schedule is determined and provided by the CM 215 along with the configuration data. The deadline-based schedule may instruct the TSN block 202 to transmit data of a configured data flow by a deadline (expressed in terms of absolute time in a periodic cycle). In some implementations, a delay budget-based approach would instruct the TSN block to transmit data frames of a configured data flow within a delay budget. Thus, under a delay budget-based approach, the TSN block 202 is required to transmit to its output port data frames that arrive at its input port within a certain duration. Such a scheme does not require all TSN blocks 202 to be time synchronized. In some implementations in which the TSN block 202 is configured under rate constraints, the TSN block 202 is configured to transmit data frames of a given data flow in a manner such that the average or peak transmission rate (in bits / second) does not exceed a configured value (according to configuration data from CM 215).

[0048] In some 5G systems, TSN blocks may be a set of shared resources made available by network slicing based on a service profile that is a periodic cycle that limits network latency, including but not limited to delay / budget. In such implementations, there may be two levels of scheduling that may allow the 5GS TSN-AF to configure the TSN blocks 202 as shared resources in addition to slice-level TSN scheduling. In either case, configuration attributes such as resource identifiers may identify the TSN blocks for appropriate configuration. As an example, a service provider may have multiple service profiles with specific periodic cycles, and multiple tenants of the service provider may utilize the same TSN blocks specified by the 5GS TSN-AF configuration and perform TSN flow aggregation. In some implementations, the service provider may offer a set of non-shared TSN blocks where only a single tier of service profiles may exist. Device-specific operation / required resource sharing modes may be available to the CNC 112 via the TSN-AF.

[0049] As an example implementation of the deadline / delay budget approach by the TSN block 202, when a data frame arrives at an input port of the TSN block 202, the TSN block 202 will record the arrival time of the data frame using its local clock. The TSN block 202 may then identify the frame as belonging to a configured data flow and then start a countdown timer equal to the configured delay budget of that data flow. Using the transmission module 408, the TSN block 202 may prioritize the transmission of data with the least amount of time remaining. If a packet's timer expires before it can be transmitted, the event is recorded as a transmission failure and accounted for by the recording module 410 in the monitoring metrics.

[0050] In some implementations, the scheduled transmission between the TSN block 202-1 (corresponding to the UE 118) and the TSN block 202-2 (corresponding to the RAN 120) may include an "enhanced" allocation (allocation and transmission) of uplink and downlink transmissions between the UE 118 and the RAN 120 to satisfy the schedule allocated by the CM 215-a to the TSN block 202-1 corresponding to the UE 118. In some implementations, the CM 215-a may take into account the UE 118 buffer status and radio conditions reported by the RAN 120 when instantiating the TSN schedule and may adjust or report necessary changes in the requested schedule. In some other implementations, the CM 215-a may send real-time feedback regarding the radio conditions received from the RAN 120 to the master CM, e.g., the CM 215-d. This feedback loop may support recalculating the TSN schedule to satisfy the packet delay budget at this particular TSN block or across TSN blocks on a given end-to-end path.

[0051] In this implementation, the link quality may be monitored and the CM 215-a may continuously adjust the radio resources in the configuration data to meet the transmission schedule. The radio resources may include, but are not limited to, logical channels, transmit power, and UE-specific slot durations. In some implementations, a static allocation of fixed / deterministic uplink and downlink slots for a given UE 118 may be made such that, for example, all UEs connected to a given RAN slice are given scheduled transmission slots. 5G native over-the-air scheduling may be used to determine whether a transmission from the UE 118 meets a transmission deadline. If not, the UE 118 may request elevated access to the RAN 120 to achieve the scheduled transmission. In some implementations, scheduling in the 5G system 106 for scheduled transmissions may be based on a quantized / discrete set of cycle times, the set including a management cycle time of at least 100 milliseconds. In accordance with the subject technology, a radio link between a UE (e.g., represented by TSN block 202-1) and a RAN (e.g., represented by TSN block 202-2) may be sliced ​​based on a cycle time. In some implementations, the uplink and downlink between TSN block 202-1 and TSN block 202-2 may have radio resources assigned to each network slice based on the cycle time of the slice. For example, a 1 millisecond cycle time slice requires radio resources (channels, airtime, etc.) that can deliver data at a 1 millisecond rate.

[0052] In some implementations, 5G resource elements (e.g., frequencies and time slots) may be scheduled such that they meet TSN flow latency requirements in addition to "standard" 5G scheduling traffic prioritization requirements. More specifically, 5G time slots for TSN flows may be assigned to transmit messages of the TSN flows at the appropriate cycle time offset (phase) and within the time limits (TSN window times) required by the TSN schedule. In this case, the 5G scheduler differs from a "traditional" TSN Ethernet port in that multiple messages may be output simultaneously if transmitted on different frequencies. In some aspects, in the presence of poor RF channel conditions, the 5G system 106 may send multiple copies of a message across different frequencies to increase the probability of meeting a transmission schedule and / or deadline.

[0053] To achieve reliable data transmission in the system 200, redundant flow paths may be implemented. The decomposed TSN block 202 of the 5G system 106 allows for handling of errors (delay, dropped, or corrupted frames) in a better manner. In some implementations, the UE 118 may initiate two redundant split PDU sessions to the UPF 122 for redundancy, in which case the 5GC may configure the NG-RAN for dual connectivity according to 3GPP 38.300. In some other implementations, FRER may be used between some TSN blocks 202 but not others. For example, redundant streams may be implemented over the air interface between the UE 118 (TSN block 202-1) and the RAN 120 (TSN block 202-2), then combined at the RAN 120 and split again over the core network (TSN blocks 202-3, 202-4) if necessary. As shown in FIG. 1, the current redundancy requirement according to 3GPP 23.501 is a maximally separated path between the UE 118 and the UPF 122. However, according to the subject technology, it is not necessary to establish redundant separated paths throughout the entire 5G system, but instead may be implemented in only a portion of the 5G system. For example, in the case of an over-the-air interface between the UE 118 and the RAN 120, the redundancy requirement may specify that the two paths should be on different frequencies or different MIMO channels or different time slots. In some instances, the subject disclosure allows for flexible use of redundancy, including but not limited to supporting more than two data paths, merging and splitting of data flows between TSN blocks, and TSN blocks with various degrees of redundancy capabilities.

[0054] Referring to FIG. 4, an application data stream may span multiple integrated TSN-5G systems 805 and 810. In this case, the TSN blocks may be geographically distributed and interconnect two or more 5G systems using TSN-enabled transports 820, including but not limited to 5G backbones, private network tunnels, or other wide area networks. In this subject invention, the TSN blocks are configured by their respective CMs 215. In some implementations, configuration across the two TSN-5G systems 805, 810 may be coordinated through a centralized configuration utility (CNC) 112. In some other implementations, the CMs 215 between the two TSN-5G systems 805, 810 may communicate directly. In some implementations, multiple CUCs and CNCs may be used to capture user requirements and generate the TSN solutions and techniques (e.g., TSN schedules, forwarding instructions, etc.) provided in this disclosure.

[0055] 3 illustrates an electronic system 700 in which one or more implementations of the subject technology may be implemented. The electronic system 700 may be and / or may be part of the TSN block 202 and / or the configuration module 215. The electronic system 700 may include various types of computer readable media and interfaces for various other types of computer readable media. The electronic system 700 includes a bus 708, one or more processing units 712, a system memory 704 (and / or buffers), a ROM 710, a permanent storage device 702, an input device interface 714, an output device interface 706, and one or more network interfaces 716, or a subset or variation thereof.

[0056] The bus 708 collectively represents all system, peripheral, and chipset buses that communicatively connect the numerous internal devices of the electronic system 700. In one or more implementations, the bus 708 communicatively connects one or more processing units 712 to the ROM 710, the system memory 704, and the permanent storage device 702. From these various memory units, the one or more processing units 712 retrieve instructions to execute and data to process in order to perform the processes of the subject disclosure. The one or more processing units 712 can be a single processor or a multi-core processor in different implementations.

[0057] The ROM 710 stores static data and instructions required by one or more processing units 712 and other modules of the electronic system 700. The permanent storage device 702, on the other hand, may be a read-and-write memory device. The permanent storage device 702 may be a non-volatile memory unit that stores instructions and data even when the electronic system 700 is off. In one or more implementations, a mass storage device (such as a magnetic or optical disk and its corresponding disk drive) may be used as the permanent storage device 702.

[0058] In one or more implementations, a removable storage device (such as a floppy disk, a flash drive, and its corresponding disk drive) may be used as the permanent storage device 702. Like the permanent storage device 702, the system memory 704 may be a read-and-write memory device. However, unlike the permanent storage device 702, the system memory 704 may be a volatile read-and-write memory, such as a random access memory. The system memory 704 may store any of the instructions and data that the one or more processing units 712 may need during execution. In one or more implementations, the processes of the subject disclosure are stored in the system memory 704, the permanent storage device 702, and / or the ROM 710 (each of which is implemented as a non-transitory computer-readable medium). From these various memory units, the one or more processing units 712 retrieve instructions to execute and data to process in order to execute the processes of the one or more implementations.

[0059] The bus 708 also connects to input and output device interfaces 714 and 706. The input device interface 714 allows a user to communicate information and select commands to the electronic system 700. Input devices that may be used with the input device interface 714 may include, for example, an alphanumeric keyboard and a pointing device (also referred to as a "cursor control device"). The output device interface 706 may allow, for example, the display of images generated by the electronic system 700. Output devices that may be used with the output device interface 706 may include, for example, printers and display devices, such as liquid crystal displays (LCDs), light emitting diode (LED) displays, organic light emitting diode (OLED) displays, flexible displays, flat panel displays, solid state displays, projectors, or any other device for outputting information. One or more implementations may include devices that function as both input and output devices, such as a touch screen. In these implementations, feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or haptic feedback, and input from the user may be received in any form, including acoustic, velocity, or tactile input.

[0060] 3, bus 708 also couples electronic system 700 to one or more networks and / or one or more network nodes via one or more network interfaces 716. In this manner, electronic system 700 can be part of a network of computers, such as a LAN, a wide area network ("WAN"), or an intranet, or a network of networks, such as the Internet. Any or all of the components of electronic system 700 can be used in conjunction with the subject disclosure.

[0061] In some embodiments, the TSN 5GS may implement the IEEE 802.1Qbv standard. The 802.1Qbv TSN standard provides scheduled transmission for safety-critical data frames in a prescribed manner and is incorporated herein by reference in its entirety. A scheduler according to the 802.1Qbv TSN standard may be referred to herein as a "TSN scheduler." The TSN scheduler determines when the TSN interfaces on each node send frames to reach their destination at the required time intervals while also avoiding conflicts with other time-dependent frames along their path. The TSN scheduler requires information about the topology and link speeds of the network, as well as the end-to-end latency required for each path, to calculate when to send frames. The IEEE 802.1Qbv standard also defines how to load the schedule into the TSN interface.

[0062] Turning now to FIG. 5A, a block diagram of the inputs and outputs of a TSN scheduler is shown, according to one embodiment of the present disclosure. The TSN scheduler 900 takes schedule input information 910, which may include network descriptions and link requirements, and returns schedule output information 920. The schedule input information may also include information related to time-dependent paths through the network and required end-to-end latency, as well as physical link bandwidth. The schedule output information 920 may include nodes, links, and frame transmission times to create a list of when each node's interface will transmit frames to achieve the required latency without interference from other time-dependent traffic.

[0063] The TSN scheduler needs to find the gate-opening time for each virtual link such that frames do not appear at the same time on the same output port. This requires knowledge of the network topology. To guarantee minimum latency, knowledge of the frame size and physical channel rate is required. The user is assumed to constrain the minimum rate by specifying the allowed cycle time during which the gate is open for each virtual link. The scheduler can shift the cycle offset based on knowledge of the network topology and other time-dependent flows.

[0064] In some embodiments, the TSN scheduling algorithm may start with an adjacency matrix that represents paths and cycle times through the network. The TSN scheduling algorithm may output a matrix with feasible cycle offsets for each virtual link. The feasible cycle time offsets are cycle time offsets for each virtual link such that frames do not appear simultaneously on the same output port. A successful TSN schedule output ensures that there is no overlap with adjacent virtual link cycles, which may be achieved by sliding the cycle times on the virtual links.

[0065] Consider an adjacency matrix A describing the network topology in terms of frame transmission times, V describing the path in terms of cycle lengths, and O describing the offset for each cycle length. Assume that each cycle time starts at time zero. Since the offset plus the transmission time cannot be greater than a cycle, each element of O+A must be less than each element of V. If any adjacent weights in O+A overlap in start and end time, the corresponding Ethernet frames will interfere. The frame transmission start time of the network is described by O, and the frame transmission end time of the network is described by O+A. This means that the adjacent links must be checked such that O of one link occurs either before O of the adjacent link or after O+A of the adjacent link. However, the propagation delay as the frame travels from one link to another must be taken into account. Thus, the previous offsets experienced by the frame need to be accumulated, keeping in mind that they may wrap around a cycle if the number of links traversed is long enough.

[0066] The IEEE 802.1Qbv TSN scheduling standard also specifies how to configure gates that control the output from queues of each port of each interface in the network. In some embodiments, up to eight queues may be used on a port that are programmed to open and close according to a periodic schedule. The essence of IEEE 802.1Qbv scheduling is that frame transmission from each queue associated with a traffic class (TC) is scheduled for a known timescale. To achieve this, a transmit gate is associated with each queue, and the state of the transmit gate determines whether a queued frame can be selected for transmission. A transmit gate has two states: open and closed. A gate control list is associated with each port and contains an ordered list of gate actions. Each gate action changes the transmit gate state of the transmit gate associated with each of the port's TC queues according to a scheduled time. The period during which the sequence of gate actions in the gate control list repeats is called a "gate cycle." IEEE 802.1Qbv specifies a list of parameters (e.g., a "gate parameter table") that support the enforcement of scheduled traffic. In the centralized model, IEEE 802.1Qcc, a centralized network controller (CNC) calculates the gate behavior based on the stream characteristics and configures the transmit gates on the ports of the TSN bridge accordingly.

[0067] In some embodiments, the TSN scheduler may be configured to perform scheduling at the output ports of the 5GS bridge using output pacing with de-jitter buffer functionality. 3GPP® S2-1901150: “TSN QoS and Traffic Scheduling in 5GS.” The 5GS may be configured to communicate TSN traffic patterns and corresponding scheduling information from the CNC to associated nodes in the 5GS. As used herein, TSN traffic patterns may include information to support TSN, such as a gating mechanism to schedule TSN traffic. This part of the information may be used as an additional parameter for 5GS to meet QoS requirements for TSN communications, which are not included in the 5G QOS model. 3GPP® Technical Report (TR) 23.734 Solution #30 (see 3GPP® TR23.734 Draft V16.0.0+(20198-102)): “Study on 5GS Enhanced support of Vertical and LAN Services”.

[0068] The TSN schedule must determine the cycle time and configure each gate to open and close at the appropriate time within the cycle. In some embodiments, the TSN scheduler may be integrated into a larger set of programming tools that may in turn be used and may include data distribution services, operating systems, one or more task schedulers, or control systems that, alone or in combination, may optimize when and where Ethernet frames should be sent and received, and what type of Quality of Service (QoS) they require.

[0069] In some TSN networks, non-TSN traffic may also need to be routed. Although non-TSN traffic may not have the strict latency and delivery time requirements of TSN traffic, network performance and throughput of non-TSN traffic is also important. Non-TSN traffic may also be referred to as "best effort" traffic. In some embodiments, the TSN scheduler may also be used to transmit best effort traffic, for example, during times not reserved for TSN traffic. Time-dependent traffic may occupy reserved periodic time slots, which may be configured at different times at each node, while unreserved time slots may remain available for best effort traffic.

[0070] In one or more embodiments, the TSN scheduler creates a set of constraints and solves a solution that satisfies the constraints. In particular, the TSN scheduler may generate a schedule that matches unscheduled communication data flows with scheduled flows of communication through the network. In one or more embodiments, the TSN scheduler may receive as input the destination of the data flow and the expected arrival time of that communication at the destination. The expected arrival time may be in the form of a maximum acceptable latency. Then, based on this information, the TSN scheduler may generate a schedule. In one or more embodiments, the schedule may include instructions on when to open and close one or more gates of one or more network queues to allow transmission of the communication. In one or more embodiments, the TSN scheduler may solve the problem of allowing multiple flows of traffic to exist on the same Ethernet network such that Ethernet frames reach their destination at a given time, regardless of the topology of the network or the rate of traffic flows running within the network. The TSN scheduler may use algorithms to perform mathematical analysis on the given inputs in order to create a feasible schedule. Inputs to the algorithm may relate to the topology of the network and the requirements of the TSN traffic in terms of time dependency, latency, and message size. In some embodiments, the TSN scheduler may operate offline to generate the schedule and then be manually installed in the networking equipment that implements the schedule. A process for generating a TSN schedule for opening and closing gates based on schedule input information is described in U.S. Pat. No. 11,121,978, which is incorporated herein by reference in its entirety.

[0071] Turning now to FIG. 5B, a diagram illustrating 5GS scheduling functionality is shown, in accordance with one embodiment of the present disclosure.

[0072] Referring to 5G scheduler diagram 901, a scheduler according to the above standards for 5GS is shown, which may be referred to herein as the "5G scheduler". The 5G scheduler may include one or more dynamic resource schedulers that allocate physical layer resources for the downlink and uplink. The 5G scheduler may take into account user equipment (UE) buffer status and quality of service (QoS) requirements (e.g., guaranteed (or non-guaranteed) bitrates, allocation and retention priorities, reflected QoS attributes, maximum bitflow rates, and maximum packet loss rates). The 5G scheduler may allocate resources by taking into account radio conditions at the UE, which may be identified through measurements made over the air and / or reported by the UE. The 5G scheduler decision may be signaled by the UE, which identifies the resources to be used by receiving a scheduling (resource allocation) channel.

[0073] The 5G scheduler may generate a schedule for one or more UEs for 5GS based on one or more inputs described herein. In some embodiments, the 5G scheduler may generate a schedule including carrier-specific transmit and receive times for 5GS components in at least one embodiment of the present disclosure. In some embodiments, 5GS may include a multi-carrier orthogonal frequency division multiplexing (OFDM) communication system that may include one or more carriers, for example, ranging from 1 to 32 carriers in the case of carrier aggregation, or from 1 to 64 carriers in the case of dual connectivity. Different radio frame structures may be supported (e.g., for frequency division duplexing (FDD) and for time division duplexing (TDD) duplexing mechanisms). Frame timing chart 910 illustrates an example frame timing according to at least one embodiment of the present invention. Downlink and uplink transmissions may be organized within a radio frame 911. In this example, the radio frame duration is 10 milliseconds. In this example, a 10 ms radio frame 911 may be divided into 10 equally sized subframes 912 having a duration of 1 ms. The subframe(s) may include one or more slots (e.g., slots 913 and 915) depending on the subcarrier spacing and / or CP length. For example, subframes with subcarrier spacing of 15 kHz, 30 kHz, 60 kHz, 120 kHz, 240 kHz, and 480 kHz may include 1, 2, 4, 8, 16, and 32 slots, respectively. In the frame timing diagram 910, the subframe may be divided into two equally sized slots 913 having a duration of 0.5 ms. For example, 10 subframes may be available for downlink transmission and 10 subframes may be available for uplink transmission with a time interval of 10 ms. The uplink and downlink transmissions may be separated in the frequency domain. The slot(s) may include multiple OFDM symbols 914. The number of OFDM symbols 604 in a slot 915 may depend on the cyclic prefix length.For example, one slot can be 14 OFDM symbols with the same subcarrier spacing of up to 480 kHz with normal CP. One slot can be 12 OFDM symbols with the same subcarrier spacing of 60 kHz with extended CP. One slot can include downlink, uplink, or downlink and / or uplink portions, and / or the like.

[0074] In some embodiments, the 5G scheduler may generate a schedule including an OFDM resource grid for one or more subframes discussed above. An exemplary resource grid is shown in resource grid diagram 950. In one example, a carrier may have a transmission bandwidth 951. In one example, the resource grid may be in a frequency domain 952 and time domain 953 structure. In one example, the resource grid may include a first number of OFDM symbols in a subframe and a second number of resource blocks, starting with a common resource block indicated by higher layer signaling (e.g., RRC signaling) for the transmission numerology and carrier. In one example, in the resource grid, the resource unit identified by the subcarrier index and symbol index may be a resource element 955. In one example, a subframe may include a first number of OFDM symbols 957 depending on the numerology associated with the carrier. For example, if the subcarrier spacing of the carrier's numerology is 15 KHz, the subframe may have 14 OFDM symbols for the carrier. If the subcarrier spacing of the numerology is 30 KHz, the subframe may have 28 OFDM symbols. If the subcarrier spacing of the numerology is 60 KHz, the subframe may have 56 OFDM symbols, etc. In one example, the second number of resource blocks included in the resource grid of a carrier may depend on the bandwidth and numerology of the carrier.

[0075] As shown in resource grid 950, resource block 956 may include 12 subcarriers. In one example, multiple resource blocks may be grouped into resource block groups (RBGs) 954. In one example, the size of the RBG may depend on at least one of an RRC message indicating an RBG size configuration, a size of the carrier bandwidth, or a bandwidth portion of the carrier. In one example, a carrier may include multiple bandwidth portions. A first bandwidth portion of a carrier may have a different frequency location and / or bandwidth than a second bandwidth portion of the carrier.

[0076] For downlink scheduling, the 5G radio can dynamically allocate resources to the UE via a Cell Radio Network Temporary Identifier (typically C-RNTI for Cell RNTI) on one or more physical downlink control channels (PDCCH(s)). The UE can monitor the PDCCH(s) to find possible allocations when its downlink reception is enabled (an activity governed by DRX when configured). If carrier aggregation (CA) is configured, the same C-RNTI applies to all serving cells.

[0077] The gNB may preempt an ongoing physical downlink shared channel (PDSCH) transmission to one UE with a latency-critical transmission to another UE. The gNB may configure the UE to monitor an interrupted transmission indication using an interrupted RNTI (INT-RNTI) on the PDCCH. If the UE receives an interrupted transmission indication, the UE may assume that no useful information to the UE was conveyed by the resource elements included in the indication, even if some of those resource elements were already scheduled to this UE.

[0078] With Semi-Persistent Scheduling (SPS), the gNB can allocate downlink resources for the first Hybrid Automatic Repeat Request (HARQ) transmission to the UE. Radio Resource Control (RRC) defines the periodicity of the configured downlink allocation, and the PDCCH addressed to the Configured Scheduling RNTI (CS-RNTI) can signal and activate or deactivate the configured downlink allocation. That is, the PDCCH addressed to the CS-RNTI indicates that the downlink allocation can be implicitly reused according to the periodicity defined by the RRC until it is deactivated. Retransmissions can be explicitly scheduled on the PDCCH(s) if they are needed.

[0079] Dynamically assigned downlink receptions overwrite configured downlink allocations in the same serving cell if they overlap in time. If they do not overlap, downlink reception is assumed according to the configured downlink allocations if activated. A UE may be configured with up to eight active configured downlink allocations for a given bandwidth portion (BWP) of a serving cell. If more than one is configured, the network decides which of the configured downlink allocations will be active at a time, which may be all the allocations. If more than one downlink allocation is configured, each configured downlink allocation is activated separately using a downlink control information (DCI) command, and deactivation of the configured downlink allocations is done using a DCI command, which may deactivate a single configured downlink allocation or may jointly deactivate multiple configured downlink allocations.

[0080] For uplink scheduling, the gNB can dynamically allocate resources to the UE via the C-RNTI on the PDCCH(s). The UE constantly monitors the PDCCH(s) to find possible grants for uplink transmissions when its downlink reception is enabled (an activity governed by Discontinuous Reception (DRX) when configured). If CA is configured, the same C-RNTI applies to all serving cells.

[0081] The gNB may cancel a UE's Sounding Reference Signal (SRS) transmission for another UE with a Physical Uplink Shared Channel (PUSCH) transmission, or a repeat of a PUSCH transmission, or a latency-critical transmission. The gNB may configure the UE to monitor a canceled transmission indication using a Cancellation RNTI (CI-RNTI) on the PDCCH. If the UE receives a canceled transmission indication, the UE cancels the PUSCH transmission from the earliest symbol that overlaps with the resource or cancels the SRS transmission that overlaps with the resource indicated by the cancellation.

[0082] With the configured grant, the gNB can allocate uplink resources for initial HARQ transmission and HARQ retransmission to the UE. In one type of configured uplink grant (type 1), the RRC directly provides the configured uplink grant (including periodicity). In a second type of configured uplink grant (type 2), the RRC defines the periodicity of the configured uplink grant, while the PDCCH addressed to the CS-RNTI can signal and activate or deactivate the configured uplink grant. In other words, the PDCCH addressed to the CS-RNTI indicates that the uplink grant can be implicitly reused according to the periodicity defined by the RRC until it is deactivated. If the UE is not configured with enhanced intra-UE overlapping resource prioritization, dynamically assigned uplink transmissions overwrite configured uplink grants in the same serving cell if they overlap in time. If they do not overlap in time, uplink transmission according to the configured uplink grant is assumed if activated.

[0083] If the UE is configured with enhanced intra-UE overlapping resource prioritization, when a configured uplink grant transmission overlaps in time with a dynamically assigned uplink transmission or with another configured uplink grant transmission in the same serving cell, the UE prioritizes the transmission based on a comparison of the highest priority of a logical channel having data to be transmitted that is or can be multiplexed in the MAC PDU associated with the overlapping resource. Similarly, when a configured uplink grant transmission or a dynamically assigned uplink transmission overlaps in time with a scheduling request transmission, the UE prioritizes the transmission based on a comparison of the priority of the logical channel that triggered the scheduling request with the highest priority of a logical channel having data to be transmitted that is or can be multiplexed in the MAC PDU associated with the overlapping resource. If a MAC PDU associated with a non-prioritized transmission has already been generated, the UE stores it to enable the gNB to schedule a retransmission. The UE may also be configured by the gNB to transmit the stored MAC PDU as a new transmission using subsequent resources of the same configured uplink grant configuration when no explicit retransmission grant is provided by the gNB.

[0084] Retransmissions other than repetitions may be explicitly assigned via the PDCCH(s) or via configuration of a retransmission timer. The UE may be configured with up to 12 active configured uplink grants for a given BWP of the serving cell. If more than one is configured, the network decides which of these configured uplink grants are active at a time (including all of them). Each configured uplink grant may be either Type 1 or Type 2. If Type 2, activation and deactivation of configured uplink grants is independent between serving cells. If more than one Type 2 configured grants are configured, each configured grant is activated separately using a DCI command, and deactivation of Type 2 configured grants is done using a DCI command, which may deactivate a single configured grant configuration or may jointly deactivate multiple configured grant configurations.

[0085] If SUL is configured, the network must ensure that an active configured uplink grant on SUL does not overlap in time with another active configured uplink grant on the other UL configuration. For both dynamic and configured grants, for a transport block, more than one repetition can be in one slot, or each repetition in one slot can cross a slot boundary in consecutive available slots. For both dynamic and configured grants type 2, the repetition count can also be dynamically indicated in L1 signaling. The dynamically indicated repetition count takes precedence over the RRC configured repetition count if both are present.

[0086] The 5G scheduler may include measurement reports that are required to enable the scheduler to operate in both uplink and downlink. These measurement reports may include transport quantities and measurements of the UE radio environment. Uplink buffer status reports (BSRs) may be used to provide support for QoS-aware packet scheduling. In NR, uplink buffer status reports may refer to buffered data for a group of logical channels (LCGs) in the UE. Eight LCGs and two formats may be used for reporting in the uplink. Formats may include a short format for reporting only one BSR (of one LCG) and a flexible long format for reporting several BSRs (up to all eight LCGs).

[0087] Uplink buffer status reports are transmitted using MAC signaling. When a BSR is triggered, which can occur when new data arrives in the UE's transmit buffer, a Scheduling Request (SR) can be sent by the UE (e.g., when there are no resources available to send a BSR). In case of integrated access and backhaul (IAB), a preemptive BSR can be configured on the backhaul link. The preemptive BSR can be sent based on expected data instead of buffered data.

[0088] Power headroom reports (PHR) may also be generated. The PHR measures the difference between the nominal UE maximum transmit power and the estimated power for uplink transmission. To allow the network to detect UL power reduction, the PHR report may also contain power management maximum power reduction (P-MPR, see TS38.101-2

[35] ) information that the UE uses to ensure compliance with the FR2 Maximum Permissible Exposure (MPE) exposure regulations, which limit RF exposure to the human body. Power headroom reports are transmitted using MAC signaling.

[0089] Turning now to FIG. 6, a block diagram of a TSN communication system including a scheduler and a corresponding 5GS network is shown. In one or more embodiments, the network may include multiple destinations 1020 or nodes, including 5GS. The nodes may be connected to the communication system via one or more communication paths 1022 or links. The communication links 1022 may be connected to each other via ports and / or switches 1020. In one or more embodiments, two or more data frame transmission paths or flows may overlap. Data frames 1004 may collide where these transmission paths overlap, and the collision may result in frames being dropped and not delivered to their respective destinations 1020. Thus, the TSN scheduler 1027 may fit non-scheduled / best effort frames into the schedule 1010 with scheduled frames, so that the data frames 1004 do not collide and instead reach the appropriate destination at the appropriate time.

[0090] In one or more embodiments, the TSN network may include multiple queues 1012 (e.g., queues 0, 1, 2, 3, 4...7, etc.) for transmitting data frames 1004 to respective destinations 1020. In one or more embodiments, queues may exist at every interface, both on end systems (e.g., devices) and at each port (connection) of the switch 1001. In one or more embodiments, each queue 1012 may include a gate 1013 that may be in an open position 1014 or a closed position 1016, and may allow transmission of the data frames 1004 only when in the open position 1014. In one or more embodiments, the operation of the queue gates may be synchronized to the same clock 1018. Of note, synchronization is important, especially for high priority traffic, to avoid collisions and ensure that the gates are closed at exactly the right time to get the data frames through the network according to the schedule 1010. In one or more embodiments, based on the received input, the scheduler 1027 performs calculations to determine the opening / closing gate times along the path of the flow to meet the destination 1020 and arrival time (e.g., within a maximum latency) as specified by the application. In one or more embodiments, the contents of the schedule 1010 specify the opening / closing of gates along the path of the flow as described in the TSN standard.

[0091] In one or more embodiments, the TSN scheduler 1027 located at the switch 1001 receives input from at least one application to create the schedule 1010. Although FIG. 6 shows the scheduler 1027 located at the switch 1001, the TSN scheduler 1027 may reside anywhere in the network. In one or more embodiments, the TSN scheduler 1027 may communicate with all switches and end systems to configure them. The input may include at least one of one or more data frames 1004, a destination 1040 for the data frames, and one or more data packets of maximum latency 1024. The TSN scheduler 1207 may receive other suitable inputs. For example, the TSN scheduler 1027 may also receive network topology descriptions and path or link requirements 1006 (e.g., indications of time-dependent paths, physical link bandwidth, size of frames ("payload")) from the application 113 and / or tool chain, or any other suitable source. The scheduler 1027 may then generate a schedule 1010 for the communication traffic through the network.

[0092] The TSN scheduler described herein, and the 802.1Qbv TSN standard, generally relate to Ethernet traffic. The TSN scheduler may be implemented in some instances, for example, in 5GS on 5G gNB radios. 3GPP TS38.300 Version 16.4.0 Release 16 Section 10 is a standard that describes the scheduling performed by 5G gNB radios to optimize network throughput therein, and is incorporated herein by reference in its entirety.

[0093] The TSN scheduler and the 5G scheduler may work together on the same network, and the 5G scheduler may use information from the TSN scheduler to inform its scheduling process. In some embodiments, this information may include the exact arrival times of the TSN messages at different nodes, and the message sizes. The TSN scheduling information may also be included in the radio conditions at the UE, which may be reported by the UE to the 5G scheduler. As discussed above, the TSN scheduler outputs a list of when each node's interface should transmit frames to achieve the required latency without interference from other time-dependent traffic. The schedule data may also include frame sizes, priority levels, packet delay budgets, and packet error rates. Knowledge of the TSN scheduling elements may be used in measurement reports and / or BSRs, and their presence may allow the 5G scheduler to allocate resources more efficiently.

[0094] As discussed above, the 5G schedule takes as input the UE buffer status and QoS requirements of each UE and associated radio bearer and allocates radio resources (e.g., resource blocks) between the UEs. The 5G scheduler may also allocate resources according to radio conditions at the UEs, which may be identified through measurements made on the gNB radio and / or measurements reported by the UE. The 5G scheduler may generate the 5G schedule based on the arrival times of TSN frames in its schedule. The presence of these arrival times that do not need to be scheduled by the 5G scheduler and are already in the schedule before the rest of the traffic is scheduled may free up the 5G scheduler process to more efficiently schedule best effort frames in the remaining available time slots after the TSN frames have already been scheduled. The presence of the equipment required to support the throughput required for on-time TSN delivery may then also be used to increase the delivery rate of non-TSN frames that are scheduled in parallel.

[0095] While normal "best effort" traffic may have some associated QoS information, the 5G schedule, having more specific information delivered from the TSN scheduler, enhances the scheduler's ability to allocate radio resources and resource blocks to best effort traffic, as the TSN scheduler informs the 5G scheduler when gates are opened, closed, and closed for transmission. The 5G scheduler can then use the gate status information to route other traffic to other gates for more efficient delivery. More specifically, the 5G scheduler will allocate resource blocks and elements such that messages and frequencies are partitioned in a manner that meets the TSN flow requirements specified in the TSN schedule.

[0096] Knowledge of TSN scheduling information can also serve to optimize multiple-input / multiple-output (MIMO) antenna operation at a lower physical level than the gNB scheduler. In other words, different antenna configurations may be implemented based on the TSN schedule. For example, more antennas may be dedicated to higher priority TSN flows than lower priority TSN flows. Additionally or alternatively, when a higher priority TSN flow is transmitting over the network, one or more antennas with higher signal transmission capabilities may be selected over antennas with lower signal transmission capabilities, and vice versa.

[0097] Output from the 5G scheduler may also be useful information to the TSN scheduler for updating the TSN schedule. The 5G scheduling information may include a Buffer Status Report (BSR) that measures data buffered in logical channel queues in the UE. These reports may be useful as part of the physical link bandwidth information input to the TSN scheduler. The 5G scheduler's Power Headroom Report (PHR) may also provide transmit power information for the UE that may inform the TSN scheduler's schedule input information about the physical link bandwidth. The 5G scheduling information may include QoS flow characteristics defined in 3GPP TS23.501 Table 5.7.4-1 Standardized 5QI to QoS Characteristics Mapping (incorporated herein by reference in its entirety). The QoS flow characteristics may include resource type, default priority level, packet delay budget, packet error rate, default maximum data burst amount, and / or default averaging window. Since the 5G scheduler may be optimized to meet the traffic classes defined in 23.501 - Table 5.7.4-1: Standardized 5QI to QoS characteristic mapping categories, in some embodiments the wired TSN scheduler may be adapted to use the latency and jitter of the aforementioned classes when creating scheduled paths over 5GS.

[0098] Referring to FIG. 6, in some embodiments, the TSN scheduler 1027 may include a 5GS emulation system (SES) 1030. The SES 1030 may be a "digital twin" of the 5GS or a component thereof, including, among other things, one or more of devices, traffic, base stations, RF channels, network slice information, MEC operating characteristics, and user data. For example, the SES 1030 may be a virtual representation of the 5GS spanning the 5GS lifecycle, updated from real-time data, and using simulation, machine learning, and inference to support decision making. The SES 1030 may evolve in sync with the 5GS throughout the 5GS lifecycle. In some embodiments, the SES 1030 may receive network configuration and operational data of the 5GS (or a component thereof). SES 1030 may utilize such network configuration and operational data (including network load data and network slice data), in addition to other input factors described herein for generating a TSN schedule, to emulate 5GS and provide 5GS forecast information to facilitate generation of a TSN schedule by TSN scheduler 1027.

[0099] Integrated TSN-5G scheduling may be implemented in some aspects using distributed algorithms and systems for cross-domain TSN scheduling. In an integrated TSN-5G environment, 5GS may be viewed as a single bridge for TSN scheduling. If 5GS were an actual bridge, this would increase latency as TSN flows would have to traverse the switching fabric on their way to the bridge output queues. However, 5GS avoids such switching fabric delays by scheduling across non-interfering time and frequency resource blocks. Therefore, a wireless self-configuring TSN scheduler mechanism can be built using 5GS schedule feedback. The 5G bidirectional scheduling status information message allows an improved model of the 5G virtual bridge to be built into the TSN scheduler. Information that is already part of the 5G standard can be used to inform the TSN scheduler to create a schedule that works better with 5GS.

[0100] Joint or integrated TSN-5G scheduling may be achieved, in one aspect of the disclosure, via distributed scheduling. Accordingly, the disclosure relates to a 5G bidirectional scheduling status information message. The message may include expected TSN user equipment (UE) configuration from the TSN application scheduler and may return 5G scheduler information such as that found in system information block (SIB) scheduling information. Further information regarding SIB scheduling information may be found in 3GPP TS38.331 Version 15.5.1 Release 15 ("5G, NR, Radio Resource Control (RRC), Protocol Specification"), which is incorporated herein by reference in its entirety.

[0101] The desired scheduling improvements, with bidirectional scheduling status information passing between the 5G scheduler and the TSN scheduler, may be achieved via a distributed scheduling algorithm, which allows multiple TSN schedulers, separated by domains, to function simultaneously, so that they may inform each other of their scheduling data, similar to that of 5GS.

[0102] At a higher level, multiple TSN schedulers may be used such that each scheduler may cover a different domain. However, the different domains are interconnected. The domains may be divided according to different criteria, and multiple criteria may be used. The domains may be separated in space. Separating the domains by space may be the easiest criterion to implement, since each device along the network may be its own domain, or may be grouped with devices that have similarities, such as proximity in space. When domains are partitioned by space, scheduling domains with common links may be treated as peers.

[0103] Regions may also be partitioned by network flows, which may more specifically include partitioning by network slices. Network slicing is a virtualization concept that allows a physical network to be virtually divided into different networks that can emphasize different characteristics or allocate different resources depending on different use cases or needs of the network. For example, the same physical, e.g., 5G wireless network, may have a need for a network that prioritizes high speed, another network that prioritizes low latency, and a third network with specific requirements for quality of service (QoS). Accordingly, the network may be virtually divided into slices that allocate resources to suit both of those needs. A distributed TSN scheduler may assign a different TSN scheduler for each network slice.

[0104] A domain may also be partitioned by time. TSN scheduling is performed based on a time cycle, e.g., each clock cycle allows time-sensitive packets to be transmitted in one slice of the cycle, with best-effort or other lower time priority packets being transmitted in the second half of the time cycle. In one embodiment, a domain may be partitioned by a subset of the cycle time. In one embodiment, a domain may be partitioned by a subset of the cycle time.

[0105] Those skilled in the art will appreciate that other domains or domain classifications may be used. Thus, a 5GS or 5G network may be one or more domains in a distributed TSN scheduler according to an aspect of the present invention. In some embodiments, TSN scheduling may interact with 5GS in many ways in accordance with 3GPP® specifications (e.g., TS23.501 section 5.27.2 TSC Assistance Information (TSCAI) and TSC Assistance Container (TSCAC), TS23.501 section 5.7.3.4 Packet Delay Budget) and TSN specifications (e.g., IEEE 802.1Q 8.6.8.4). The 3GPP® specifications and TSN specifications are incorporated herein by reference in their entirety. In some embodiments, the distributed TSN scheduler (e.g., CNC) provides TSN scheduling information (e.g., TSN Qbv gate control list as defined in IEEE 802.1Q 8.6.8.4) to the TSN AF 124. The TSN AF 124 may interact with the 5G scheduler (independently of the TSN scheduler) to find feasible allocations of 5G resource blocks and frequencies.

[0106] Each TSN scheduler in the set of distributed TSN schedulers may be configured to iteratively compute a schedule and exchange information with peers for each domain. This iterative process may be repeated until a globally feasible schedule is achieved that meets the requirements of the network in terms of speed, latency, QoS, or other requirements. In one or more embodiments, each domain may have a different schedule, including a different cycle time, and scheduling may operate independently within each domain. A locally feasible solution is one that meets all constraints and requirements within a domain's local region (a subset of the entire network). A globally feasible solution is one that meets all constraints and requirements of all local regions or domains in the network.

[0107] Some domains may be configured as "transit domains." TSN streams may traverse within a transit domain on their way to a TSN listener device. In some aspects of the disclosure, virtual streams may be created to indicate a particular slack scheduling capacity. Transit information for TSN streams may include message size and maximum latency. Transit information may be shared across and within domains. In some aspects, all transit information may be shared between domains.

[0108] Turning now to FIG. 7, an exemplary algorithm according to one aspect of the disclosure is shown. Within each domain, a local virtual schedule may be calculated (1101). The virtual schedule is a TSN schedule that is kept in memory and has not yet been downloaded to the network device to configure the network device. This is the TSN schedule under consideration and may change as the algorithm progresses. For non-transit streams, calculating the local virtual schedule may include scheduling a path to an inter-domain link as soon as possible (1102). For transit streams, the latency may be set to the negative of the maximum latency reported from the peer domain or domains. When a stream arrives between domains, the remaining maximum latency of the stream may be updated (1103). In some embodiments, virtual streams may be included in the schedule as needed and may be represented or indicated as being virtual by an indicator. Some uses of virtual streams may include anticipating transit streams or reducing the likelihood that a current domain will have to be rescheduled during a distributed scheduling process.

[0109] The distributed scheduling algorithm may continue with the TSN domain exchanging virtual schedules with peers (1104) and checking feasibility. If the peer schedule does not match the local schedule (1105), the peer may be asked to recalculate (1101) the local virtual schedule. The exchange of virtual schedules with peers (1104) may then be repeated. If any stream is in transit (1106), the exchange of virtual schedules (1104) may also need to be repeated. If any remaining maximum latency is negative (1107), in other words, if the exchanged virtual schedule results in packets not meeting their latency requirements, this defect in the schedule may be reported to the peer that generated the schedule, and the calculation (1101), scheduling (1102), update (1103), and exchange (1104) of the virtual schedule may be repeated. If a feasible schedule is obtained using only the peer virtual streams, there is no need to repeat the exchange of virtual schedules and the schedule is completed (1108).

[0110] Turning now to FIG. 8, a block diagram of the exchange of schedules between three domains 1200 is shown. Those skilled in the art will appreciate that three domains is an exemplary number of domains 1200 for illustrative purposes and that more domains may be used. As discussed above, if feasible, intra-domain schedules including transit streams are completed and then shared to create inter-domain schedules. Inter-domain streams may have potential excess delays 1201 that may exceed their maximum latency. In such instances, rescheduling may be required to reduce inter-domain stream delays. This may be accomplished using a binary search type algorithm to search for feasible inter-domain stream solutions. To distribute the burden and difficulty of creating feasible schedules across domains, each domain may attempt to reduce the excess half latency in some embodiments. In other embodiments, domains may adjust their plans for their excess schedule capacity.

[0111] In some embodiments, the TSN domains may form a network where the vertices of the network are domains and the edges of the network are inter-domain links. In other embodiments, the network topology may form a tree, such as a hierarchical topology, or a cycle topology may be used. In a cycle topology, a cycle of the TSN domain network may result in an infinite set of rollbacks. In a hierarchical topology, the tree structure may allow for the composition of schedules from children. Thus, a hierarchical topology with a tree structure may be preferred. In some aspects, the tree may be a spanning tree.

[0112] As part of this algorithm, a distributed scheduler message may be created. FIG. 9 includes an example distributed scheduler message 1300 according to one embodiment of the present disclosure. The distributed scheduler message fields may include an inter-domain Ethernet link 1301. The inter-domain Ethernet link 1301 may be used to identify an Ethernet link in the current domain that is connected to a neighboring domain. The message field that includes the inter-domain Ethernet link 1301 may also be used to propagate transit stream scheduling inputs to other domains. The message fields that convey the inter-domain Ethernet link information 1301 may include an Ethernet to and from link identifier, a scheduling domain identifier, and transit details. The transit details may include a message size, a route, and a maximum latency and / or a maximum remaining latency.

[0113] The distributed scheduler message 1300 may also include intra-domain scheduling information 1302. The intra-domain scheduling information 1302 may include information in associated fields that may include a domain identifier, one or more stream identifiers, data identifying the stream path through the network, total and / or maximum remaining latency per stream, and a gate control list. The maximum remaining latency may be calculated to be less than zero where the maximum latency requirement is not met. If the passage of the stream is successfully scheduled within the domain, the maximum remaining latency may be zero or greater than zero. The intra-domain schedule field 1302 may be used to define the TSN schedule for the domain. The intra-domain scheduling information 1302 may include the intra-domain schedule of the transit stream, which may cause the need for adjustments to the intra-domain schedule.

[0114] The distributed scheduler message 1300 may also include rollback request information 1303. The rollback request may be sent from a first domain to a second domain, with the first domain requesting that the second domain recalculate its schedule to accommodate conflicting transit streams. The rollback request information 1303 may include a rollback request field that may include a domain identifier, an identifier of the conflicting transit stream, and a maximum latency and / or a maximum remaining latency. The rollback request information 1303 may include one or more gate control lists for gates adjacent to one or more conflicting domains that schedule the conflicting streams.

[0115] The distributed scheduler message 1300 may also include slack scheduling capacity information 1304. The slack scheduling capacity information 1304 may be used to inform neighboring domains that the scheduling capacity has a certain slack capacity available. Slack capacity in this context may refer to a certain capacity in which flows that do not conflict with the current schedule may be scheduled. The slack scheduling capacity information 1304 may include fields including domain identifier information, virtual stream identifier information, virtual stream path information, and one or more gate control lists. The gate control list may include a list of virtual streams that are neighbors to the TSN domain.

[0116] In some embodiments of the present disclosure, a communication network configured to support time-dependent deterministic communication based on a time-dependent network (TSN) mechanism is provided. The communication network may include a plurality of network domains in the communication network connected to each other, and a plurality of schedulers in the plurality of network domains. Each of the plurality of schedulers may be configured to calculate a respective local schedule for the plurality of network domains, and a first scheduler and a second scheduler of the plurality of schedulers may be configured to repeatedly exchange the first local schedule and the second local schedule.

[0117] The first local schedule and the second local schedule may include different cycle times. The first network domain transmits inter-domain Ethernet link information related to a link connecting the first network domain and the second network domain and intra-domain schedule information related to the first local schedule. The intra-domain schedule information may include a stream identifier and a remaining maximum latency for each stream identified by the stream identifier.

[0118] The second local scheduler may be configured to determine whether the first local schedule is consistent with the second local schedule based on the received first distribution message, and in response to a determination that the first local schedule is inconsistent with the second local schedule, send a second distribution message to the first local scheduler including a request to recalculate the first local schedule. In response to at least one transit stream spanning within the second network domain, the second local scheduler may be configured to determine that the first local schedule is inconsistent with the second local schedule. The request to the first local scheduler to recalculate the first local schedule may include a conflicting transit stream identifier to identify each of the at least one transit stream spanning within the second network domain. Further, in response to a remaining maximum latency per stream having a value less than zero (0), the second local scheduler may be configured to determine that the first local schedule is inconsistent with the second local schedule.

[0119] The first distributed message may include slack scheduling capacity information associated with one or more virtual streams scheduled according to the first local schedule. At least some of the domains in the plurality of domains are separated by space, network flows, network slices, and / or time.

[0120] In some embodiments, a method is provided for generating a schedule for transmission of a stream in a communication network configured to support time-dependent deterministic communication based on a Time-Sensitive Network (TSN) mechanism. The method may include: computing, at a first network domain of a plurality of network domains in the interconnected communication network, a first local schedule for the first network domain; computing, at a second network domain of the plurality of network domains, a second local schedule for the second network domain; and iteratively exchanging the first local schedule and the second local schedule between the first network domain and the second network domain.

[0121] The method may further include receiving, in the second network domain, a first distribution message including inter-domain Ethernet link information associated with a link connecting the first network domain and the second network domain and intra-domain schedule information associated with the first local schedule, determining whether the first local schedule is consistent with the second local schedule based on the received first distribution message, and in response to determining that the first local schedule is inconsistent with the second local schedule, sending a second distribution message to the first local scheduler including a request to recalculate the first local schedule. The intra-domain schedule information may include a stream identifier and a remaining maximum latency for each stream identified by the stream identifier.

[0122] Determining whether the first local schedule is consistent with the second local schedule based on the received first distribution message may include determining whether at least one transit stream crosses into a second network domain. In response to determining that the at least one transit stream crosses into the second network domain, the request to recalculate the first local schedule may include a conflicting transit stream identifier to identify each of the at least one transit stream crossing into the second network domain. Determining whether the first local schedule is consistent with the second local schedule based on the received first distribution message may include determining whether a remaining maximum latency per stream in the received first distribution message has a value less than zero (0).

[0123] The first distributed message further includes slack scheduling capacity information associated with one or more virtual streams scheduled according to the first local schedule.

[0124] These functions described above may be implemented in computer software, firmware, or hardware. The techniques may be implemented using one or more computer program products. The programmable processor and computer may be included in or packaged as a mobile device. The processes and logic flows may be executed by one or more programmable processors and by one or more programmable logic circuits. The general-purpose and special-purpose computing devices, as well as the storage devices, may be interconnected via a communication network. The functions disclosed herein may be implemented using quantum computing, pulse-coupled oscillation (PCO) / Ising computing.

[0125] Some implementations include electronic components such as microprocessors, storage, and memory that store computer program instructions on a machine-readable or computer-readable medium (also referred to as a computer-readable storage medium, machine-readable medium, or machine-readable storage medium). Some examples of such computer-readable media include RAM, ROM, read-only compact discs (CD-ROMs), recordable compact discs (CD-Rs), rewritable compact discs (CD-RWs), read-only digital versatile discs (e.g., DVD-ROMs, dual-layer DVD-ROMs), various recordable / rewritable DVDs (e.g., DVD-RAMs, DVD-RWs, DVD+RWs, etc.), flash memory (e.g., SD cards, mini SD cards, micro SD cards, etc.), magnetic and / or solid-state hard drives, read-only and recordable Blu-Ray™ disks, extra-high density optical disks, any other optical or magnetic media, and floppy disks. The computer-readable medium may store a computer program that is executable by at least one processing unit and includes a set of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as produced by a compiler, and files containing higher level code that are executed by a computer, electronic component, or microprocessor using an interpreter.

[0126] Although the above discussion primarily refers to microprocessors or multi-core processors executing software, some implementations are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some implementations, such integrated circuits execute instructions stored on the circuit itself.

[0127] As used herein and in any claims of this application, the terms "computer," "server," "processor," and "memory" all refer to electronic devices or other technological devices. These terms exclude people or groups of people. For purposes of this specification, the terms display or display mean display on an electronic device. As used herein and in any claims of this application, the terms "computer-readable medium" or "computer-readable media" are entirely limited to tangible, physical objects that store information in a form readable by a computer. These terms exclude any wireless signals, wired download signals, and any other ephemeral signals.

[0128] To provide for interaction with a user, implementations of the subject matter described herein may be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user, and a keyboard and pointing device, e.g., a mouse or trackball, by which the user can provide input to the computer. Other types of devices may also be used to provide for interaction with a user, e.g., feedback provided to the user may be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback, and input from the user may be received in any form, including acoustic, kinetic, or tactile input. In addition, a computer may interact with a user by sending documents to and receiving documents from a device used by the user, e.g., by sending a web page to a web browser on the user's client device in response to a request received from the web browser.

[0129] Aspects of the subject matter described herein can be implemented in a computing system that includes a back-end component, e.g., as a data server, a middleware component, e.g., as an application server, or a front-end component, e.g., a client computer having a graphical user interface or a web browser through which a user can interact with an implementation of the subject matter described herein, or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communications network. Examples of communications networks include local area networks ("LANs") and wide area networks ("WANs"), internetworks (e.g., the Internet), and peer-to-peer networks (e.g., ad-hoc peer-to-peer networks).

[0130] Those skilled in the art will appreciate that the various exemplary blocks, modules, elements, components, methods, and algorithms described herein may be implemented as electronic hardware, computer software, or a combination of both. To illustrate this interchangeability of hardware and software, the various exemplary blocks, modules, elements, components, methods, and algorithms have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. The described functions may be implemented in various ways for each particular application. The various components and blocks may be arranged in different ways (e.g., arranged in a different order or partitioned in a different way), all without departing from the scope of the subject technology.

[0131] It is understood that the specific order or hierarchy of steps in the disclosed processes is an illustration of example approaches. Based upon design preferences, it is understood that the specific order or hierarchy of steps in the processes may be rearranged. Some of the steps may be performed simultaneously. The accompanying methods present elements of the various steps in a sample order, and are not intended to be limited to the specific order or hierarchy presented.

[0132] The foregoing description is provided to enable any person skilled in the art to practice the various aspects described herein. The foregoing description provides various examples of the subject technology, and the subject technology is not limited to these examples. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Thus, the claims are not intended to be limited to the aspects set forth herein, but are to be given the full scope consistent with the language of the claims, and references to elements in the singular do not mean "one and only one" unless otherwise expressly stated, but rather "one or more". The term "some" refers to one or more, unless otherwise stated. Pronouns in the masculine (e.g., his) include the feminine and neuter (e.g., her and its), and vice versa. Headings and subheadings, if present, are used for convenience only and do not limit the disclosure set forth herein.

[0133] The predicates "configured to," "operable to," and "programmed to" do not imply any particular tangible or intangible modification of the subject matter, but rather are intended to be used interchangeably. For example, a processor configured to monitor and control operations or components may also mean that the processor is programmed to monitor and control operations, or that the processor is operable to monitor and control operations. Similarly, a processor configured to execute code may be interpreted as a processor programmed to execute code or operable to execute code.

[0134] As used herein, the term automatic may include performance by a computer or machine without user intervention, for example, by instructions in response to a predicated action by a computer or machine, or other initiation mechanism. The word "example" is used herein to mean "serving as an example or illustration." Any aspect or design described herein as "example" is not necessarily to be construed as preferred or advantageous over other aspects or designs.

[0135] A phrase such as "aspect" does not imply that such aspect is essential to the subject technology or that such aspect applies to all configurations of the subject technology. Disclosure regarding an aspect may apply to all configurations, or to one or more configurations. An aspect may provide one or more examples. A phrase such as an aspect may refer to one or more aspects, or vice versa. A phrase such as "embodiment" does not imply that such embodiment is essential to the subject technology or that such embodiment applies to all configurations of the subject technology. Disclosure regarding an embodiment may apply to all embodiments, or to one or more embodiments. An embodiment may provide one or more examples. A phrase such as "embodiment" may refer to one or more embodiments, or vice versa. A phrase such as "configuration" does not imply that such configuration is essential to the subject technology or that such configuration applies to all configurations of the subject technology. Disclosure regarding a configuration may apply to all configurations, or to one or more configurations. A configuration may provide one or more examples. A phrase such as a configuration may refer to one or more configurations, or vice versa.

[0136] All structural and functional equivalents to the elements of the various embodiments described throughout this disclosure that are known or later become known to those of skill in the art are intended to be expressly incorporated herein by reference and encompassed by the claims. Furthermore, nothing disclosed herein is intended to be dedicated to the public, regardless of whether such disclosure is expressly recited in the claims. No claim element is to be construed under the provisions of 35 U.S.C. § 112(f) unless it is expressly recited using the phrase "means for" or, in the case of a method claim, the phrase "step for."

Claims

1. 1. A system configured to optimize network resource allocation in a communication network based on a time-dependent network (TSN) mechanism, comprising: a first scheduler configured to generate scheduling information for allocating network resources for data transmission between network components in the communication network, the scheduling information comprising quality of service (QoS) parameters of a user equipment (UE) and availability information of the network resources of the communication network; a second scheduler, generating a TSN schedule for transmission of TSN flows from a source device to a destination device in the communications network; receiving the scheduling information from the first scheduler; and updating the TSN schedule to be consistent with the allocation of the network resources based on the scheduling information; a second scheduler, the first scheduler further configured to update the scheduling information based on the updated TSN schedule, and adjust allocation of the network resources to support the transmission of the TSN flow based on the updated scheduling information; and a configuration module configured to manage the TSN flow from the source device to the destination device through at least one of the network components based on the updated TSN schedule and the updated scheduling information.

2. The system of claim 1 , wherein the first scheduler is configured to generate the scheduling information based on system information block (SIB) information that includes network configuration information.

3. The system of claim 1 , wherein the first scheduler is configured to generate the scheduling information based on a buffer status of the UE and radio conditions at the UE.

4. The system of claim 1 , wherein the second scheduler is configured to generate the TSN schedule based on time-dependent path information and a maximum latency for the transmission of the TSN flows.

5. The system of claim 3 , wherein the scheduling information further comprises the buffer status of the UE.

6. The system of claim 1 , wherein the network components include the UE and a radio access network (RAN).

7. 7. The system of claim 6, wherein the configuration module is configured to adjust allocation of uplink and downlink transmissions between the UE and the RAN based on the updated TSN schedule and the updated scheduling information.

8. The system of claim 7 , wherein the configuration module is configured to monitor a link quality between the UE and the RAN.

9. The system of claim 1 , wherein the scheduling information includes an available transmit power headroom for the UE.

10. 2. The system of claim 1, wherein the first scheduler is configured to schedule multiple-input multiple-output (MIMO) operation of antennas to support the transmission of the TSN flows based on the updated TSN schedule.

11. 2. The system of claim 1, wherein the second scheduler includes a communication network emulator associated with the communication network, the communication network emulator configured to receive network configuration and operational data of the communication network from the communication network and to provide forecast information for the communication network.

12. 2. The system of claim 1, wherein the second scheduler is distributed among a plurality of local schedulers in a plurality of network domains in the communication network that interconnect with each other, each of the plurality of local schedulers being configured to calculate a respective local schedule for the plurality of network domains, and a first local scheduler and a second local scheduler of the plurality of local schedulers being configured to repeatedly exchange first local schedules and second local schedules.

13. The system of claim 12 , wherein the first local schedule and the second local schedule include different cycle times.

14. 1. A method for optimizing network resource allocation in a communication network based on a time-dependent network (TSN) mechanism, the method comprising: generating scheduling information for allocating network resources for data transmission between network components in the communication network, the scheduling information including quality of service (QoS) parameters for a user equipment (UE) and availability information of the network resources of the communication network; receiving a TSN schedule for transmission of TSN flows from a source device to a destination device in the communications network, the TSN schedule being updated based on the scheduling information to be consistent with available network resources; updating the scheduling information based on the updated TSN schedule to adjust allocation of the network resources to support the transmission of the TSN based on the updated scheduling information; and managing the data transmission of the TSN flow from the source device to the destination device via the network component based on the updated TSN schedule and the updated scheduling information.

15. The method of claim 14, wherein generating the scheduling information is based on a buffer status of the UE and radio conditions at the UE.

16. The method of claim 14 , wherein the scheduling information includes a buffer status of the UE.

17. 15. The method of claim 14, wherein the network components include the UE and a Radio Access Network (RAN), and managing the data transmission of the TSN flow from the source device to the destination device via the network components further includes adjusting allocation of uplink and downlink transmissions between the UE and the RAN based on the updated TSN schedule and the updated scheduling information.

18. The method of claim 14 , wherein the scheduling information includes an available transmit power headroom for the UE.

19. The TSN schedule is: calculating, in a first network domain of a plurality of network domains in the interconnected communications network, a first local schedule for the first network domain; calculating, at a second network domain of the plurality of network domains, a second local schedule for the second network domain; and repeatedly exchanging the first local schedule and the second local schedule between the first network domain and the second network domain.

20. 20. The method of claim 19, wherein the first local schedule and the second local schedule include different cycle times.

Images