Multi-access Edge Computing (MEC) Control and Resource Characterization

Abstract

The communication system includes a core network and a radio access network including a central unit communicatively connected to (i) the core network via a backhaul network and (ii) a plurality of distributed units via a fronthaul network, each of the distributed units being co-located with a radio unit and including a multi-access edge computing (MEC) module, each of the MEC modules including one or more processors and a memory storing one or more programs executed by the one or more processors, the one or more programs including instructions for controlling a time sensitive networking (TSN) configuration and for executing one or more MEC applications using the TSN configuration.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 63 / 316,808, filed March 4, 2022, the entire contents of which are incorporated herein by reference.

[0002] This specification relates generally to the integration of wireless communication systems with time-sensitive networks (TSN), and more specifically, to configuring wireless communication systems as multiple components of a TSN, for example. [Background technology]

[0003] Some applications, such as industrial automation and manufacturing, require ubiquitous and seamless connectivity with strict, deterministic timing requirements for communication between the various devices or components of the application (industrial controllers, sensors, actuators, etc.). To meet such requirements, TSN systems, a technical term for deterministic networks that provide deterministic communication with relatively strict Quality of Service (QoS) parameters such as latency jitter and reliability requirements for data traffic, can be integrated with 5G wireless communication systems that provide high reliability services such as Ultra-Reliable Low Latency Communication (URLLC) services.

[0004] Particular features of the subject technology are set forth in the appended claims, but for purposes of illustration, certain aspects of the subject technology are illustrated in the following drawings. [Brief description of the drawings]

[0005] [Figure 1] FIG. 1 illustrates an example of a conventional integrated TSN-5G system 100 according to one or more implementations. [Diagram 2] FIG. 2 is a block diagram of an example integrated TSN-5G system 200 architecture according to one or more implementations. [Diagram 3]A diagram showing corresponding elements of an integrated TSN-5G system 100 and an integrated TSN-5G system 200 in accordance with one or more embodiments. [Figure 4] FIG. 1 is a block diagram of an example TSN block according to one or more implementations. [Diagram 5] FIG. 1 illustrates an example set of parameters for an example TSN block according to one or more implementations. [Figure 6A] FIG. 1 is a block diagram of an example integrated TSN-5G system architecture including a MEC, a type of edge-enabled device, in accordance with one or more implementations. [Figure 6B] FIG. 1 is a block diagram of an example integrated TSN-5G system architecture including a MEC, a type of edge-enabled device, in accordance with one or more implementations. [Figure 7] FIG. 1 illustrates an electronic system in which one or more implementations of the subject technology may be implemented. [Figure 8] FIG. 8 is a block diagram of an example integrated TSN-5G system 800 architecture according to one or more implementations. [Figure 9] FIG. 1 illustrates a 5G system including a TSN block representing or corresponding to an MEC module. [Figure 10] A diagram showing an improved scheme for implementing MEC in a 5G network in some implementations. [Figure 11] A diagram showing a communication system implementing an improved scheme for implementing MEC in a 5G network in some implementations. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0006] The detailed description set forth below is intended to describe various configurations of the subject technology and is not intended to represent the only configurations in which the subject technology may be practiced. The accompanying drawings are incorporated herein and form a part of the detailed description. The detailed description includes specific details intended to provide a thorough understanding of the subject technology. However, the subject technology is not limited to the specific details set forth herein and may be practiced using one or more other implementation forms. In one or more implementation forms, structures and components are shown in block diagram form in order to avoid obscuring the concepts of the subject technology.

[0007] As mentioned above, in some applications such as industrial automation, manufacturing, aerospace, and in-vehicle communication in automobiles, TSN systems providing deterministic communication may be integrated with fifth-generation (5G) wireless communication systems providing flexible and ultra-reliable low-latency communication (URLLC) services. However, typically, in such integrated TSN-5G systems, the entire 5G system is configured to operate as a single TSN component (e.g., TSN bridge), and the integrated system is configured as a fully centralized configuration model, such as using one centralized TSN configuration controller. Therefore, such integrated systems may not support the deployment of 5G systems that include various components provided by different vendors or the flexibility for distributed TSN configuration of the integrated system. In addition, 5G systems may support reliable communication by using redundant paths for data transmission. However, since a typical TSN-5G integrated system configures the 5G system as one TSN component, redundant paths are configured across all components of the entire 5G system (e.g., from the user equipment (UE) to the user plane function (UPF)). This does not allow the flexibility to set up redundant data transmission paths only for parts of the 5G system, such as the 5G system parts (such as the air interface between the UE and the Radio Access Network (RAN) / gNodeB) that are sensitive to data delays and / or errors.

[0008] To address the above problems in an integrated TSN-5G system, the subject technology provides a novel architecture in which a 5G system is configured as a set of individual 5G components, with each 5G component configured as one individual TSN block, rather than configured as a single TSN component or block. In other words, the present disclosure provides an architecture in which the 5G system of an integrated TSN-5G system is divided into multiple TSN blocks, with each TSN block configured according to a TSN specification (e.g., compliant with IEEE 802.1Q and related standards), e.g., as a TSN bridge, a TSN end device, or a combination of the two. Furthermore, instead of having a centralized configuration controller to control the TSN-5G system, the subject technology provides multiple distributed configuration modules in the TSN-5G system. The configuration modules may be interconnected in one or more topologies (e.g., mesh, star, tree, random, etc.), and each configuration module may be responsible for communication with and configuration of one or more TSN blocks.

[0009] As described in more detail below, each TSN block of the multiple TSN blocks of the 5G system includes a set of parameters that describe the ability to support and execute data flows (e.g., carrying URLLC data traffic) through that TSN block. Additionally, each TSN block may include at least one configuration interface configured to support interaction between the TSN block and a respective configuration module. The configuration interface may be used to provide the respective configuration module with a set of parameters for the TSN block and to receive configuration data (e.g., transmission schedules, data flow IDs, policing rules, etc.) from the configuration module to support one or more data flows through the TSN block. Each TSN block may be configured to execute or operate on the configuration data (received via the configuration interface) and transmit data for each data flow according to the specifications provided in the configuration data. Finally, each TSN block may also include a monitoring and diagnostics module configured to monitor the runtime behavior of the TSN block and report the behavior through the configuration interface or via another interface of the TSN block.

[0010] Each configuration module of the multiple distributed configuration modules may be an external utility responsible for configuring one or more TSN blocks, or may be configured as a software module within the TSN block that configures only that TSN block. Each configuration module may be configured to determine and provide configuration data including a TSN schedule for one or more data flows to be transmitted through the TSN block to the TSN block controlled by the configuration module. The configuration modules may exchange information with each other using a standardized application programming interface (API). The exchanged information may include information regarding the cycle time of the TSN system (e.g., supported administrative cycle times (ACTs) including individual levels of ACT buckets, maximum / minimum cycle times, each corresponding to a specific data flow), configuration data including a transmission schedule for one or more TSN blocks (including time offsets / durations / resources of transmission), and information for requests for resource allocation or responses to requests. In some embodiments, one or more of the multiple configuration modules may be adapted as a "controller" or "controller module", which may include a component configured or adapted to provide instructions, control, operations, or any form of communication for an operable component to affect its operation. The controller module may include any known processor, microcontroller, or logic device, including, but not limited to, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a full authority digital engine control (FADEC), an aircraft system, a proportional controller (P), a proportional integral controller (PI), a proportional derivative controller (PD), a proportional integral derivative controller (PID controller), hardware accelerated logic controllers (encoding, decoding, transcoding, etc.), or the like, or combinations thereof.

[0011] 3GPP Release 8 classifies self-organizing networks (SONs) into three main categories: self-configuring, self-optimizing, and self-healing. Self-organization is considered as a mechanism or process that allows a system or network to change its organization at run time without explicit commands. Self-configuration is defined as the process of incorporating new network elements (NEs) into a service that requires minimal human operator intervention, where a network element is a manageable logical entity that combines one or more physical devices. In some implementations, TSN blocks (described below) can self-configure and be incorporated into a 5G system as NEs with minimal human intervention. This can be achieved by TSN applications automatically sharing TSN flow (stream) characteristics and latency requirements with TSN blocks. TSN blocks can collaborate to share flow characteristics and determine feasible TSN schedules.

[0012] FIG. 1 illustrates a non-limiting example of a conventional integrated TSN-5G system 100 configured such that a 5G system 106 is emulated as a single TSN component (e.g., a TSN bridge). Overall, the system 100 is configured as a deterministic TSN system for communicating data between end devices, e.g., input / output (I / O) devices 102 and controllers 104, using a TSN controller 110 via the 5G system 106 (emulated as a TSN bridge) and one or more (conventional) TSN bridges 108. The system 100 is configured based on standard methods for time synchronization and traffic management, enabling deterministic communication over a standard Ethernet network between end devices, e.g., I / O devices 102 and controllers 104. For example, the system 100 may operate according to the IEEE 802.1Q TSN specification suite, which standardizes Layer 2 communication of networking protocols that provide deterministic communication while sharing the same infrastructure. For example, numerous standards establish various technical paradigms for TSN systems, namely clock synchronization (802.1AS, generalized precision time protocol (gPTP)), frame preemption (802.3br and 802.IQbu), scheduled traffic (802.1Qbv), and redundancy management (Frame Replication and Elimination for Reliability (FRER) IEEE 802.1CB). These standards must work together at Ethernet Layer-2 to ensure that critical control and safety functions are performed while meeting their respective deadlines and constraints. As an alternative implementation, a similar integrated system may be configured to implement TSN technologies over wireless local area networks (WLANs), such as Wi-Fi networks based on Wi-Fi 6 and other popular WLANs.

[0013] For example, the 802.1Qbv TSN standard provides for scheduled transmission of safety-critical data frames in a prescribed manner and is incorporated herein in its entirety. As used herein, a "TSN schema" may refer to, but is not limited to, a network, component, element, unit, node, hub, switch, control, module, path, data, data frames, traffic, protocol, operation, transmission, and combinations thereof that conform to, are configured to, or correspond to one or more of the IEEE 802.1 TSN standards. The 802.1Qbv TSN standard addresses the transmission of critical and non-critical data traffic within a TSN. Critical data traffic is guaranteed to be delivered at a scheduled time, while non-critical data traffic is typically given a lower priority. Various traffic classes are established by IEEE 802.1Q that are used to prioritize different types of data traffic.

[0014] Preemption of Ethernet frames is defined by the IEEE 802.3br and IEEE 802.1Qbu standards and allows pausing the transmission of non-critical Ethernet frames, which also helps to reduce latency and latency variation of critical traffic. The basis of resource management is defined by the TSN configuration model (IEEE 802.1Qcc). A centralized network configuration (CNC) 112 can be applied to network devices (bridges, e.g., 5G system bridges 106, bridges 108), while a centralized user configuration (CUC) 114 can be applied to user devices (end stations, e.g., I / O devices 102). The fully centralized configuration model follows the Software Defined Networking (SDN) approach. In other words, the CNC 112 and CUC 114 in the controller 110 provide a control plane instead of a distributed protocol. In contrast, a distributed control protocol is applied to a fully distributed model where there is no CNC or CUC.

[0015] High availability as a result of ultra-reliability can be provided by Frame Replication and Elimination (FRER) (IEEE 802.1CB) for data flow reliability through per-packet level reliability mechanisms, which provide reliability by transmitting multiple copies of the same data packets over separate paths in the network. Per-stream filtering and policing (802.1Qci) improves reliability by protecting against bandwidth violations, malfunctions, and malicious behavior. Furthermore, time synchronization in TSN systems can be defined by the generalized Precision Time Protocol (gPTP) (802.1AS), which is a profile of the Precision Time Protocol standard (IEEE 1588). gPTP provides reliable time synchronization and can be used with other TSN tools such as scheduled traffic (802.1Qbv).

[0016] To achieve a desired level of reliability, TSN employs time synchronization and time-aware data traffic shaping. In data traffic shaping, schedules are used to control the gating of transmissions on network switches and bridges (e.g., nodes). In some aspects, such data traffic schedules in TSN can be determined before the operation of the network. In other aspects, data traffic schedules can be determined during an initial design phase based on system requirements and updated as needed. For example, in addition to defining the TSN topology (including communication paths, bandwidth reservations, and various other parameters), network-wide synchronization times for data transfer can also be defined in advance. Such planning of data transmissions on the communication paths of a network is typically referred to as a "communication schedule" or simply a "schedule." Data traffic schedules on TSN can be determined for specific data packets, over specific paths, at specific times, and for specific durations. Non-limiting examples of techniques for generating schedules for TSN data traffic are described in U.S. Patent Application No. 17 / 100,356, which is incorporated herein by reference in its entirety.

[0017] Time-critical communications between TSN end devices or nodes (e.g., I / O device 102 and controller 104) include "TSN flows," also referred to as "data flows" or simply "flows." For example, a data flow may comprise datagrams, such as data packets or data frames. Each data flow is unidirectional, travels from a first originating or source end device (e.g., I / O device 102) to a second destination end device (e.g., controller 104) in the system, and has a unique identity and time requirement. These source and destination devices are commonly referred to as "talkers" and "listeners." Specifically, the "talkers" and "listeners" are the source and destination, respectively, of a data flow, and each data flow is uniquely identified by an end device operating in the system. It will be appreciated that for a given network topology comprising multiple interconnected devices, a set of data flows between the interconnected devices or nodes may be defined. For example, a set of data flows may exist between the interconnected devices. Various subsets or permutations of data flows may be additionally defined for the set of data flows. Furthermore, time-critical communications between TSN end devices or nodes include "TSN streams" or "streams," and each TSN stream may originate from a particular talker node intended to communicate to one or more listener nodes. Thus, each TSN stream may include one or more data flows, and each data flow exists between a talker node (the source of the TSN stream) and a listener node.

[0018] Both end devices (e.g., 102, 104) and switches (commonly referred to as "bridges" or "switching nodes") (e.g., 106, 108) transmit and receive data (in one non-limiting example, Ethernet frames) in data flows based on a predefined time schedule. The switching nodes and end devices need to be time synchronized to ensure that the predefined time schedule of the data flows is followed correctly throughout the network. For example, in FIG. 1, clock 116 represents that the various switching nodes and end devices of the TSN system 100 (including the 5G system 106) are time synchronized with reference to a global clock (grand master clock timing). In some other aspects, only the switches can transmit data based on a predefined schedule, and the end devices, e.g., legacy devices, can transmit data in an unscheduled manner.

[0019] Data flows in TSN can be scheduled using a single device (e.g., controller 110) that assumes a fixed, unchanging path through the network between talker / listener devices and switching nodes of the network. Alternatively, a set of devices or modules can be used to schedule data flows. The scheduling device, whether a single device or a set of devices, can be arranged to define a centralized scheduler. In yet other aspects, the scheduler device can include a distributed arrangement. TSN can also receive non-deterministic communications, such as rate-constrained communications. In one non-limiting example, the scheduling device may include an offline scheduling system or module.

[0020] TSN traffic may be tagged using a variety of mechanisms, including VLAN tags, Ethernet addresses, IP header information, and combinations of VLAN tags Ethernet addresses and IP header information. Traffic may be identified and tagged anywhere in the system before protocol data units (PDUs) need to be identified. TSN talkers may create multiple TSN flows (streams) with different TSN latency and determinism requirements, and different paths that meet the requirements may be assigned. In some implementations of the subject invention, latency and determinism values ​​may be specified and provided to TSN applications as a limited set of static, discrete values, rather than being provided to accommodate an unlimited set of continuous values.

[0021] In some implementations, the I / O end devices 102 may be complex mechanical entities such as, in various aspects, a factory production line, a gas-fired power plant, an aircraft avionics data bus, a jet engine of an aircraft in a fleet (e.g., two or more aircraft), an aircraft digital backbone, an avionics system, a mission or flight network, a wind farm, a locomotive, etc. In various implementations, the I / O end devices 102 may include any number of end devices such as sensors, actuators, motors, software applications, etc. The sensors may include any conventional sensor or transducer, such as a camera that generates video or image data, an x-ray detector, an acoustic pickup device, a tachometer, a global positioning system receiver, a wireless device that transmits a radio signal and detects reflections of the radio signal to generate image data, or other devices.

[0022] Additionally, actuators (e.g., devices, equipment, or machines that move to perform one or more operations of an I / O device 102) can communicate using the TSN system 100. Non-limiting examples of actuators may include brakes, throttles, robotic devices, medical imaging devices, lights, turbines, etc. An actuator can communicate its status data to one or more other devices (e.g., other I / O devices 102, controller 104 via the TSN system 100). The status data may represent the location, state, health, etc. of the actuator sending the status data. An actuator can receive command data from one or more other devices (e.g., other I / O devices 102, controller 104) of the TSN system 100. The command data may represent instructions that dictate how or when the actuator should move, operate, etc.

[0023] In some implementations, the controller 104 can communicate various data between the I / O end devices 102 via the TSN 100. For example, the control system 104 can communicate command data to one or more devices 102 or receive data, such as status data or sensor data, from one or more devices 102. Thus, the controller 104 can be configured to control the operation of the I / O devices 102 based on data acquired or generated by or communicated between the I / O devices 102, for example, to enable automated control of the I / O devices 102, and to provide information to operators or users of the I / O devices 102. The controller 104 can define or determine data flows and data flow characteristics of the TSN system 100.

[0024] Referring now to the 5G system 106 in the system 100, the 5G system 106 is a wireless communication system used to carry TSN traffic between various TSN end devices, e.g., the I / O device 102 and the controller 104. In some implementations, the 5G system 106 is configured to emulate one TSN bridge per user plane function (UPF) (similar to the TSN bridge 108, per the TSN standards discussed above). The 5G system 106 may be a New Radio (NR) network implemented per the 3GPP 23 and 38 series specifications, which are incorporated herein in their entirety, and integrated into the system 100 per the 3GPP Release 17 23.501 standards v17.1.1 and v17.2.0, which are incorporated herein in their entirety. As shown, the 5G system 106 may include a user equipment (UE) 118, a RAN (gNB) 120, a user plane function (UPF) 122 in the 5G user plane, and an application function (AF) 124 and a policy control function (PCF) 126 in the 5G control plane, among other components. In some implementations, the 5G system 106 may be configured to provide ultra-reliable low latency communication (URLLC) services. The 5G system 106 based on the New Radio (NR) interface includes several features to achieve low latency for selected data flows. NR allows for shorter slots in the radio subframe, which benefits low latency applications. NR also introduces minislots, which further reduce latency by allowing prioritized transmissions to begin without waiting for a slot boundary. As part of giving priority and high-speed radio access to URLLC traffic, NR introduces preemption, which allows URLLC data transmissions to preempt ongoing non-URLLC transmissions. Additionally, NR applies very fast processing, allowing retransmissions even within low latency ranges.

[0025] In some implementations, 5G defines highly robust transmission modes to increase the reliability of both data and control radio channels, which is further improved by various techniques such as multi-antenna transmission based on multiple-input and multiple-output (MIMO) technology, the use of multiple carriers, and packet duplication over independent radio links.

[0026] Time synchronization is built in as an integral part of the operation of 5G cellular radio systems, which is already common practice in earlier cellular network generations. Radio network components themselves are also time synchronized, for example through the Precision Time Protocol Telecom Profile. This provides an excellent basis for providing synchronization for time-critical applications. For URLLC services, the 5G system 106 uses time synchronization for its own operation as well as multiple antennas and radio channels that provide reliability. In addition to 5G RAN capabilities, the 5G system 106 may also provide a Core Network (CN) solution for Ethernet networking and URLLC. The 5G CN supports native Ethernet Protocol Data Unit (PDU) sessions. 5G supports the establishment of redundant user plane paths through the 5GS, including the RAN, CN, and transport networks. 5GS also enables redundant user planes between RAN and CN nodes, as well as between UE and RAN nodes separately.

[0027] As described above, in the integrated system 100, the 5G system 106 includes one TSN (virtual) bridge for each UPF. The 5G system 106 includes a TSN translator (TT) function for adapting the 5G system 106 to the TSN domain in both the user plane and the control plane, and the internal procedures of the 5G system 106 are hidden from the TSN bridge network. The 5G system 106 provides the operation of the input and output ports of the TSN bridge through the TT function. For example, the TT supports hold and forward functions for digital ring. Although FIG. 1 shows when the 5G system 106 connects the end station 102 to the bridge network 108, the 5G system 106 may also interconnect the bridges 108.

[0028] For the 5G system 106 integrated into the TSN system 100, the requirements of the TSN stream can only be met when the resource management allocates network resources to each hop along the entire path. In line with the TSN configuration (802.1Qcc), this is realized through the interaction between the 5G system 106 and the CNC 112. The interface between the 5G system 106 and the CNC allows the CNC 112 to learn the characteristics of the 5G virtual bridge and allows the 5G system 106 to establish a connection with specific parameters based on the information received from the CNC 112. The bounded latency requires deterministic latency from 5G as well as QoS alignment between the TSN and 5G domains. For example, when the 5G virtual bridge acts as a TSN bridge, the 5G system 106 emulates time-controlled packet transmission along scheduled traffic (802.1Qbv). For the 5G control plane, the TT of the AF 124 receives transmission time information of the TSN traffic class from the CNC 112. In the 5G user plane, the TT at the UE 118 and the TT at the UPF 122 may adjust time-based packet transmissions accordingly. Different TSN traffic classes may be mapped to different 5G QoS Indicators (5QIs) in the AF 124 and PCF 126 as part of the QoS alignment between the TSN and 5G domains, and the different 5QIs are processed according to their QoS requirements.

[0029] With regard to time synchronization, the 5G system 106 may implement gPTP in the connected TSN network. The 5G system 106 may act as a virtual gPTP time-aware system and support the transfer of gPTP time synchronization information between the end stations 102 and the bridges 108 over the 5G user plane TT.

[0030] Referring now to FIG. 2, a block diagram of a system 200 architecture according to some embodiments of the subject technology is shown. Broadly speaking, the system 200 is implemented as an integrated TSN-5G system similar to the system 100 described above, and may also include similar physical components as the system 100 described above. However, unlike the system 100, the system 200 provides a novel architecture of an integrated TSN-5G system, in which the 5G system 106 is configured as a set of individual 5G components, each 5G component configured to emulate one individual TSN block 202. In other words, in the system 200, the 5G system 106 is configured as a distributed structure including multiple TSN blocks 202-1 to 202-N, and each TSN block 202 is configured as, for example, a TSN bridge, a TSN end device, or a combination of the two, according to a TSN specification (e.g., compliant with the above-mentioned IEEE 802.1 and related standards). Further, instead of having a centralized configuration controller 110 for controlling the TSN-5G system, the subject technology provides a plurality of distributed configuration modules 215 in the distributed controller 210 of the TSN-5G system 200. The configuration modules 215-a through 215-f may be interconnected in one or more topologies (mesh, star, tree), with each configuration module 215 responsible for communicating with and configuring one or more TSN blocks 202. As used herein, a "topology" may refer to one or more arrangements of a network that may include multiple nodes of the network (e.g., sending devices, receiving devices, switches, or bridges) and connecting lines (e.g., communication links or "hops," including wired and wireless communication links) between the nodes. Each link may communicatively couple a corresponding pair of nodes. A set of links may be coupled in turn through respective nodes to define, for example, a link path between a source node and a destination node. The topology may comprise, but is not limited to, one or more of mesh, star, bus, ring, and tree topologies.

[0031] In some implementations, the system 200 is configured to support and manage deterministic TSN data flows between data sources 204 and data destinations 206 through the 5G system 106 via a TSN configuration including a TSN schedule determined by one or more configuration modules 215. The data sources 204 and data destinations 206 may include one or more of the I / O devices 102 and the controllers 104. Although not shown, the system 200 may also include a TSN bridge 108 and other TSN components.

[0032] In some implementations, in a distributed structure, each of the multiple TSN blocks 202-1 to 202-N may correspond to one particular component of a 5G system, such as the 5G system 106 shown in FIG. 1 and described above. For example, as shown in FIG. 3, the UE 118 may be configured to emulate as the TSN block 202-1, the RAN 120 may be configured to emulate as the TSN block 202-2, the UPF 122 may be configured to emulate as the TSN block 202-3, and other typical components of the core network and / or 5G system (e.g., fronthaul, backhaul, multi-access edge computing (MEC) modules) may be configured to emulate as one or more TSN blocks 202-N. Each TSN block 202-1 to 202-N may be configured in accordance with a TSN specification (e.g., compliant with IEEE 802.1 and related standards described above), such as a TSN bridge, a TSN end device, or a combination of the two.

[0033] 4, each TSN block 202 includes a processor 402, a memory device 404, an internal configuration interface (ICI) 406, a transport module 408, a reporting module 410, and TSN translators (TT)-1, 412-1, TT-2 412-2, and TT-3 412-3. The processor 402 may be a microprocessor or multi-core processor, integrated circuit, field programmable gate array, etc. that processes TSN configuration data and executes instructions (e.g., stored in the memory device 404) for processing and transmitting TSN data traffic from one or more TSN data flows according to the TSN configuration data.

[0034] The memory device 404 may store a set of parameters describing the capability to support and perform data flows (e.g., carrying URLLC data traffic) through the corresponding TSN block 202. In some implementations, the set of parameters may include, but are not limited to, ID, link quality, link bandwidth, etc. The ID parameter(s) may include device type (i.e., whether the TSN block 202 is a TSN bridge or a TSN end station). The latency parameter(s) may include at least port-to-port (from the start of the TSN block to the end of the TSN block) latency and latency variation (commonly known as "jitter"). The link quality parameter(s) may include at least packet error rate. The link bandwidth parameter(s) may include at least available bandwidth in bits / second.

[0035] In some implementations, the set of parameters of the TSN block 202 may include a subset of parameters specific to 5G RAN, including short transmission time intervals, TSC assistance information (TSCAI), configured grant (CG) information, semi-persistent scheduling (SPS) allocation, and / or other parameters specified, for example, in 3GPP TS 28.540. Additionally, in some implementations, the set of parameters of the TSN block 202 may include a subset of parameters specific to TSN, including time synchronization properties, scheduled transmission (Qbv) attributes, redundancy attributes including several connected RANs, number of paths to UPF, path diversity, number of available frequencies, propagation characteristics, available radios, and various physical media (e.g., free space optics). As an example, FIG. 5 illustrates an example set 502 of parameters of an example TSN block 202.

[0036] In some implementations, the set of parameters for the TSN block 202 may define a worst case time synchronization error, a worst case gate operation error, a maximum gate control list size, a maximum cycle time, a maximum gate interval period, a transmission start delay, etc., or a combination thereof. The set of parameters may include a separate set of deterministic parameters that may vary depending on the type of traffic handled by the TSN block 202. The set of parameters may be leveraged, at least in part, to generate a TSN schedule, configuration, etc., that is realizable on the hardware of the TSN block 202. In the absence of such a set of parameters, the TSN scheduling module would have to adopt a least common denominator approach, where all devices in the system 200, including the TSN block 202, are assumed to have the most restrictive characteristics, resulting in a suboptimal solution. In this sense, the set of parameters allows for better scheduling solutions in the TSN system 200, improving performance metrics such as latency, jitter, packet delay variation, and bandwidth utilization.

[0037] In some implementations, the sets of parameters of the TSN block 202 may further describe or relate to devices created or programmed by different operating or manufacturing origins, or otherwise. In this sense, the sets of parameters may define different sets or subsets of devices (e.g., heterogeneous ones from multiple vendors) as opposed to homogenous or all similar devices. For example, the sets of parameters may include, but are not limited to, definitions of specific configuration models, error tolerances, hardware limitations, software limitations, and firmware options for each end node and switching node of the system 200. In this sense, the parameter sets enable the scheduling and configuration of heterogeneous networks with devices of various characteristics from multiple vendors.

[0038] In some implementations, the set of parameters for the TSN block 202 may facilitate the specific TSN features supported by each end node and switching node of the TSN system 200. For example, the set of parameters may define whether a node or the TSN block 202 supports one or more of time synchronization, time aware shaping, asynchronous shaping, frame duplication and deletion for reliability, frame preemption, ingress policing, and other TSN features. The set of parameters may further define the specific versions or variants of features or standards supported by the end nodes and switching nodes of the TSN system 200. These sets of parameters enable the scheduling and configuration of mixed function networks in which the end nodes and switching nodes have various degrees of support (including no support) for required TSN features and versions.

[0039] Further non-limiting examples of the set of parameters of the TSN block 202 device may include, but are not limited to, additional parameters utilized for the programming function of each TSN block 202. For example, the additional set of parameters may define or enable the programming of each TSN block 202 with a generated or scheduled TSN configuration. Non-limiting examples of the set of parameters that may define or enable the programming of each TSN block 202 may include, but are not limited to, a programming method, a communication protocol, a device login name, a device login password, a device programming port, a device programming file structure or file path for programming data, a device programming file format, a device configuration file format, a device schedule file format, and the like, or combinations thereof. In this sense, this set or subset of parameters defines or enables the programming of each TSN block 202 with a generated or scheduled TSN configuration (collectively, "programming parameters") to enable or enable the system 200 to update, install, program, configure, or otherwise modify the set of TSN blocks 202 to operate in accordance with or by a schedule or configuration of the TSN system 200.

[0040] Additionally, each TSN block 202 may include at least one internal configuration interface (ICI) 406 configured to support interaction between the TSN block 202 and, for example, a respective configuration module 215. The ICI 406 may be used to provide some or all of a set of parameters for the TSN block 202 to the respective configuration module 215 and to receive configuration data (e.g., transmission schedules, data flow IDs, policing rules) from the configuration module 215 to support one or more data flows through the TSN block 202. Each TSN block 202 may be configured to execute or operate on the configuration data (received via the ICI 406) and transmit data for each data flow according to specifications provided in the configuration data.

[0041] In some implementations, the configuration data received from each configuration module 215 via the ICI 406 of the TSN block 202 may include stream identification information, transmission schedules or deadlines or delay budgets, or (in the case of rate-constrained traffic) data rates, filtering and policing configuration information, redundancy schemes, and / or other TSN configuration information.

[0042] TSN talker information may be split into different frequency components that require different TSN flow latency and determinism requirements. Conversely, TSN flows from different TSN talkers may be aggregated into a single TSN flow to achieve greater capacity and higher channel utilization. Having separate cycle times in an integrated TSN-5G system may help ensure easy aggregation of TSN flows.

[0043] In some implementations, each TSN block 202 may include a transmission module 408 configured to transmit transmissions of a particular data flow in compliance with a schedule, deadline, delay budget, or data rate received in configuration data from the configuration module 215 via the ICI 406. In one example where the TSN block 202-1 corresponds to a UE 118 of a 5G system, the transmission module 408 is configured to transmit data transmissions based on resource scheduling of the 5G air interface (between the UE 118 and the RAN 120). The resource scheduling of the 5G air interface may be phase aligned to the cycle time of the integrated TSN-5G system. The transmission module 408 may allocate resource elements of the 5G component corresponding to the TSN block 202 (of which 408 is a part) in such a way that the scheduled transmissions of each Qbv stream can be met. For example, instead of using traditional 802.1Qbv-style gating, the UE 118 corresponding to the TSN block 202-1 may use a phase offset (relative to the cycle time) to align the transmission of the TSN data stream to the assigned schedule. In this example, instead of the UE 118 obtaining this phase offset from the CNC, the UE 118 may obtain the phase offset as a special command from the RAN 120.

[0044] In some implementations, each TSN block 202 may also include a reporting module 410 configured to monitor the runtime behavior of the TSN block 202 and report the behavior through the ICI 406 or via another interface of the TSN block 202. This behavior monitoring includes, but is not limited to, metrics such as packet drops and missed transmission windows.

[0045] Returning to FIG. 2, the subject technology provides a number of distributed configuration modules 215 in the distributed controller 210 of the TSN-5G system 200. The configuration modules 215-a through 215-f may be interconnected in one or more topologies (mesh, star, tree), and each configuration module (CM) 215 may be responsible for communication and configuration with one or more TSN blocks 202. For example, CM 215-a may be responsible for and operatively and communicatively connected to TSN blocks 202-1 and 202-2. CM 215-b may be responsible for and operatively and communicatively connected to TSN block 202-3. CM 215-c may be responsible for and operatively and communicatively connected to TSN blocks 202-3 and 202-N. Thus, TSN block 202-3 may be configured and controlled by both CM 215-b and CM 215-c. For example, some of the functionality of TSN block 202-3 (e.g., related to a first type of TSN application) may be configured and controlled by CM 215-b, and another part of the functionality of TSN block 202-3 (e.g., related to a second type of TSN application) may be configured and controlled by CM 215-c.

[0046] In the example shown in FIG. 2, the configuration modules 215 are arranged in a tree structure, with CM 215-f forming the top level of the tree structure, CM 215-a, 215-b, and 215-c forming the bottom level of the tree structure, and CM 215-d and 215-e being between the top and bottom levels of the tree structure. However, the configuration modules 215 are operatively and communicatively connected to each other via API 230. In some implementations, in a tree structure (e.g., as shown in FIG. 2), a configuration module 215 may communicate with other configuration modules 215 at adjacent tree levels (one tree level above or one tree level below). However, in other topologies (e.g., mesh or peer-to-peer structures), any two configuration modules 215 of the distributed controller 210 may be directly connected to and communicate with each other.

[0047] In some implementations, each configuration module 215 may be an external utility responsible for configuring one or more corresponding TSN blocks 202 or may be configured as a software module within the TSN block 202. Each configuration module 215 may be configured to determine and provide configuration data including a TSN schedule for one or more data flows transmitted through the TSN block 202 to the TSN block 202 controlled by the configuration module 215. The configuration modules 215 may exchange information with each other using a standardized API 230. The exchanged information may include information regarding cycle times of the TSN system (e.g., supported management cycle times (ACTs) including individual levels of ACT buckets, maximum / minimum cycle times, each corresponding to a specific data flow), configuration data including transmission schedules (including time offsets / durations / resources of transmission) of one or more TSN blocks 202, and information for requests for resource allocation or responses to requests. In some implementations, one or more of the configuration modules 215 may be configured as the CNC 112 and / or CUC 114 of the TSN controller 110 described above.

[0048] In some implementations, each configuration module (CM) 215 receives some or all of the set of parameters (described above) for the TSN block 202 from the ICI 406 of the TSN block 202 controlled by the CM 215. The CM 215 also receives information about the data flows to be configured through the TSN block 202 from another entity of the system 200 through the API 230. In a non-limiting example, the data source 204 and / or the data destination 206 provide the requirements for the data flows between the data source 204 and the data destination 206 to the CM 215, either directly or through an intermediary such as the CUC 114. In some implementations, the CM itself may allow a user to define a set of data flows to be configured through a user interface. As used herein, information about data flows may include or be defined as a set of data flows, data streams, transmission paths (predetermined or otherwise adapted), etc., to define a desired TSN communication path between the data source 202 and the data destination 206. A non-limiting example set of information for a data flow may include maximum tolerable latency, data rate, data frame size ("payload"), destination of the data frame, bandwidth allocation gap, and the like, or combinations thereof.

[0049] Based on at least the received data flow information and a set of parameters for the TSN block 202, the CM 215 determines a "solution" (or configuration data) that indicates how to handle each data flow passing through the TSN block 202. The solution may include, among other things, time-aware schedules, policing rules, etc., as described in U.S. Patent Application No. 17 / 100,356, which is incorporated by reference in its entirety. The CM 215 may then transmit the solution or configuration data to the TSN block 202 via the ICI 406. The TSN block 202 may then execute the solution and transmit the data for each flow according to its configuration. In some implementations, as an example of a process for having the distributed CM 215 compute a solution, the same system modulo theory (SMT) solver may be used at each level of the CM 215 tree structure, where the data flows and their requirements are expressed as constraints, and a feasible solution is found using linear programming. Solutions from lower levels of the CM tree are entered as resources (also represented by constraints) to be used at the next higher level of the CM tree, and this process is repeated until the top level of the CM tree is reached and a global solution is determined.

[0050] In some implementations, different data sources (e.g., data source 204) and their applications operate at different cycle times or intervals. Relatedly, various data sources and destinations (and their applications) require different levels of time determinism for the many data flows between them. In conventional TSN systems, a convergence cycle time (commonly referred to as a "management cycle time") is determined that operates on all data flows in the network. However, in some implementations of the subject disclosure, the integrated TSN-5G system may use a set of discrete / quantized cycle times in the network. Each data flow selects one of the available quantized cycle times to operate. Scheduling of scheduled transmissions in the TSN-5G system 100 may be based on a quantized / discrete set of cycle times. As an example, the integrated TSN-5G system 100 may limit the available stream intervals, and therefore the corresponding cycle times, to a set of discrete values, including but not limited to 1, 10, 100, 1000 milliseconds in nature. Similarly, stream or data flow requirements may be limited, for example, jitter (packet delay variation) requirements may be essentially limited to a predefined set of discrete values, including but not limited to 1, 10, 100, 1000 microseconds. In some implementations, different sets of discrete values ​​may be used depending on the applications and use cases supported by the integrated TSN-5G system. For example, a geographically distributed system may use discrete cycle times in milliseconds. In yet another example, a system limited to a local factory may use discrete cycle times in microseconds. The members of this discrete set may not default to a contiguous set of cycle times, but may be regularly or irregularly spaced or follow other statistical distributions (including but not limited to logarithmic, linear, Gaussian). In another implementation, the set of cycle times is standardized in such a way that all TSN blocks have cycle times that are products of elements selected from a small common set of prime numbers.This allows all composite cycle times to be easily calculated by the TSN scheduler, resulting in one common network cycle time.

[0051] Each TSN block of the integrated TSN-5G system may support a set of cycle times (where the set includes one or more cycle times). The CM 215 configures the TSN-5G system 106 or 200 to enable scheduled transmission of data flows between TSN blocks 202 operating with different cycle times. In some implementations, the TSN blocks 202 may be required to operate with compatible cycle times, where compatible means that the cycle times are integer multiples of each other. When an application requests an interval that does not map directly to the available set of individual cycle times across the set of TSN blocks 202 that the flow traverses, the CM 215 may match the closest available cycle time. The closest available cycle time should be an integer multiple or an integer divisor of the available cycle times. The CM 215 may exchange supported quantized / discrete sets of cycle time information with each other during the configuration process. In this example, the subject disclosure enables a distributed TSN-5G system to create viable configurations for multiple data streams / flows. Without quantized cycles / intervals, construction would typically require long computation times, which may prevent even the discovery of a feasible solution.

[0052] In some examples, each integrated TSN-5G network slice of the 5G system 106 may have a predefined set of supported cycle times and jitter bounds. In some examples, the network slices may be more granular than a typical 5G network slice, in accordance with 3GPP specification 23.501, which is incorporated herein by reference. In some implementations, the TSN-5G system 106 or 200 may be sliced ​​based on the TSN cycle time. For example, a 5G network supporting multiple critical services may have URLLC slices dedicated to the periodicity of the applications and their streams. For example, a service with an application that operates with a periodicity of about 1 millisecond may have a dedicated slice that operates with a cycle time of 1 millisecond. Similarly, a service and application that operates with a periodicity (or interval) of 100 milliseconds may have a dedicated slice that operates with a cycle time of 100 milliseconds in the integrated TSN-5G system. Such cycle time slicing improves both the speed of configuration as well as the overall performance of the network. In some implementations, the TSN blocks 202 expose the supported cycle times for a particular slice to their respective CMs 215 through the ICI 406. The CMs 215 can then exchange the supported cycle times of the TSN blocks under their management with each other via the interconfiguration module API 230 to create a configuration solution for a sliced ​​TSN-5G system.

[0053] In some implementations, the solutions or configuration data determined by the CM 215 may include, but are not limited to, a collective or configuration set of timing, commands, controls, instructions, etc., or combinations thereof, for operating each TSN block 202 according to the characteristics (e.g., defined by a set of parameters) of the TSN block 202. In some aspects, the configuration data may include specific transmission information regarding individual or collective (e.g., “global”) data frame transmissions to one or more respective TSN blocks 202. The transmission information may include time information regarding the transmission of the data frames. In one or more aspects, the configuration data of the data frames may include a transmission start time. For example, the transmission start time may be a time when the transmission of the data frames from the respective TSN block 202 begins. In an aspect, the transmission of the data frames may be initiated by selectively opening a gate of the respective TSN block 202 to transmit the data frames as a data flow to a destination node (e.g., another TSN block 202). Conversely, the transmission of the data frames may be stopped or prevented by selectively closing a gate of the respective TSN block 202 for transmitting the data frames. The configuration data may define or assign specific paths or links communicatively coupling each TSN block 202 with another node to transmit data flows thereover. Additionally, the configuration data may define a period of transmission of each data flow from each TSN block 202. In one aspect, the period of data flow transmission may be defined by a time interval between selective opening of a gate (i.e., transmitting data frames) and selective closing of a gate of the respective node (i.e., ceasing transmission of data frames to a destination node).

[0054] In conventional TSN systems, the TSN schedule is expressed as an absolute time offset of a periodic cycle in which the TSN block is instructed to transmit its data. However, for a 5G system 106 composed of components from multiple vendors, this may be too strict. In some implementations, a deadline-based schedule is determined and configuration data is provided by the CM 215. The deadline-based schedule may instruct the TSN block 202 to transmit data of the configured data flow by a deadline (expressed as an absolute time in a periodic cycle). In some implementations, a delay budget-based approach instructs the TSN block 202 to transmit data frames of the configured data flow within a delay budget. Thus, in a delay budget-based approach, the TSN block 202 is required to transmit data frames arriving at an input port to an output port within a certain period. In such a scheme, it is not necessary to time-synchronize all the TSN blocks 202. In some implementations in which the TSN block 202 is configured under rate constraints, the TSN block 202 is configured to transmit data frames of a given data flow in such a way that the average or peak transmission rate (in bits / second) does not exceed a configured value (according to configuration data from the CM 215).

[0055] In some 5G systems, TSN blocks may be a set of shared resources made available by network slicing based on service profiles that limit periodic cycles including but not limited to network latency, delay / budget. In such implementations, there may be two levels of scheduling, and in 5GS TSN-AF, in addition to slice-level TSN scheduling, it may be possible to configure the TSN blocks 202 as shared resources. In either case, the TSN blocks of the appropriate configuration may be identified by configuration attributes such as resource identifiers. As an example, a service provider may have multiple service profiles with specific periodic cycles, and multiple tenants of the service provider may utilize the same TSN blocks specified in the 5GS TSN-AF configuration and perform aggregation of TSN flows. In some implementations, the service provider may provide a set of non-shared TSN blocks where only a single layer of service profiles may exist. Device-specific operation / required resource sharing modes may be available to the CNC 112 through the TSN-AF.

[0056] As an exemplary implementation of a deadline / delay budget approach by the TSN block 202, when a data frame arrives at an input port of the TSN block 202, the TSN block 202 records the arrival time of the data frame using a local clock. The TSN block 202 may then identify that the frame belongs to a configured data flow and start a countdown timer equal to the configured delay budget for that data flow. Using the transmission module 408, the TSN block 202 may prioritize transmission of data with the least amount of time remaining. If a packet's timer expires before the packet is transmitted, the event is recorded as a transmission failure and detailed in the monitoring metrics by the recording module 410.

[0057] In some implementations, for scheduled transmissions between TSN block 202-1 (corresponding to UE 118) and TSN block 202-2 (corresponding to RAN 120), an "extended" allocation (allocation and transmission) of uplink and downlink transmissions between UE 118 and RAN 120 may be included to satisfy the schedule assigned by CM 215-a to TSN block 202-1 corresponding to UE 118. In some implementations, CM 215-a may consider UE 118 buffer status and radio conditions reported by RAN 120 when instantiating the TSN schedule and may adjust or report necessary changes to the requested schedule. In some other implementations, CM 215-a may send real-time feedback regarding radio conditions received from RAN 120 to a master CM, e.g., CM 215-d. This feedback loop may support recalculating the TSN schedule to satisfy the packet delay budget for this particular TSN block or across TSN blocks on a given end-to-end path.

[0058] In this implementation, link quality is monitored and the CM 215-a may continuously adjust the radio resources of the configuration data to meet the transmission schedule. Radio resources may include, but are not limited to, logical channels, transmission power, and UE-specific slot durations. In some implementations, a static assignment of fixed / deterministic uplink and downlink slots may be made for a given UE 118, e.g., all UEs connected to a given RAN slice are given scheduled transmission slots. 5G native radio scheduling may be used to determine whether a transmission from the UE 118 meets a transmission deadline. If not, the UE 118 may request elevated access to the RAN 120 to fulfill the scheduled transmission. In some implementations, scheduling in the 5G system 106 for scheduled transmissions may be based on a quantized / discrete set of cycle times, the set including a management cycle time of at least 100 milliseconds. In accordance with the subject technology, a radio link between a UE (e.g., represented by the TSN block 202-1) and a RAN (e.g., represented by the TSN block 202-2) may be sliced ​​based on cycle times. In some implementations, the uplink and downlink between TSN block 202-1 and TSN block 202-2 may have radio resources assigned to each network slice based on the cycle time of the slice. For example, a cycle time slice of 1 millisecond requires radio resources (channels, airtime, etc.) that can deliver data at a rate of 1 millisecond.

[0059] In some implementations, 5G resource elements (e.g., frequencies and time slots) may be scheduled to meet the latency requirements of a TSN flow in addition to the "standard" 5G scheduling traffic prioritization requirements. More specifically, a 5G time slot for a TSN flow may be assigned to transmit the messages of the TSN flow with both the appropriate cycle time offset (phase) and time constraints (TSN window time) required by the TSN schedule. In this case, the 5G scheduler differs from a "traditional" TSN Ethernet port in that multiple messages may be output simultaneously if transmitted on different frequencies. In some aspects, under adverse RF channel conditions, the 5G system 106 may transmit multiple copies of a message on different frequencies to increase the likelihood of meeting the transmission schedule and / or deadline.

[0060] To achieve reliable data transmission in the system 200, redundant flow paths may be implemented. The distributed TSN block 202 of the 5G system 106 allows errors (delay, dropped, or corrupted frames) to be handled in a better way. In some implementations, the UE 118 may initiate two redundant split PDU sessions to the UPF 122 for redundancy, in which case the 5GC may configure the NG-RAN for dual connectivity per 3GPP 38.300. In some other implementations, FRER may be used between some TSN blocks 202 but not between other TSN blocks 202. For example, redundant streams may be implemented over the air interface between the UE 118 (TSN block 202-1) and the RAN 120 (TSN block 202-2), then combined at the RAN 120, and potentially split again via the core network (TSN blocks 202-3, 202-4) if necessary. As shown in FIG. 1, the current redundancy requirement according to 3GPP 23.501 is to maximize separation of paths between the UE 118 and the UPF 122. However, according to the subject technology, redundant separated paths need not be established throughout the entire 5G system, but instead may be implemented only for a portion of the 5G system. For example, for the air interface between the UE 118 and the RAN 120, the redundancy requirement may specify that the two paths must be on different frequencies, or different MIMO channels, or different time slots. In some examples, the subject disclosure allows for flexible use of redundancy, including, but not limited to, support for more than two data paths, combining and splitting data flows between TSN blocks, and TSN blocks with various degrees of redundancy capabilities.

[0061] The concept of splitting 5GS into multiple TSN blocks (e.g., as described above with respect to FIG. 2) may potentially enhance security if strict scheduling can prevent the flow of malicious traffic between said blocks. However, enabling 5GS to operate as multiple TSN blocks may introduce security vulnerabilities, primarily through configuration. Specifically, TSN users may become aware of internal 5GS details and connectivity that impact other users sharing the same physical and logical infrastructure. This may occur during the required TSN network discovery phase (e.g., through information returned by the Link Layer Discovery Protocol (LLDP)). TSN users may also misconfigure their own or other users' configurations. This may be partially addressed by using the NETCONF concept of secure subtrees, where users have a limited reference to their own data model subtrees. 5G systems may have a concept of essentially isolated network slices, depending, for example, on whether TSN is implemented virtually (through software) or physically (through hardware). Infrastructure providers may need to set limits on the physical and logical capabilities exposed to each TSN user. This may be done via a 5G Network Exposure Function (NEF). The subject disclosure provides a solution to the security problem by using a "Virtual TSN Block". A Virtual TSN Block is an internal 5G TSN Block that contains only the capabilities provided by the user's 5G network slice. In other words, the user can only see and configure the TSN block information exposed via the 5G network slice, and nothing more. In this sense, an internal 5G TSN Block is the intersection of the set of information contained in the internal TSN Block and the 5G network slice provided to the user.

[0062] In some implementations, the IETF DETNET standard (provided by the IETF: "Deterministic Networking Working Group" - https: / / datatracker.ietf.org / wg / detnet / about / ) may be implemented or integrated into the 5GS to interconnect small islands of Ethernet that are TSN compliant. The 5GS may utilize DETNET to enable transport of TSN messages over IP Layer 3 (rather than Layer 2). Envisioning such a system, the techniques described in this disclosure include 5GS supporting DETNET edge, relay, and transit nodes that interconnect spatially separated TSN networks, creating a larger composite TSN over 5G. All aspects of the integrated TSN-5G system provided in this disclosure are applicable (but not limited to) to TSN islands of 5G DETNET, particularly TSN blocks of distributed TSN.

[0063] DETNET consists of the following components: (1) TSN End Systems: IEEE compliant end systems that communicate with DETNET Edge Nodes; (2) DETNET Edge Nodes: process TSN frames into DETNET; (3) DETNET Relay Nodes: provide congestion avoidance for time-sensitive messages; (4) DETNET Transit Nodes: provide congestion avoidance for time-sensitive messages. DETNET is routed, not bridged, allowing routable TSN messages between TSN LANs. Essential TSN Ethernet frame information is forwarded or reconstructed in the transport between LANs. DETNET achieves higher layer functionality by adding sublayers: (1) DetNet Service Sublayer: provides DetNet services (e.g., service protection) to higher layers of protocol stacks and applications; and (2) DetNet Transport Sublayer: supports DetNet services (e.g., by providing explicit routes and congestion protection) in the underlying network to DetNet flows and encapsulates TSN Ethernet frames. In DETNET routing, IP headers are modified according to standard router behavior, e.g., time-to-live (TTL processing), which specifies the maximum time a routable IP message is allowed to live, which has a clear relationship to the maximum latency requirements of TSN.

[0064] The DETNET component may reside within any computational element of the 5GS, specifically within the 5G MEC or core. Thus, in one implementation, the 5GS may be a fully compliant DETNET that interconnects TSN LANs. To provide determinism, the DETNET may reserve data plane resources for the DetNet flow at some or all of the intermediate nodes along the path of the flow. The DETNET may provide explicit routes for the DetNet flows. The DETNET may distribute data from the DetNet flow packets across time and / or space to ensure that the data in each packet is delivered even if a path is lost. Thus, as described above, the TSN CNC / DNC and scheduler may need to interact with the DETNET, specifically the ability to configure flow paths (including redundant flow paths), TTLs, and obtain routing latency and jitter. TSN traffic shapers, time-aware shaping, and network computation may be used to achieve the required level of determinism in the 5G DETNET.

[0065] It should also be noted that hybrid 5GS, part TSN (Layer 2) and part DETNET (Layer 3), may coexist and interoperate, in which case the DETNET portion of the network itself may be treated as a TSN block (or blocks) 202, as described above.

[0066] Additionally, with regard to caching within 5GS to minimize latency and increase determinism for TSN applications, the amount of storage, location, and naming of information in 5GS may be managed to allow for fast and short-term close access to minimize jitter. Each piece of information may be cryptographically signed to improve security and access, and provided through encrypted signatures and hashes of information stored in 5GS components such as MEC. The cache forwarding unit tracks each data request, allowing optimal forwarding, placement, and servicing of cached data. The TSN CNC / DNC and scheduler may calculate where to place cached information within the network (specifically 5GS) to maximize access between 5G applications and minimize jitter (variation in packet delay).

[0067] Real-time MEC applications typically have well-defined characteristics of their behavior. Cloud (Cloud Computing) and Fog (Fog Computing) 5G MEC applications can be divided into microservices with hard real-time constraints provided to the TSN scheduler. The constraints can be the longest time to complete a call to a service (worst case) or a well-defined statistical description according to the network computation requirements of the arrival or service curve. In some implementations, microservices can be chained together to create a complete MEC application. By dividing the computation into a series of small microservices, each service can be better controlled and managed, providing more determinism. Each microservice can be abstracted as an internal TSN block, consisting of deterministic inputs, outputs, schedulable behavior, and coordination with the 5G-TSN AF. The microservices can reside on the same or spatially different processing systems interconnected via TSN scheduled communication.

[0068] Such hard real-time processing may include 5G MEC and 5G core functions. Messages flow in and out of the MEC TSN block according to a deterministic schedule that can be calculated by the TSN scheduler. Note that such TSN blocks are called “computational TSN blocks”. Since the messages generated by the MEC and their corresponding size and transmission times may vary depending on the computational complexity of the MEC processing tasks, the processing load, and the application state, the TSN scheduling component may utilize an internal model, such a model being a simulation, emulation, or purely analytical, or a hybrid of each (also called a digital twin), of the MEC processor for the purposes of TSN scheduling. The TSN scheduling component can then generate a complete end-to-end TSN schedule for all messages flowing between the 5G UE, MEC, 5G core, and any cloud processing required for real-time applications, with the 5G core and cloud processing being modeled similarly by the TSN scheduler. The TSN schedule can dynamically recalculate the schedule as necessary to maintain the determinism required for 5G TSN real-time MEC / cloud applications as the effective processing speed, and therefore the transmission time of the outgoing message, changes. The TSN scheduler can provide feedback to the application developer (and for management and deployment) as to the optimal location (UE, MEC, cloud) for each processing component of the real-time 5G application, taking into account link speeds, variations, processing power, available memory, etc. TSN systems may employ rate control mechanisms such as gate-based approaches or leaky buckets, and may use any number of optimization techniques and network computations to determine a feasible schedule. Thus, the complete flow of a TSN application includes not only the end-to-end path of a particular message through the network, but also the complete processing path through all computational TSN blocks (microservices) that act on the message's information.IEEE 802.1CB redundant paths can be configured through redundant or parallel computational TSN blocks (microservices). In this invention, it should be possible to visualize the complete real-time processing activity encoded in the TSN schedule, where the computational TSN blocks are displayed as Ethernet bridges with the difference that they can process messages or create new messages that are sent on a deterministic schedule.

[0069] MEC applications are configured to use TSN flows (join as TSN talkers or listeners) using NETCONF, RESTCONF, or RESTful APIs. The messages defined in the above protocols contain the IEEE 802.1Qcc information necessary to configure the TSN flows used by the MEC application. Additionally, MEC applications are designed to move from one MEC platform to another, and a RESTful API is defined to query the TSN configuration of the current platform to ensure that it has the necessary deterministic communication requirements, including latency and jitter in particular. It also has a RESTful API that contains the above-mentioned information necessary to inform the TSN CNC of its new location, and to dynamically reschedule TSN traffic to the new location. As mentioned above, IEEE 802.1CB can be used to establish redundant TSN flows to expected locations to which the MEC application may migrate. The TSN scheduler, i.e., Centralized Network Configurator (CNC) or Distributed Network Configurator (DNC), can be a MEC application that provides 5G TSN scheduling as a service and configuration as a service, which is useful for dynamic rescheduling where low latency is required.

[0070] A TSN application may transmit a timing performance profile query microservice request with the goal of determining statistical information regarding processing times. When responding to such a request, the microservice executes the request and returns both the results and a timing performance profile. The timing performance profile may include at least the network start and end times of the microservice invocation, and may optionally include the start and end times of all subfunctions that are invoked. The information may also include the average MEC processor load during the timing performance profile. The timing performance profile may be used by the TSN application to adjust the TSN scheduling required by the microservice to process the TSN traffic flow. The timing performance profile may be obtained during live operation, where the output from the profile is typically used, or as a special test sample prior to live operation. The timing performance profile information may be used to determine which MEC hardware to employ for current and future operation, and as constraint information for the TSN scheduler.

[0071] The subject disclosure also provides additional new aspects, namely: (1) TSN scheduling 5G core functions; (2) ensuring that CPUs have direct time synchronization with Ethernet hardware timestamp mechanisms; (3) enabling TSN scheduling of specific 5G network functions and processes; (4) adding new time-aware programming capabilities, such as, for example, time-based event processing; and (5) integrating time-based conditional processing, e.g., implementing service chaining with real-time programming implemented via YANG configuration of microservice scheduling (see https: / / www.rfc-editor.org / rfc / pdfrfc / rfc7758.txt.pdf for examples of YANG scheduled operations).

[0072] Additionally, a LinkDelayStatistic (implemented as a YANG model) cumulative distribution function data model may be implemented within the TSN-5G system 106 or 200. In such an implementation, the wireless links may collect and provide information to the CNC's scheduler, which then utilizes it to schedule the variable speed links. The LinkDelayStatistic YANG model may also provide information on whether the link delay is stationary and ergodic. The scheduler may use this information to predict the future based on a given sample and create an accurate schedule. The CNC may then leverage this knowledge for each TSN block to deploy the best possible TSN traffic shaping or gating schedule, including using network calculations in determining the outcome.

[0073] The subject disclosure also contemplates Virtualized Network Functions (VNF-TSN) that can exist as software within a 5G system. Specialized processor hardware may be required to support real-time operation. In some implementations, TSN may be provided as Software-Defined TSN (SDN-TSN) and 5G TSN-as-a-service (TaaS). In some implementations, VNF-TSN may be configured within all 5G subcomponents (TSN blocks), such as UE, radio head, CU / DU, RAN, MEC, and core. As discussed above, microservices may be chained together as part of TSN scheduling. Each microservice may expose its service time characteristics to the TSN schedule. Services are part of the delay, but may have larger variations than the communication links. In such implementations, services become the link in the integration of microservices and TSN scheduling. Network calculations may be used to incorporate processing delays. TSN inputs provide well-defined arrival curves. Processor execution times provide service curves (assuming 5G equipment has well-characterized processing times).

[0074] In another aspect of the subject disclosure, a time-aware MEC platform is defined to a platform that functions as a PTP client (compliant with 802.1AS end station) and, if necessary, a PTP bridge (compliant with 802.1AS bridge). A PTP bridge is required for a virtualized MEC platform that runs multiple slices (OS, VMs, containers) in parallel, along with network bridging capabilities. Typically, virtual bridges / switches are used in virtualized computing platforms. In this disclosure, a TSN-enabled virtual switch is defined that includes time awareness. The time-aware PTP client of the MEC platform runs a PTP state machine along with a servo to synchronize a locally available clock with the network's grandmaster clock. Additionally, the MEC platform synchronizes the system clock to the PTP clock, where the system includes the operating system, network stack, application stack, or any other software and hardware element that utilizes clocks. This allows each MEC application to operate with synchronized PTP time. In one example, the MEC host runs this PTP client and provides synchronized time (referred to as system / host time) to all MEC applications through the virtualization infrastructure. In another example, every MEC application may run a separate instance of a PTP client connected to the host via a time-aware bridge.

[0075] As a configuration interface, 3GPP 23.501 specifies a centralized configuration model where the CNC configures the 5GS as a time-aware bridge. Similarly, the MEC platform needs to be configurable with the CNC as a time-aware end station or a time-aware bridge. This MEC may support configuration using the 802.1Qcw yang model of TSN features such as time-aware shaping, forwarding, and frame duplication and deletion for redundancy. Additionally, the MEC host may have a CUC component that uses an 802.1Qcc interface to provide the data flow requirements of resident applications to the CNC. The MEC may also provide information about its TSN capabilities to the CNC so that the CNC can accurately model the MEC. For example, the MEC may present itself as a TSN end station originating a certain set of data flows. The CNC will model the MEC appropriately in the network and generate the correct configuration. The MEC will work with the OS and applications to support data stream identification. The specific capabilities will vary depending on the TSN awareness of the MEC component. Non-TSN-aware applications on MEC hosts require IP stream identification at the MEC bridge.

[0076] The TSN fronthaul / backhaul (e.g., TSN block 202) may directly connect to the MEC and provide deterministic input to the MEC. However, MEC applications are intended to continually migrate, or perhaps more intuitively, "float", to the edge of the 5G network to stay closest to potential mobile clients. Therefore, MEC applications that require determinism specify a minimum acceptable packet delay variation (MPDV) threshold. MPDV may be built into the specification of all MEC applications, which may limit the location of executable applications to those MEC platforms that exist as TSN blocks in 5GS. Applications need to ensure that appropriate TSN stream identification and translation rules are implemented in the new MEC platform (which may be a single processor or a sub-network of processors) so that MEC talker / listener message frames are tagged and processed appropriately. MEC applications may need to carry their own configuration instructions, when possible, to "self-install" when migrating to a new MEC platform. Note that load balancing mechanisms may be employed to ensure that users do not overload a set of MEC applications and that the MEC applications are distributed in an optimal manner. In other scenarios, redundant MEC platforms and applications may be instantiated and / or MEC applications may be migrated, not strictly due to mobility, but due to noise, and multiple MECs may be connected as a TSN redundant system.

[0077] 6A and 6B show an integrated TSN-5G system 600 (similar to system 200) that includes a TSN block 605 (similar to TSN blocks 202-N) that represents or corresponds to a multi-access edge computing (MEC) module. The TSN block 605 is configured as a data source / sink (i.e., a TSN end station), but rather than being located at the edge of the 5G system 106, it is located within the 5G system 106. The TSN block 605 may be configured as a bridge end station.

[0078] Referring to FIG. 8, an application data stream may span multiple integrated TSN-5G systems 805 and 810. In this case, the TSN block may be geographically distributed, interconnecting multiple 5G systems using a TSN compatible transport 820, including but not limited to a 5G backbone, a private network tunnel, or other wide area network. In the subject invention, the TSN block is configured by each CM 215. In some implementations, the configuration between the two TSN-5G systems 805, 810 may be coordinated through a centralized configuration utility (CNC) 112. In some other implementations, the CMs 215 between the two TSN-5G systems 805, 810 may communicate directly. In some implementations, multiple CUCs and CNCs may be used to capture user requirements and generate the TSN solutions and techniques (e.g., TSN schedules, forwarding instructions, etc.) provided in this disclosure.

[0079] 7 illustrates an electronic system 700 in which one or more implementations of the subject technology may be implemented. The electronic system 700 may be and / or be part of the TSN block 202 and / or configuration module 215. The electronic system 700 may include various types of computer readable media and interfaces for various other types of computer readable media. The electronic system 700 includes a bus 708, one or more processing units 712, a system memory 704 (and / or buffers), a ROM 710, a persistent storage device 702, an input device interface 714, an output device interface 706, and one or more network interfaces 716, or a subset and variations thereof.

[0080] The bus 708 collectively represents all system, peripheral, and chipset buses that communicatively connect the various internal devices of the electronic system 700. In one or more implementations, the bus 708 communicatively connects one or more processing units 712 with the ROM 710, the system memory 704, and the persistent storage device 702. From these various memory units, the one or more processing units 712 obtain instructions to execute and data to process to perform the processes of the subject disclosure. The one or more processing unit(s) 712 can be a single processor or a multi-core processor in various implementations.

[0081] ROM 710 stores static data and instructions required by one or more processing unit(s) 712 and other modules of electronic system 700. Persistent storage device 702, on the other hand, may be a readable and writable memory device. Persistent storage device 702 may be a non-volatile memory unit that stores instructions and data even when electronic system 700 is off. In one or more implementations, a mass storage device (such as a magnetic or optical disk and its corresponding disk drive) may be used as persistent storage device 702.

[0082] In one or more implementations, a removable storage device (such as a floppy disk, flash drive, and corresponding disk drive) may be used as the persistent storage device 702. Like the persistent storage device 702, the system memory 704 may be a readable / writeable memory device. However, unlike the persistent storage device 702, the system memory 704 may be a volatile readable / writeable memory, such as a random access memory. The system memory 704 may store any of the instructions and data that may be needed by the one or more processing unit(s) 712 during execution. In one or more implementations, the processes of the subject disclosure are stored in the system memory 704, the persistent storage device 702, and / or the ROM 710 (each implemented as a non-transitory computer-readable medium). From these various memory units, the one or more processing unit(s) 712 retrieves instructions to execute and data to process to execute the processes of one or more implementations.

[0083] The bus 708 is also connected to an input device interface 714 and an output device interface 706. The input device interface 714 allows a user to communicate information or select commands to the electronic system 700. Input devices that may be used with the input device interface 714 may include, for example, an alphanumeric keyboard and a pointing device (also referred to as a "cursor control device"). The output device interface 706 may allow, for example, the display of images generated by the electronic system 700. Output devices that may be used with the output device interface 706 may include, for example, a printer and a display device such as a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a flexible display, a flat panel display, a solid state display, a projector, or other device for outputting information. One or more implementations may include a device that functions as both an input device and an output device, such as a touch screen. In these implementations, the feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, haptic feedback, etc., and the input from the user may be received in any form, including acoustic, voice, and tactile input.

[0084] 7, the bus 708 couples the electronic system 700 to one or more networks and / or one or more network nodes through one or more network interface(s) 716. As such, the electronic system 700 can be part of a network of computers, such as a LAN, a wide area network ("WAN"), or an intranet, or a network of networks, such as the Internet. Any or all of the components of the electronic system 700 can be used in combination with the subject disclosure.

[0085] These functions described above can be implemented in computer software, firmware, or hardware. The techniques can be implemented using one or more computer program products. The programmable processor and computer can be embedded in or packaged as a mobile device. The processes and logic flows can be executed by one or more programmable processors and one or more programmable logic circuits. General-purpose and special-purpose computing devices and storage devices can be interconnected through a communication network. The functions disclosed herein can be implemented using quantum computing, pulse-coupled oscillations (PCO) / Ising computing.

[0086] Some implementations include electronic components such as microprocessors, storage, memory, etc. that store computer program instructions on machine-readable or computer-readable media (also referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of such computer-readable media include RAM, ROM, read-only compact discs (CD-ROMs), recordable compact discs (CD-Rs), rewritable compact discs (CD-RWs), read-only digital versatile discs (e.g., DVD-ROMs, dual-layer DVD-ROMs), various recordable / rewritable DVDs (e.g., DVD-RAMs, DVD-RWs, DVD+RWs, etc.), flash memory (e.g., SD cards, mini SD cards, micro SD cards, etc.), magnetic and / or solid-state hard drives, read-only and recordable Blu-Ray® discs, ultra-high density optical discs, any other optical or magnetic media, and floppy disks. A computer-readable medium can store a computer program that is executable by at least one processing unit and includes a set of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as produced by a compiler, and files containing higher level code that are executed by a computer, electronic component, or microprocessor using an interpreter.

[0087] While the above description primarily refers to a microprocessor or multi-core processor executing software, some implementations are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some implementations, such integrated circuits execute instructions stored on the circuit itself.

[0088] As used in this specification and in all claims of this application, the terms "computer," "server," "processor," and "memory" all refer to electronic devices or other technological devices. These terms exclude people or groups of people. For purposes of this specification, the terms display or display mean display on an electronic device. As used in this specification and in all claims of this application, the terms "computer-readable medium" and "computer-readable media" are entirely limited to tangible physical objects that store information in a form that a computer can read. These terms exclude any wireless signals, wired download signals, and any other ephemeral signals.

[0089] To provide for user interaction, implementations of the subject matter described herein can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user, and a keyboard and pointing device, e.g., a mouse or trackball, by which the user can provide input to the computer. Other types of devices can also be used to provide for user interaction, e.g., feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or haptic feedback, and input from the user can be received in any form, including acoustic, kinetic, or tactile input. Additionally, the computer can interact with the user by sending documents to and receiving them from a device used by the user, e.g., sending a web page to a web browser on the user's client device in response to a request received from the web browser.

[0090] Aspects of the subject matter described herein may be implemented in a computing system that includes a back-end component, e.g., as a data server, a middleware component, e.g., as an application server, or a front-end component, e.g., a client computer having a graphical user interface or a web browser through which a user can interact with embodiments of the subject matter described herein, or any combination of one or more such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication, e.g., a communications network. Examples of communications networks include local area networks ("LANs") and wide area networks ("WANs"), internetworks (e.g., the Internet), and peer-to-peer networks (e.g., ad-hoc peer-to-peer networks).

[0091] Those skilled in the art will appreciate that the various exemplary blocks, modules, elements, components, methods, and algorithms described herein may be implemented as electronic hardware, computer software, or a combination of both. To illustrate this interchangeability of hardware and software, the various exemplary blocks, modules, elements, components, methods, and algorithms have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design constraints imposed on the overall system. The described functionality may be implemented in a variety of ways for each particular application. The various components and blocks may be arranged differently (e.g., arranged in a different order or divided in a different way) without departing from the scope of the subject technology.

[0092] It is understood that the specific order or hierarchy of steps in the processes disclosed is an illustration of example approaches. Based on design preferences, it is understood that the specific order or hierarchy of steps in the processes may be rearranged. Some steps may be performed simultaneously. The accompanying method claims present elements of the various steps in a sample order, and are not intended to be limited to the specific order or hierarchy presented.

[0093] The above description is provided to enable those skilled in the art to practice the various aspects described herein. The above description provides various examples of the subject technology, but the subject technology is not limited to these examples. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Thus, the claims are not intended to be limited to the aspects set forth herein, but are to be accorded the full scope consistent with the language of the claims, and references to elements in the singular do not mean "only one" unless otherwise specified, but rather "one or more." The term "some" refers to one or more, unless otherwise specified. Masculine pronouns (e.g., his) include feminine and neuter pronouns (e.g., her and its), and vice versa. Headings and subheadings are used for convenience only and are not intended to limit the disclosure set forth herein.

[0094] The predicates "configured to," "operable to," and "programmed to" do not imply any particular tangible or intangible modification of the subject matter, but rather are intended to be used interchangeably. For example, a processor configured to monitor and control operations or components may also refer to a processor that is programmed to monitor and control operations, or a processor that is operable to monitor and control operations. Similarly, a processor configured to execute code may be interpreted as a processor that is programmed to execute code, or a processor that is operable to execute code.

[0095] As used herein, the term automatic may include being performed by a computer or machine without user intervention, such as by instructions in response to a predicated action by a computer or machine or other initiating mechanism. The word "example" is used herein to mean "serving as an example or illustration." Any aspect or design described herein as "example" is not necessarily to be construed as preferred or advantageous over other aspects or designs.

[0096] A phrase such as "aspect" does not imply that such aspect is essential to the subject technology or that such aspect applies to all configurations of the subject technology. Disclosure regarding an aspect may apply to all configurations, or to one or more configurations. An aspect may provide one or more examples. A phrase such as an aspect may refer to one or more aspects, or vice versa. A phrase such as "embodiment" does not imply that such embodiment is essential to the subject technology or that such embodiment applies to all configurations of the subject technology. Disclosure regarding an embodiment may apply to all embodiments, or to one or more embodiments. An embodiment may provide one or more examples. A phrase such as an embodiment may refer to one or more embodiments, or vice versa. A phrase such as "configuration" does not imply that such configuration is essential to the subject technology or that such configuration applies to all configurations of the subject technology. Disclosure regarding a configuration may apply to all configurations, or to one or more configurations. A configuration may provide one or more examples. A phrase such as "configuration" may refer to one or more configurations, or vice versa.

[0097] All structural and functional equivalents to the elements of the various aspects described throughout this disclosure that are known or later become known to those of ordinary skill in the art are intended to be expressly incorporated herein by reference and encompassed by the claims. Moreover, nothing disclosed herein is intended to be dedicated to the public, regardless of whether such disclosure is expressly recited in the claims. No claim element shall be construed under the provisions of 35 U.S.C. § 112(f) unless the element is expressly recited using the word "means" or, in the case of a method claim, using the word "step." Edge Control

[0098] Messages in control networks need to be reliable and exhibit low and deterministic latency, i.e. they need to be delivered over the network with minimal delay variation or jitter. Control messages are usually relatively short but highly critical. Examples include vehicle control (automotive or aircraft), power grid monitoring and control (DNP3 and IEC 61850), and avionics full-duplex switched Ethernet (AFDX).

[0099] FIG. 9 illustrates a 5G system 900 (similar to 200 and 600 above) including a TSN block representing or corresponding to a MEC module (also referred to as a MEC host). The MEC host includes one or more multi-access edge services, such as a radio network information service (RNIS), and MEC traffic rules (e.g., including one or more TSN configurations). The MEC host further includes one or more MEC applications. In one example, the MEC application can be a vehicle engine control application configured to transmit and receive engine control data to and from an aircraft or automobile within a service area of ​​a radio access network (e.g., one or more of the distributed units of the radio access network). In another example, the MEC application is a power grid monitoring and control application configured to transmit and receive control data to and from a power grid infrastructure within a service area of ​​a radio access network (e.g., one or more of the distributed units of the radio access network).

[0100] Wireless systems are inherently less reliable, more susceptible to longer latency, and more variability in message delays than wired networks (e.g., wired Ethernet) required for control systems. Wireless 5G messages (e.g., control data) in system 900 are transmitted from the user equipment (UE) through the RAN and other centralized telecommunications processing (e.g., User Plane Function (UPF) and Local Data Network (LDN) processing) to the MEC module and back to the UE. While assumed to be fast and efficient for telecommunications resources, this is a long journey for every wireless 5G message, which can lead to more opportunities for delay variation and message loss. To address this, 3GPP standards have adopted the Multi-Access Edge Computing (MEC) standard from ETSI to allow for custom processing to be deployed within the 5G network.

[0101] In some implementations, one or more components (e.g., edge-enabled devices, applications, and / or services) of the 5G system 900 may be implemented by one or more of the following specifications, which are incorporated by reference in their entirety herein: 3GPP TR 23.758-Studies on Application Architecture for Enabling Edge Applications, 3GPP TS 23.558-Architecture for Enabling Edge Applications, 3GPP TR 23.748-5GC Studies on Enhancements to Support Edge Computing, 3GPP TR 33.839-5GC Studies on Security Aspects for Supporting Edge Computing, 3GPP TR 26.803-Studies on Streaming Architecture Extensions for Edge Processing, and 3GPP TR 28.814-Studies on Enhancements to Edge Computing Management.

[0102] FIG. 10 illustrates an improved scheme 1000 for implementing MEC in a 5G network according to some implementations. A radio access network (gNB) comprises at least one central unit (CU) coupled to a central network via a backhaul network, and multiple distributed units (DUs) coupled to the CU via a fronthaul network. The CU provides support for higher layers of the protocol stack, such as SDAP, PDCP, RRC, and each DU provides support for lower layers of the protocol stack, such as RLC, MAC, and physical layers. In some implementations, there is a single CU in each gNB, but one CU may control and be connected (communicatively coupled) to multiple DUs (e.g., more than 100 DUs). Each DU can support one or more cells, so one gNB can control hundreds of cells.

[0103] For 5G networks, the CU is a logical node that includes gNB functions such as user data forwarding, mobility control, radio access network sharing, positioning, and session management, except for those functions exclusively assigned to the DU. The CU controls the operation of the DU via the fronthaul (Fs) interface. The CU may also be called a baseband unit (BBU), radio equipment control (REC), rural connectivity community (RCC), centralized radio access network (C-RAN), or virtualized radio access network (V-RAN).

[0104] Each DU contains a subset of gNB functions depending on the functional partitioning option. Its operation is controlled by the CU. The DUs may also be called Remote Radio Heads (RRHs), Remote Radio Units (RRUs), Resource Elements (REs), or Radio Units (RUs), with which they are coupled, co-located, or otherwise associated.

[0105] As shown in FIG. 10, the implementation of MEC in a 5G network may be improved by moving MEC services and / or functions to the DU. Such a configuration may result in lower latency, more deterministic latency, and more customizable latency. This improved implementation may allow Wireless Time Sensitive Networking (WTSN) control applications to be deployed in a 5G network. A single 5G control application running on the MEC may be able to control a large number of devices over the wireless network, allowing for easier management and maintenance of the control software. In the implementation described herein, the MEC is so efficient and reliable that engine control may be made wireless and located entirely in the MEC.

[0106] The above improvements may be obtained by (1) accessing 5G MEC services with deterministic network control via Time Sensitive Networking (TSN) and / or (2) placing the MEC closer to the US using a direct connection to 5G fronthaul, resulting in even more efficient performance.

[0107] With regard to deterministic access to 5G MEC, this implementation differs from the standard for TSN over 5G (currently defined by 3GPP) because the MEC application resides in the 5G network and is not directly connected to the UE. TSN is specified to operate over 5G, assuming that a TSN translator is associated with the UE. Thus, with the implementation described herein, TSN configuration and control are redesigned to interoperate with MEC.

[0108] Regarding the placement of MEC, the closer the MEC processing can be placed to the UE, the better the communication performance can be. There are several ways to place MEC processing closer to the user.

[0109] In some implementations, MEC processing may be brought closer to the UE by connecting the MEC directly to the fronthaul, particularly the fronthaul over TSN.

[0110] In some implementations, MEC processing may be brought closer to the UE by enabling MEC homomorphic processing directly on I / Q data received from (or for transmission via) the DU's respective radio unit.

[0111] In some implementations, MEC processing may be moved closer to the UE by enabling migration of MEC services, allowing the MEC services to be efficiently migrated between nodes (DU to DU) so that the MEC services are as close as possible to the target application (e.g., a fast moving aircraft) while remaining integrated with the fronthaul network.

[0112] Figure 11 illustrates a communication system 1100 implementing an improved scheme (1000) for implementing MEC in a 5G network according to some embodiments. The communication system of Figure 11 is exemplary and may include more or less than one CU, more or less than five DUs, etc. In some implementations, the edge-enabled devices of Figure 11 may be interconnected with each other using an edge computer network (ECN).

[0113] The exemplary communication system shown in FIG. 11 includes a core network and a radio access network including a CU communicatively coupled to the core network via a backhaul network and to a plurality of DUs via a fronthaul network. Each of the plurality of DUs is co-located with and communicatively coupled to a respective radio unit (shown as an antenna) and includes a MEC module. The MEC module of each of the plurality of distributed units includes one or more processors and a memory that stores one or more programs executed by the one or more processors, the one or more programs including instructions for controlling a time-sensitive networking (TSN) configuration and for executing one or more MEC applications with the TSN configuration. In some implementations, the MEC module uses a representational state transfer application programming interface (RESTful API) to configure TSN on the MEC.

[0114] In some implementations, the MEC module of each of the multiple DUs is directly connected to a fronthaul network. In some implementations, the instructions for executing the one or more MEC applications include instructions for transmitting and receiving data (e.g., control data) over the fronthaul network via a TSN configuration.

[0115] In some implementations, the instructions for executing one or more MEC applications (at each DU) include instructions for processing analog user data associated with a co-located radio unit (a radio unit co-located with a particular DU). In some implementations, the user data is encoded in an in-phase quadrature (I / Q) signal, and the instructions for processing the analog user data include instructions for processing the I / Q signal with one or more homomorphic processing functions. Homomorphic processing includes applying a non-linear mapping to a different domain where a linear filter technique is applied, and then mapping back to the original domain. In other words, homomorphic processing may include converting data received from a CU for transmission via a radio head or from a radio head for transmission to a CU into a linear system (to separate I and Q signals), processing the converted signal (e.g., analyzing the signal converted by a particular MEC application), converting the signal back to its original form, and transmitting the signal (to the radio head or CU, in either case) in the received I / Q form.

[0116] In some implementations, the instructions for executing one or more MEC applications include instructions for executing a MEC service on a first DU 1102 of the multiple DUs, instructions for determining that a user device (e.g., an aircraft UE) associated with the MEC service is moving from a service area of ​​the first DU 1102 of the multiple DUs to a service area of ​​a second DU 1104, and instructions for transferring the MEC service to the second DU 1104 upon the determination (e.g., performing a handoff operation between the DU 1102 and the DU 1104).

[0117] In some implementations, the instructions for migrating the MEC service to the second DU 1104 include instructions for maintaining integration of the MEC service with the fronthaul network while migrating the MEC service to the second DU 1104. In some implementations, the instructions for migrating the MEC service to the second DU 1104 include instructions for converting the MEC application to a TSN flow and migrating the TSN flow from the MEC module of the first DU 1102 to the MEC module of the second DU 1104. In some implementations, the instructions for migrating the MEC service to the second DU 1104 include instructions for migrating the MEC service with a deterministic flow (e.g., a TSN flow) over the fronthaul network.

[0118] The improved MEC scheme 1000 (as described with reference to the exemplary system 1100) allows a 5G wireless engine control system (e.g., a distributed generic authority digital engine control (FADEC)) to be located within the MEC and communicate with multiple aircraft simultaneously in real time over 5G. The 5GS engine control software is fast, deterministic, and reliable by using the MEC's ​​direct integration with fronthaul and low-level I / Q data processing enabled by this improved scheme. The MEC exposes the low-level I / Q data to MEC application developers, enabling highly efficient programming similar to assembly code processing on a computer, including the ability to inspect the raw I / Q data without the overhead of converting it to bits, if necessary. Finally, the MEC application itself is converted to TSN flows when migrating from one MEC platform to another, and the migration must be completed in a fast and deterministic time, especially in unexpected emergency situations that were not pre-arranged. Resource characteristics of 5G MEC container applications

[0119] As mentioned above, 5G MEC is intended to improve real-time performance on 5G networks. However, conventional networks do not have a standard way to characterize MEC performance (e.g., processing time, system latency, processing determinism, available load, UE speed). This disclosure describes several examples of standard ways to characterize MEC performance against requested QoS.

[0120] In some implementations, the performance characteristic metric may represent, for example, processing time, system latency, available capacity, available load, UE speed, and / or processing determinism as a function of the load of the MEC processing system (e.g., the MEC system of system 600 and / or the MEC host of system 900 described above). The MEC processing system may use the performance characteristic metric as a basis for determining whether and how to manage TSN traffic flows between DUs of the network and / or MEC application processing speeds. In other words, the MEC processing system may select which DU (which MEC) to migrate an application to based on the performance characteristic metric of each of multiple candidate MECs in range of the UE. In this manner, the performance characteristic metric provides a way to determine and compare the capabilities of multiple MECs to accommodate TSN traffic corresponding to one or more MEC applications (i.e., MEC planning, handover, and transition).

[0121] The MEC performance characteristics may be used as a basis for determining which DU should be selected for handoff of an MEC application.

[0122] Performance characterization metrics may be associated with data tags of data processed by the MEC processing system. Examples of data tags include metadata describing aspects of the underlying data, elements of a YANG model, and / or a RESTful API. In some implementations, a multi-tag approach (e.g., two or more tags) may be defined for MEC resource characterization for MEC planning, handover, and transitions, including profile generation. Performance characteristics include channel conditions, including Doppler, characterizing the rate of change of location, and predicting new locations.

[0123] A first tag in the multi-tag approach may be associated with the TSN configuration and schedule of data processed in the MEC processing system. In assigning this tag, the MEC processing system may convert unlimited latency of the processed data into bounded latency. In some implementations, the MEC processing system may assign this tag with deadline-based latency requirements.

[0124] A second tag in the multi-tag approach may be associated with core resource allocation. Exemplary metrics on which this tag may be based include one or more of the following: (1) MIPS math functions (e.g., FFT, logic, multiply-and-accumulate, etc.) and / or processing-related metrics such as available memory, available processing resources, and / or duty cycle, (2) Kolmogorov complexity metrics (e.g., algorithmic information theory), (3) active network timing metrics (e.g., compression and transfer of processing algorithms from one MEC to another), and / or (4) metrics classified relative to benchmark applications (which allows metrics to be standardized across applications).

[0125] In some implementations, the multi-tag approach described above may accommodate time-varying application requirements (e.g., profile adaptation). While TSN may be configured to accommodate application and processing requirements, the MEC resource characterization scheme described herein may additionally or alternatively consider application and processing speed in response to network requirements, which is a more symmetric and dynamic relationship.

[0126] The MEC resource characterization scheme described herein enables better performance and optimization of 5G systems for real-time control applications. Thus, a 5G wireless engine control system (e.g., a distributed FADEC) may be fully located within the MEC and communicate in real-time with user equipment (e.g., automobile or aircraft) over 5G. In one example, an MEC application may comprise multiple interacting MEC processes that can rapidly interact with the user equipment to maintain efficient performance.

[0127] The UE / MEC application network topology described herein allows the MEC app to migrate at some rate (referred to as the MEC migration rate) proportional to the UE change of MEC based on the MEC performance characteristics described above (i.e., based on the performance characteristic metrics).

[0128] In some implementations, the MEC transition rate is an abstraction that consists of changes in the bandwidth, determinism, path, and location of an MEC application within a network graph that represents the link interconnection topology of the application flows.

[0129] The MEC processor must be able to support the required computing, and therefore also exposes a performance characterization metric that represents determinism as a function of the load of the MEC processing system. The determinism of the processing may decrease with the load, as the overhead required to manage the load interferes with the main calculations. Such a curve is made possible by the means already described in this disclosure (MIPS, Kolmogorov complexity, etc.), so that the 5GS can better decide how to manage the traffic flow rate (TSN) and the processing rate (MEC applications) to achieve the desired performance. In particular, based on the performance characterization metric, the 5GS can decide when to migrate MEC apps, when to throttle the traffic, when to throttle the processing rate to maintain determinism and not overflow the network, and / or when to expand the MEC processing capabilities (e.g., add more / faster processors, split the application between more processors).

[0130] Thus, in one exemplary communications system comprising a core network, a radio access network communicatively coupled to the core network, and an MEC processing system communicatively coupled to the radio access network, the MEC processing system includes one or more processors and a memory storing one or more programs executed by the one or more processors, the one or more programs including instructions for: (i) determining a performance characteristic metric representing determinism as a function of load on the MEC processing system; and (ii) adjusting a time-sensitive networking (TSN) traffic flow rate of the radio access network based on the performance characteristic metric.

[0131] In some implementations, the instructions for determining a performance characteristic metric include instructions for tagging data processed by the MEC processing system with a first tag corresponding to one or more TSN configuration and scheduling metrics, and instructions for determining the performance characteristic metric based on the first tag.

[0132] In some embodiments, the instructions for determining the performance characteristic metric include instructions for tagging the data processed by the MEC processing system with a second tag corresponding to one or more resource allocations from the core network, and instructions for determining the performance characteristic metric based on the second tag.

[0133] In some implementations, the instructions for tagging the data with the second tag include instructions for determining a MIPS, memory, processing, and / or duty cycle metric associated with the core network, and instructions for assigning the second tag based on the MIPS, memory, processing, and / or duty cycle metric.

[0134] In some implementations, the instructions for tagging the data with the second tag include instructions for determining a Kolmogorov complexity metric associated with the core network and instructions for assigning the second tag based on the Kolmogorov complexity metric.

[0135] In some implementations, the instructions for tagging the data with the second tag include instructions for determining an active network timing metric associated with the core network and instructions for assigning the second tag based on the active network timing metric.

[0136] In some implementations, the instructions for tagging the data with the second tag include instructions for determining a resource allocation metric associated with the core network, instructions for comparing the resource allocation metric to a benchmark metric, and instructions for assigning the second tag based on the comparison.

[0137] In some embodiments, the instructions for tagging the data with the first tag and / or the second tag include instructions for detecting time-varying requirements of the MEC processing system and instructions for assigning the first tag and / or the second tag based on the time-varying requirements.

[0138] In some implementations, the instructions for determining the performance characteristic metric include instructions for tagging data processed by the MEC processing system. In some implementations, the one or more programs further include instructions for executing the MEC application and instructions for determining whether to migrate the MEC application to a second MEC processing system based on the performance characteristic metric. For example, the system determines which edge-enabled device (e.g., MEC processing system) of the multiple second edge-enabled devices to migrate the application to based on a comparison of the performance characteristic metric corresponding to each of the multiple second edge-enabled devices (e.g., evaluate the performance characteristic metric of the multiple DUs 1104 in FIG. 11). In some implementations, the system assigns one of the multiple second edge-enabled devices to continue executing the application based on the determination (e.g., selects the DU 1104 with the best performance characteristic metric), migrates the application to the assigned edge-enabled device, and continues executing the application on the assigned edge-enabled device according to the TSN configuration of the assigned edge-enabled device.

[0139] In some embodiments, the one or more programs further include instructions for executing the MEC application and for adjusting a processing rate of the MEC application based on the performance characteristic metric.

[0140] In some implementations, the instructions to adjust the processing rate of the MEC application include instructions to scale or adjust MEC processing traffic.

[0141] In some embodiments, the one or more programs further include instructions for determining whether to modulate processing traffic at the MEC processing system to maintain a deterministic threshold and not overflow the radio access network based on the performance characteristic metric. others

[0142] The above description has been described with reference to specific implementations. However, the above exemplary discussion is not intended to be exhaustive or to limit the claims to the precise forms disclosed. Many variations are possible in light of the above teachings. The implementations have been selected and described in order to best explain the principles of operation and practical applications and thereby enable those skilled in the art to use them.

[0143] The various figures show a number of elements in a particular order, although elements that are not order dependent may be rearranged and other elements may be combined or separated. While some rearrangements or other groupings are specifically mentioned, other groupings will be apparent to those of ordinary skill in the art, and the rearrangements and groupings presented herein are not an exhaustive list of alternatives.

[0144] As used in this specification, the singular forms "a," "an," and "the" include the plural forms unless the context clearly dictates otherwise, the term "and / or" includes all possible combinations of one or more of the associated listed items, terms such as "first," "second," etc. are used only to distinguish one element from another and do not limit the elements themselves, the term "if" may be interpreted to mean "when," "when," "in response to," or "due to," depending on the context, and the terms "include," "including," "comprise," and "comprising" specify particular features or operations but do not exclude additional features or operations.

Claims

1. 1. A communication system comprising: A core network; (i) a radio access network comprising a central unit communicatively connected to said core network via a backhaul network and (ii) to a plurality of distributed units via a fronthaul network; each of the plurality of distributed units is co-located with a wireless unit and comprises an edge-enabled device; The edge-enabled device of each of the plurality of distributed units includes a communications data link, one or more processors, and at least one memory storing instructions that, when executed by the one or more processors, cause the edge-enabled device to: Controlling deterministic network configuration; running one or more applications over said deterministic network configuration; Determining performance characteristic metrics; and adjusting a deterministic network flow rate of the radio access network based on the performance characteristic metric.

2. the edge-enabled device of each of the plurality of distributed units is directly connected to the fronthaul network; 2. The communication system of claim 1, wherein executing the one or more applications comprises transmitting and receiving data over the fronthaul network with the deterministic network configuration.

3. 10. The communication system of claim 1, wherein executing the one or more applications comprises processing analog user data associated with a co-located wireless unit.

4. the analog user data is encoded with an in-phase / quadrature (I / Q) signal; The communication system of claim 3 , wherein the analog user data is processed by processing the I / Q signals with one or more homomorphic processing functions.

5. executing the one or more applications, executing a service on a first distributed unit of the plurality of distributed units; determining whether at least one criterion associated with the service is satisfied; The communication system of claim 1 , further comprising migrating the service to a second distributed unit of the plurality of distributed units upon determining that the at least one criterion associated with the service is satisfied.

6. Migrating the service to the second distributed unit; The communication system of claim 5 , further comprising: maintaining integration of the service with the fronthaul network while migrating the service to the second distributed unit.

7. the service is a deterministic message flow service; the edge-enabled device is a multi-access edge computing (MEC) device; The communication system of claim 1 , wherein the deterministic network configuration is a Time Sensitive Networking (TSN) configuration.

8. The one or more applications a vehicle engine control application configured to transmit and receive engine control data to and from an aircraft or automobile within a service area of ​​one or more of the plurality of distributed units; or 10. The communications system of claim 1 , further comprising: a power grid monitoring and control application configured to transmit and receive control data to and from a power grid infrastructure within a service area of ​​one or more of the plurality of distributed units.

9. The at least one criterion is a user device associated with the service moves from a service area of ​​the first distributed unit to a service area of ​​the second distributed unit; the second distributed unit being closer to the user device associated with the service than the first distributed unit, regardless of whether the user device is mobile; the radio channel conditions at the second distributed unit are better than the radio channel conditions at the first distributed unit; or The communication system of claim 6, further comprising at least one of: the first distribution unit being overloaded.

10. The communications system of claim 6, wherein the services are migrated according to a migration rate that represents a migration rate proportional to a change in user devices associated with the services based on the performance characteristic metrics.

11. The communications system of claim 10, wherein the migration rate is a function of changes in bandwidth, determinism, path, and location of the one or more applications within the radio access network according to a network interconnection topology of an application flow.

12. The communications system of claim 1, wherein the performance characteristic metric represents at least one of processing time, system delay, available capacity, available load, user equipment speed, or processing determinism as a function of load on the edge-enabled device.

13. The performance characteristic metric, one or more deterministic network configuration and scheduling metrics; one or more resource allocations from the core network; limited delay in processing data; MIPS, memory, processing, and / or duty cycle metrics associated with said core network; a Kolmogorov complexity metric associated with the core network; an active network timing metric associated with the core network; (i) a comparison of a resource allocation metric associated with said core network with (ii) a benchmark metric; the time-varying requirements of said edge-enabled device; or and data processed by the edge-enabled device.

14. The operation Running the application; and 2. The communication system of claim 1, further comprising: determining a second edge-enabled device among the plurality of second edge-enabled devices to which to migrate the application based on a comparison of performance characteristic metrics corresponding to each of the plurality of second edge-enabled devices.

15. The operation Allocating the second edge-enabled device to continue execution of the application; and The communication system of claim 14 , further comprising: executing the application on the assigned second edge-enabled device via a deterministic network configuration of the assigned second edge-enabled device.

16. The operation Running the application; and The communication system of claim 1 , further comprising: adjusting a processing rate of the application based on the performance characteristic metric.

17. The communications system of claim 16, wherein adjusting the processing speed of the application includes expanding or throttling processing traffic associated with the edge-enabled device.

18. The operation 2. The communication system of claim 1, further comprising: determining whether to modulate traffic handled at the edge-enabled device based on the performance characteristic metric to maintain a deterministic threshold and not overflow the radio access network.

19. A distributed unit in a communication system, comprising: an edge-enabled device including a communications data link, one or more processors, and at least one memory that stores instructions; The communication system includes: A core network and a radio access network, the radio access network comprises: (i) a central unit communicatively connected to the core network via a backhaul network; and (ii) a central unit communicatively connected to a plurality of distributed units, including the distributed unit, via a fronthaul network; The instructions, when executed by the one or more processors, cause the edge-enabled device to: Controlling deterministic network configuration; running one or more applications over said deterministic network configuration; Determining performance characteristic metrics; and and adjusting a deterministic network flow rate of the radio access network based on the performance characteristic metric.

20. A method performed by a device in a radio access network of a communication system, comprising: Controlling deterministic network configuration; running one or more applications over said deterministic network configuration; Determining performance characteristic metrics; and adjusting a deterministic network flow rate of the radio access network based on the performance characteristic metric.

Images