Isolating time synchronization traffic using virtualization

Abstract

This disclosure describes techniques for customer isolation using virtualization to provide time synchronization services. For example, one method includes: receiving, by a computing device, Internet Protocol (IP) addresses of customer networks among multiple customer networks connected to a cloud exchange performed by the computing device; configuring, by the computing device, a time synchronization server connected to the cloud exchange using a virtualized local area network (VLAN) associated with the IP addresses of the customer networks, the time synchronization server including multiple instances providing time synchronization services; and configuring, by the computing device, the time synchronization server using Virtual Routing and Forwarding (VRF) or network namespaces for VLANs, wherein the VRF or network namespace includes routes for sending time synchronization services between the customer networks and specific instances among the multiple instances providing time synchronization services.

Description

[0001] This application claims the benefit of U.S. Provisional Patent Application No. 62 / 994,277, filed on March 24, 2020, the contents of which are incorporated herein by reference in their entirety. Technical Field

[0002] This disclosure involves time synchronization. Background Technology

[0003] Typically, a synchronization system synchronizes the clocks of multiple devices based on the clock of a selected master device (also known as a "primary" or "leader" device). A master device is a computing device that obtains time synchronization data from other master devices or intelligent engines deployed internally or externally to a synchronization system such as GPS. A typical synchronization system has one layer of master devices, where other terminal devices (also known as "slave," "copy," or "follower" devices) are connected to at least one master device. The master device is connected to a more precise timestamp source. In some implementations, the master device may belong to a service provider and may be designed to support precise time processing, while the terminal devices may belong to service clients and receive timestamp offsets from the master device. These master and terminal devices can be arbitrarily selected or assigned by the network.

[0004] Time synchronization can be beneficial in many applications, including but not limited to the financial, scientific, military, and gaming industries. For example, this knowledge would be used to define trading orders in high-frequency trading systems and player responses in multi-user games. Summary of the Invention

[0005] Generally, this disclosure describes techniques for client isolation when using virtualization to provide time synchronization services. For example, end devices can access time synchronization services using a switch such as a cloud-based service exchange (referred to herein as a "cloud exchange"). For instance, the cloud exchange provides a resilient and independent cloud-based service exchange through which cloud-based service clients ("cloud clients" or simply "clients") and / or cloud-based service providers ("cloud providers") connect to receive and provide cloud services, respectively. One such cloud service may include a time synchronization service provided by a time synchronization protocol such as Network Time Protocol (NTP) or Precision Time Protocol (PTP) to synchronize the clocks of devices throughout a computer network. As an example, time synchronization protocols such as NTP or PTP describe a master computing device (e.g., a time synchronization server) that acts as a reference clock (e.g., using the master computing device's hardware clock) to provide client computing devices with a reference timing signal to synchronize their system time with the master computing device's system time.

[0006] According to the technology described in this invention, a single master computing device (e.g., a time synchronization server (e.g., a PTP master)) is configured with multiple instances for time synchronization services for multiple client computing devices (including those where the clients are different client entities), regardless of whether the multiple clients have overlapping IP addresses.

[0007] As an example, this disclosure describes a method comprising: a programmable network platform executed by a computing device receiving Internet Protocol (IP) addresses of customer networks connected to a plurality of customer networks of a cloud exchange managed by the programmable network platform; configuring a time synchronization server connected to the cloud exchange by the programmable network platform using a Virtualized Local Area Network (VLAN) associated with the IP address of the customer network, the time synchronization server comprising a plurality of instances providing time synchronization services; and configuring the time synchronization server by the programmable network platform using Virtual Routing and Forwarding (VRF) or network namespaces for the VLANs, wherein the VRF or network namespaces include routes for sending time synchronization services between the customer networks and specific instances of the plurality of instances providing the time synchronization services.

[0008] As another example, this disclosure describes a computing device including: one or more computer processors; and a memory including instructions that, when executed by the one or more computer processors, cause the one or more computer processors to: receive Internet Protocol (IP) addresses of customer networks among a plurality of customer networks connected to a cloud exchange; configure a time synchronization server connected to the cloud exchange using a Virtualized Local Area Network (VLAN) associated with the IP address of the customer network, the time synchronization server including a plurality of instances providing time synchronization services; and configure the time synchronization server using a network namespace of Virtual Routing and Forwarding (VRF) or VLANs, wherein the VRF includes routes for sending time synchronization services between the customer networks and specific instances among the plurality of instances providing time synchronization services.

[0009] As another example, this disclosure describes an interconnection system including: a plurality of customer networks connected to a cloud exchange; a time synchronization server connected to the cloud exchange, the time synchronization server including: a plurality of instances providing time synchronization services; a virtualized local area network (VLAN) associated with an Internet Protocol (IP) address of a particular customer network among the plurality of customer networks; and a virtual routing and forwarding (VRF) or network namespace for the VLAN, wherein the VRF or network namespace includes routes for sending time synchronization services between the particular customer network and the particular instance among the plurality of instances providing the time synchronization services.

[0010] In this way, a single master computing device (e.g., a PTP master computing device) can act as a reference clock to provide reference timing signals to multiple client computing devices, offering a higher level of timing accuracy compared to multiple connections to multiple master computing devices (e.g., by using a single hardware clock). Furthermore, by configuring VLANs and VRFs or network namespaces for each of the multiple customers, customers can connect to a single master computing device regardless of whether they have overlapping IP addresses, providing greater scalability.

[0011] Details of one or more examples of this disclosure are set forth in the accompanying drawings and the following description. Other features, objects, and advantages of this disclosure will be apparent from the description and drawings, as well as from the claims. Attached Figure Description

[0012] Figure 1 This is a block diagram of an example network system that uses cloud switching to provide time synchronization services based on one or more technologies described in this disclosure.

[0013] Figure 2 The invention is described in more detail according to one or more techniques. Figure 1 A block diagram of a network system.

[0014] Figure 3A This is a block diagram illustrating an example of a time synchronization server that further details one or more aspects of the technology described in this invention.

[0015] Figure 3B This is a block diagram that further details another example of a time synchronization server according to one or more aspects of the technology described in this invention.

[0016] Figure 4 This is a conceptual diagram illustrating a redundant, precise timing system according to one or more techniques of the present invention.

[0017] Figure 5 This is a block diagram showing a more detailed view of a time synchronization server configured to perform one or more techniques according to the present invention.

[0018] Figure 6 This is a flowchart illustrating an example operation of using cloud exchange to provide time synchronization services according to the technology described in this disclosure. Detailed Implementation

[0019] Generally, the techniques described herein are techniques for using time synchronization services via exchange.

[0020] For the purposes of this disclosure, a time synchronization system or system refers to a complex set of devices, algorithms, programs, modules, and components that allow time synchronization operations to be performed.

[0021] For the purposes of this invention, the device clock indicates the internal clock of the device. The device may have a single device clock or more than one device clock, including one or more of a network interface card (NIC) clock, a graphics processing unit (GPU) clock, a central processing unit (CPU) clock, or other clocks.

[0022] For the purposes of this invention, the system clock indicates the clock associated with the time synchronization system. The system clock can be a high-precision clock that provides an accurate time signal and produces an accurate timestamp, such as the clock on a GPS device. The time synchronization system may have more than one system clock.

[0023] For the purposes of this disclosure, a timestamp indicates a separate time signal measurement recorded by a time measuring device. For the purposes of this disclosure, a device timestamp indicates a timestamp generated by the device. For the purposes of this disclosure, a system timestamp indicates a timestamp calculated by a time synchronization system. For the purposes of this disclosure, a timestamp offset (also referred to herein as "time synchronization offset" and "offset") indicates the difference between two timestamps. For example, a timestamp offset can be calculated as the difference between a device timestamp and a system timestamp.

[0024] For the purposes of this disclosure, Network Time Protocol (NTP) is a network protocol for synchronizing clocks between computer systems on packet-switched, variable-delay data networks. For the purposes of this disclosure, Precision Time Protocol (PTP) is a protocol for synchronizing clocks throughout a computer network.

[0025] For the purposes of this disclosure, a switch refers to a computer network device that connects devices on a computer network using packet switching to receive, process, and forward data to a destination device. For the purposes of this disclosure, a PTP switch refers to a switch that supports the PTP standard protocol and allows the receiving, processing, and forwarding of timestamps to a destination device. For the purposes of this disclosure, an NTP switch refers to a switch that supports the NTP standard protocol and allows the receiving, processing, and forwarding of timestamps to a destination device. For the purposes of this invention, a master switch refers to a special device that allows the receiving, processing, and forwarding of time signals from a GPS antenna. For the purposes of this disclosure, a system status indicator set of parameters allows estimation of how the system is loaded and "ready for time synchronization."

[0026] For the purposes of this disclosure, a master computing device (also referred to as a “master device,” “primary device,” or “leader” device) indicates a specific type of device that obtains time synchronization data from other master devices or intelligent engines deployed within a synchronization system, such as a GPS module communicating with GPS. For the purposes of this invention, a terminal device (also referred to as a “slave” device, a “copy” device, or a “follower” device, all of which may be used interchangeably in this invention) indicates the name of all non-master devices that are typically connected to one or more master devices.

[0027] Modern technologies leverage virtualized resources, distributed systems, and edge computing, each of which increases the need for precise timing to realize performance benefits. Two main vectors currently drive the need for sub-microsecond timing. The first vector is the requirement of new regulations and industry standards for greater accuracy, reliability, resilience, transparency, and traceability for time and synchronization. These standards include the Markets in Financial Instruments Directive II – European Securities and Markets Authority (“MIFIDII-ESMA”, Finance), G.8275 – International Telecommunication Union (“ITU”, Telecommunications), Uniform Audit Trail – Securities and Exchange Commission Rule 613 (“CAT-SEC”, Finance), and C37.238-2011 – Institute of Electrical and Electronics Engineers (“IEEE”, Power).

[0028] The second vector driving demand is the number of emerging applications, business models, and use cases requiring more stringent timing and synchronization as fundamental components. These include Long Term Evolution (LTE), advanced Pro and 5G frequency and phase synchronization (telecom), globally distributed databases using timestamps for strict consistency (enterprise systems), distributed ledgers (blockchain) and high-frequency transaction granular regulatory record chains (financial and enterprise systems), transportation or logistics synchronization for security-critical applications (enterprise systems, such as drones and autonomous vehicles), synchronization and integration of industrial automation or IoT components (enterprise systems), and national preparedness missions for critical infrastructure protection and feasible backups of GPS (federal).

[0029] Highly precise timing is a fundamental component for high-frequency trading to create trusted records and for emerging applications such as globally distributed databases, distributed ledgers, and autonomous vehicles. Using the techniques described in this paper, data centers or clusters can achieve sub-microsecond precision and be easily accessed and consumed by customer applications on a global scale, at the intersection of networks and the cloud.

[0030] An example implementation of the technology described herein takes place within one or more data centers. A data center can house multiple different types of devices, each owned by a different entity, but each device can be linked throughout the data center architecture. Some devices within a data center can exchange data with other devices in the same data center, but can also exchange data with devices in data centers located in different geographic or metropolitan areas. Thus, the data center can implement the technology described herein to effectively and accurately synchronize clocks on devices within the same or different data centers. The technology described herein can be implemented as a downloadable software plugin on client devices in the data center, enabling devices in the data center to perform the time synchronization process described herein. The technology described herein balances several characteristics, including scalability, versatility, accuracy, cost, security, and redundancy.

[0031] Figure 1 This is a conceptual diagram of an example interconnect system 100 that provides time synchronization services using one or more technologies described in this disclosure, such as cloud switching. The interconnect system 100 provides a geographically distributed, Precise Timing Service (PTS) solution that can be scaled locally, regionally, or globally. When utilizing delivery mechanisms including White Rabbit, PTP (Precise Time Protocol), NTP (Network Time Protocol), software clients, and IP or Ethernet distribution via E-sync mechanisms, the design can be built on precise clocks and advanced algorithms to deliver microsecond-level precision, and in some cases, nanosecond-level precision.

[0032] Figure 1 Cloud exchange 102 provides a flexible and independent exchange of cloud-based services, through which cloud-based service customers (“cloud customers”) and / or cloud-based service providers (“cloud providers”) connect to receive and provide cloud services, respectively. Cloud exchange 102 provides secure, private, virtual connections to multiple cloud service providers (CSPs) globally to the exchange’s customers (e.g., enterprises, network operators, network service providers, and software-as-a-service (SaaS) customers). Multiple CSPs participate in cloud exchange by having at least one accessible port in the cloud exchange, through which customers can connect to one or more cloud services provided by the CSPs, respectively. Cloud exchange 102 allows any customer’s private network to directly cross-connect to any other customer at a public point, thereby allowing direct exchange of network services between customer networks.

[0033] In this example, cloud client network 104A operating computing devices 108A1-108AN (collectively referred to as "computing devices 108A") and cloud client network 104B operating computing devices 108B1-108BN (collectively referred to as "computing devices 108B") connect to cloud exchange 102 to receive services, such as time synchronization services provided by time synchronization server 110. Cloud exchange 102 can provide one or more cloud exchange points (such as...) Figure 2 As further described herein, each cloud exchange point represents, for example, a data center geographically located within the same metropolitan area (“metro-based”, such as Silicon Valley, California; New York City, New York City; Dallas, Texas; Chicago, Illinois; etc.). As used herein, references to “cloud exchange” or “cloud-based service exchange” may refer to a cloud exchange point. A cloud exchange provider may deploy instances of cloud exchange 102 in multiple different metropolitan areas, each instance of cloud exchange 102 having one or more cloud exchange points.

[0034] Cloud switch 102 includes network infrastructure and an operating environment through which computing devices 108A of customer network 104A and 108B of customer network 104B receive time synchronization services provided by time synchronization server 110. In this example, the network infrastructure of cloud switch 102 may include one or more network devices, such as network device 114, to provide customer networks 104A and 104B (collectively referred to as "customer network 104" or "customer 104") with access to the time synchronization services provided by time synchronization server 110. Network device 114 may represent a router or switch that supports time synchronization protocols such as PTP or NTP, allowing the reception, processing, and forwarding of timestamps to destination devices.

[0035] Computing devices 108A and 108B (collectively, “Computing Devices 108”) may be computing devices located within the respective customer networks of a data center in one of the cloud exchange points, or via a transit network service provider (NSP). Figure 1 (Not shown) A computing device in a client network receiving services. In this example, computing device 108A can access cloud switch 102 via network device 116A. Similarly, computing device 108B can access cloud switch 102 via network device 116B. Network devices 116A and 116B can represent edge network devices, such as routers or switches.

[0036] exist Figure 1In the example, cloud exchange 102 includes a programmable network platform 120, which is used to dynamically program cloud exchange 102 to responsively and reliably meet service requests encapsulated with business requirements for the services provided by cloud exchange 102. One such service includes a time synchronization service provided by time synchronization server 110, which is coupled to (or co-located with) cloud exchange 102. As a result, programmable network platform 120 can orchestrate business-level services to time synchronization server 110 based on well-defined service policies, quality of service policies, service level agreements, and costs, and further, based on the service topology used for business-level services.

[0037] The programmable network platform 120 enables cloud service providers managing the cloud exchange 102 to dynamically configure and manage it, for example, to facilitate virtual connections for service delivery from the time synchronization server 110 to one or more client devices 108. The cloud exchange 102 allows customers to bypass the public internet to connect directly to the time synchronization server 110, thereby improving performance, reducing costs, increasing connection security and privacy, and leveraging cloud computing for additional applications. Thus, for example, enterprises, network operators, and SaaS customers can integrate cloud services such as time synchronization services with their on-premises applications, at least in some respects, as if these services were part of or directly coupled to their own data center network.

[0038] Programmable network platform 120 may represent an application running within one or more data centers of cloud exchange 102, or alternatively, an application running off-site in a cloud provider's (e.g., a back office or branch office). Programmable network platform 120 may be distributed wholly or partially across data centers, each associated with a different cloud exchange point to form cloud exchange 102. Although shown as managing a single cloud exchange 102, programmable network platform 120 can control service provisioning for multiple different cloud exchanges. Alternatively or additionally, multiple individual instances of programmable network platform 120 can control service provisioning for corresponding multiple different cloud exchanges.

[0039] exist Figure 1In the example, cloud switch 102 provides client 104 with access to a time synchronization service that allows for precise and accurate time synchronization with devices throughout the computer network. The time synchronization service can support both NTP and PTP protocols. Further information about NTP is provided in “Network Time Protocol Version 4: Protocol and Algorithm Specification,” RFC 5905, Internet Engineering Task Force (IETF), June 2010, available at https: / / tools.ietf.org / html / rfc5905, the entire contents of which are incorporated herein by reference. Further information about PTP is provided in “1588-2008 - IEEE Standard for Precision Clock Synchronization Protocol for Networked Measurement and Control Systems” dated July 24, 2008, and in the “Precision Time Protocol Version 2 (PTPv2) Management Information Base”, RFC 8173, IETF, June 2017, available at https: / / tools.ietf.org / html / rfc8173, the entire contents of which are incorporated herein by reference.

[0040] In these and other examples, the time synchronization service can be configured by a programmable network platform 120 of a cloud exchange (e.g., cloud exchange 102). An example of a time synchronization service in a cloud exchange system is provided in U.S. Application No. 16 / 438,310, filed June 11, 2019, entitled “Timed Synchronization Service and Distribution System,” the entire contents of which are incorporated herein by reference.

[0041] In industry, there are certain challenges in providing accurate and precise time synchronization in a scalable manner. Typically, without the techniques described in this invention, the host computing device may implement, for example, a time synchronization protocol in unicast mode, where each client computing device is configured with the host computing device's IP address, allowing each client computing device to connect to the host computing device individually. However, in these examples, a single instance of the time synchronization server is provided to another time synchronization server. If the time synchronization instance fails, every connection to the host computing device also fails.

[0042] In other examples, the master computing device can implement a time synchronization protocol in multicast mode, where it can multicast time synchronization messages (e.g., timestamps) to multiple client devices. However, in some examples, if the client subnets have overlapping IP addresses (e.g., one or more client computing devices in one subnet have the same IP address as one or more client computing devices in another subnet), the master computing device may not be able to accurately provide reference timing signals to the client computing devices.

[0043] In some examples, client computing devices can connect to one of multiple master computing devices through a process that discovers the master computing device using, for example, an optimal master clock algorithm. However, in these examples, multiple client computing devices may use different time synchronization servers that use different hardware clocks, resulting in lower accuracy across different client networks.

[0044] According to the technology of the present invention, a single main computing device (e.g., time synchronization server 110) is configured with multiple instances of time synchronization service, wherein the time synchronization service of each instance is isolated from the time synchronization service of other instances using virtualization.

[0045] When customer network 104 is onboard (i.e., connected) to cloud exchange 102, programmable network platform 120 can receive an IP address from at least one customer network 104. This IP address will be used to receive time synchronization services provided by time synchronization server 110. As an example, the administrator of customer network 104A can provide programmable network platform 120 with IP address 10.10.20.20 / 24 for customer network 104A. Similarly, the administrator of customer network 104B can provide programmable network platform 120 with IP address 10.10.20.20 / 24 for customer network 104B.

[0046] In response to receiving an IP address from customer network 104, programmable network platform 120 can configure time synchronization server 110 using VLANs for IP addresses specific to a customer network. For example, programmable network platform 120 can configure VLAN 200 for IP addresses of customer network 104A and VLAN 201 for IP addresses of customer network 104B. In this way, each VLAN can provide connectivity to its corresponding customer network while maintaining service isolation from other customer networks.

[0047] The programmable network platform 120 can configure Virtual Router and Forwarder (VRF) instances (or, in some examples, network namespaces) for specific VLANs. For example, the programmable network platform 120 can configure VRF 112A associated with VLAN 200 and VRF 112B associated with VLAN 201. Each of VRF 112A and VRF 112B (collectively, “VRF 112”) includes one or more routes to send time synchronization services (such as PTP or NTP messages) for its associated VLAN. For example, a VRF can send time synchronization services for a VLAN associated with a specific instance of a specific customer network and time synchronization service.

[0048] For example, the programmable network platform 120 can configure timing procedures (e.g., timing daemons such as a PTP daemon) executed by the time synchronization server 110 to provide a specific instance of time synchronization service to a corresponding customer network. Each timing procedure (e.g., a daemon) can use the same hardware clock of the time synchronization server 110 (e.g., the hardware clock of the NIC). As further described below, the VRF 112A may include a first timing procedure for providing a first instance of time synchronization service (e.g., a timing process for a first instance of time synchronization service). Figure 3A The VRF 112B may include a route for sending time synchronization services between the time synchronization daemon 304A and the customer network 104A. Similarly, the VRF 112B may include a second timing procedure (e.g., for a second instance providing time synchronization services) for sending time synchronization services between the time synchronization daemon 304A and the customer network 104A. Figure 3A The time synchronization service route is sent between the timed daemon 304B and the customer network 104B.

[0049] In this way, by configuring a single master computing device (e.g., time synchronization server 110) using multiple time synchronization service instances, multiple customer networks can use this single master computing device as a reference clock (e.g., by utilizing a single hardware clock) to provide reference timing signals to multiple customer computing devices within the customer network. This provides a higher level of timing accuracy compared to multiple connections to multiple master computing devices (each with its own hardware clock and therefore different timing signals). Furthermore, by configuring VLANs and VRFs (or network namespaces) for each of the multiple customer networks, customer networks can connect to a single master computing device regardless of whether the customer networks have overlapping IP addresses. This single master computing device essentially enables the time synchronization service to operate in unicast mode, which provides even greater accuracy and scalability.

[0050] Figure 2 To show in more detail Figure 1 A block diagram of the cloud exchange of the interconnected system 100. Figure 2 In the example, colocation facility 200 can represent Figure 1 An example implementation of cloud exchange 102. Interconnection system 100 depicts an exchange point 128 within a hosting facility 200, such as a data center or warehouse, which, for example, provides access to time synchronization services provided by time synchronization server 202 to customer networks 104A and 104B. Although Figure 2 The hosting facility 200 is illustrated as having a single exchange point, but the hosting facility 200 may include multiple exchange points.

[0051] As used herein, the term "customer" can include a tenant of a hosting facility deployed by a hosting facility provider, from which the customer purchases one or more of hosting, interconnection, and / or other networking services. In some examples, a customer receives such services by hosting at hosting facility 200, such as renting space and / or electricity to access services offered at hosting facility 200. In other examples, a customer may not be a physical tenant of the hosting facility, but may be virtually connected to a cloud exchange without requiring a physical presence at hosting facility 200.

[0052] Within hosting facility 200, space and power can be flexibly partitioned and leased to customer network 104 in increments in the form of cages (areas of public space enclosed by fences or other boundaries), cabinets, racks, suites (enclosed rooms not part of the public space), or other spaces where customers can locate their network equipment to provide and / or receive network services to / from other customers(s) hosted in hosting facility 200. Customer network 104 may lease space within hosting facility 200 for co-hosting with other tenants to improve efficiency on a standalone facility and to interconnect network equipment with the network equipment of other tenants / customers within hosting facility 200 or on campus to reduce latency / jitter and improve reliability, performance, and security relative to the transport network. Hosting facility 200 can host numerous customers, such as customer network 104 and its network, server, and / or storage equipment. Each customer network 104 may have specific reasons for choosing to host at the hosting facility 200, including capacity, geographical proximity, connectivity to other customers, hosting with other customers, and price. Although Figure 2 Two clients are shown, but interconnection system 100 may include one or more additional clients hosted in (or virtually connected to) hosting facility 200 to receive network services such as time synchronization services from hosting facility 200 provider.

[0053] Cloud customers operating computing device 108 can receive cloud-based services directly via a Layer 3 peering and physical connection to cloud exchange point 128, or via a network service provider (ISP). Figure 2One of the cloud-based services (not shown in the image) (also referred to as an "operator") indirectly receives cloud-based services. The NSP provides "cloud relay" by maintaining a physical presence within one or more cloud exchange points and aggregating Layer 3 access from one or more device clients 108. The NSP can peer directly at Layer 3 with cloud exchange point 128 of hosting facility 200, and in doing so, provides indirect Layer 3 connectivity and peering to one or more client devices 108 through which clients (e.g., operating client devices 108) can obtain cloud services from hosting facility 200. Each cloud exchange point of hosting facility 200 is assigned a different Autonomous System Number (ASN). Therefore, each cloud exchange point is the next hop in the path vector routing protocol (e.g., BGP) path from the cloud service provider to client device 108. As a result, although each cloud exchange point is not a relay network with one or more WAN links and accompanying Internet access and relay policies, each cloud exchange point can peer with multiple different autonomous systems via external BGP (eBGP) or other external gateway routing protocols to exchange, aggregate, and route service traffic from one or more cloud service providers to customers. In other words, cloud exchange points can internalize the eBGP peering relationships that cloud service providers and customers maintain on a pairwise basis. Conversely, customers can leverage cloud exchange points to configure a single eBGP peering relationship and receive multiple cloud services from one or more cloud service providers via the cloud exchange point. While this paper primarily describes eBGP or other Layer 3 routing protocol peering between cloud exchange points and customer, NSP, or cloud service provider networks, cloud exchange points can learn routes from these networks in other ways, such as through static configuration or via Routing Information Protocol (RIP), Open Shortest Path First (OSPF), Intermediate System to Intermediate System (IS-IS), or other routing distribution protocols.

[0054] As an example above, a customer may contract with a cloud exchange provider for hosting facility 200 to directly access Layer 3 cloud services via the cloud exchange point. In this way, the customer receives redundant Layer 3 connectivity to the cloud service provider. This contract is exemplified in the cloud exchange point's network infrastructure through L3 peering configurations within the NSP and the cloud exchange point's switching equipment, and L3 connections (e.g., Layer 3 virtual circuits) established within the cloud exchange point to interconnect the cloud service provider's network to the NSP and the customer's network, all having at least one port providing connectivity within one or more cloud exchange points.

[0055] In some examples, hosting facility 200 allows corresponding customers of any NSP and other cloud customers, including customers operating customer equipment, to be directly connected to any other customer network and / or any CSP via a virtual Layer 2 (L2) or Layer 3 (L3) connection, thereby allowing direct exchange of network traffic between the customer network and the CSP. A virtual L2 or L3 connection may be referred to as a “virtual circuit.”

[0056] Each NSP can represent a network service provider associated with a transit network through which NSP subscribers can access cloud services provided by the CSP via hosting facility 200. Typically, CSP customers can include network operators, large enterprises, managed service providers (MSPs), and customers of Software as a Service (SaaS), Platform as a PaaS, Infrastructure as a Infrastructure (IAAS), Virtualization as a Virtualization (VAAS), and Data Storage as a Data Storage (DSAAS) services used by the CSP to provide such cloud-based services via hosting facility 200.

[0057] In this way, the hosting facility 200 streamlines and simplifies the process of enabling CSPs and customers to collaborate (either via the operator NSP or directly) in a transparent and neutral manner. An example application is a hosting and interconnect data center where CSPs and NSPs and / or customers operating customer equipment 108 may already have a network presence, for example, by having one or more accessible ports within the data center available for interconnection, which can represent a cloud exchange point. This allows participating NSPs, customers, and CSPs to have a wide range of interconnection options within the same facility. In this way, NSPs / customers can have the option to create many-to-many interconnects, where they are hooked to one or more cloud exchange points only once. In other words, the hosting facility 200 allows customers to interconnect to multiple CSPs and cloud services, rather than having to establish separate connections across a transit network to access different cloud service providers or different cloud services from one or more cloud service providers.

[0058] Hosting facility 200 may provide one or more different types of interconnection services via network devices in the network infrastructure between customer networks 104 hosted within hosting facility 200. For example, hosting facility 200 may provide physical or "Layer-1" (in the Open Systems Interconnection model (OSI model)) interconnections between tenants of hosting facility 200. Physical interconnections may include, for example, physical cross-connections established by Category 5 or Category 6 (CAT5 / 6) cables, coaxial cables, and / or fiber optic cables. In some examples, hosting facility 200 may provide data link or "Layer-2" (in the OSI model) interconnections between tenants of hosting facility 200. In some examples, hosting facility 200 providing Layer-2 interconnections may be referred to as Ethernet switching, where Ethernet is the underlying Layer 2 protocol. In some examples, hosting facility 200 may provide network and / or transport or "Layer-3 / 4" (in the OSI model) interconnections between tenants of hosting facility 200. In some examples, hosting facility 200 may provide Layer 3 / 4 interconnect (referred to as Internet switching), where TCP / IP is the underlying Layer 3 / 4 protocol. For example, hosting facility 200 may provide Internet switching to allow routers (e.g., those of the tenants of hosting facility 200) to... Figure 1Network device 116) uses a Layer 3 routing protocol, such as a Border Gateway Protocol, to directly peer with other tenants of hosting facility 200 to exchange routes that facilitate Layer 3 service exchange, thereby providing private peering. In some examples, hosting facility 200 may provide indirect Layer 3 routing protocol peering, whereby each customer network 104 advertises its Layer 3 routes to an Autonomous System (AS) deployed by the hosting facility provider within the hosting facility's network infrastructure to provide AS-mediated private peering. The AS may then typically relay these routes in conjunction with tunneling or other forwarding mechanisms to establish a connection to customer network 104. In some examples, hosting facility 200 may provide indirect Layer 3 routing protocol peering to facilitate service service exchange (referred to as cloud-based service exchange, or more simply cloud exchange). Additional descriptions of the exchange can be found in U.S. Patent No. 9,948,552, filed April 14, 2016, entitled “CLOUD-BASED SERVICES EXCHANGE,” and U.S. Patent No. 10,015,268, filed January 20, 2016, entitled “MULTI-CLOUD, MULTI-SERVICE DATA MODEL,” the entire contents of which are incorporated herein by reference. Additional descriptions of interconnection services provided by the exchange can also be found in U.S. Patent No. 9,886,267, filed October 29, 2015, entitled “INTERCONNECTION PLATFORM FOR REAL-TIME CONFIGURATION AND MANAGEMENT OF A CLOUD-BASED SERVICES EXCHANGE,” the entire contents of which are incorporated herein by reference. As used herein, interconnection is an example of network services provided by network devices of a network infrastructure.

[0059] exist Figure 2 In the example, time synchronization server 202 can provide time synchronization services, such as according to NTP or PTP. Figure 2 Time synchronization server 202 can represent Figure 1 An example of time synchronization server 110. In this example, time synchronization server 202 is hosted in hosting facility 200, but in some examples, time synchronization server 202 may be located outside hosting facility 200, which is connected to a service provider (e.g., network service provider or operator) hosted in hosting facility 200.

[0060] exist Figure 2In one example, switching point 128 includes network infrastructure and an operating environment through which customer network 104 receives network services, such as time synchronization services provided by time synchronization server 202. In some examples, the network infrastructure of switching point 128 may include network devices such as routers and / or switches (e.g., switch 226) to provide customer network 104 with access to the time synchronization services provided by time synchronization server 202. Switch 226 may represent... Figure 1 An example implementation of network device 114 is provided. The number of network devices within switching point 128 is shown as a simplified example and can include any number of network devices to provide customer network 104 with access to the time synchronization service of interconnected system 100. Each network device includes multiple resources and features to provide access to the time synchronization service. For example, switch 226 may include network device resources for implementing features provided by the switch, including a control plane central processing unit (“CPU”), a data plane CPU, control plane memory, data plane memory, ports, line cards, and other network device resources. Network device features may include logical components that provide support for network services and utilize hardware-based network device resources to implement network services. For example, network device features may include Virtual LAN (VLAN) support and other features supported by the network device to implement operator-configured network services (directly or via configuration devices such as a Software-Defined Networking (SDN) controller).

[0061] In some examples, a customer can request time synchronization services from a hosting facility 200 provider via a programmable network platform 120 (“PNP 120”). As described above, PNP 120 may represent an application running within one or more data centers of interconnected system 100, or alternatively, an application running off-site / remotely at a hosting facility provider’s back office or branch office. Although shown as managing a single hosting facility 200, programmable network platform 120 can control service provisioning for multiple different hosting facilities. Alternatively or additionally, multiple individual instances of programmable network platform 120 can control service provisioning for corresponding multiple different hosting facilities. Programmable network platform 120 may include, for example, a service interface 214 (or “service application programming interface (API)”) that can exchange information with applications(multiple) 230 to receive service requests. Service interface 214 defines methods, fields, and / or other software primitives that applications 230 can invoke on programmable network platform 120.

[0062] (Multiple) applications 230 represent at least one application that communicates with PNP 120 to request time synchronization services for a customer network. Application 230 represents client-side software that interfaces with PNP 120 and may include a customer portal, customer applications, and / or a console, such as a command-line interface or a graphical user interface. Users or customers of (multiple) applications 230 may include customer network 104, such as enterprise customers, cloud service and content providers, carriers, network service providers (NSPs), or other customers of hosting facility 200. Users of (multiple) applications 230 may also include operators / administrators of the hosting facility 200 provider. In some examples, (multiple) applications 230 and PNP 120 may represent different functions or modules of the same application.

[0063] In some examples, an administrator 204 of customer network 104A (such as an operator or software agent) can provide the IP address of customer network 104A to the programmable network platform 120 via service interface 214. Similarly, an administrator 206 of customer network 104B can provide the IP address of customer network 104B to the programmable network platform 120 via service interface 214. In some examples, a single administrator can provide the IP address of different customer networks (e.g., such as...). Figure 4 As further described, different IP addresses are provided for the primary customer network and the backup customer network.

[0064] In response to receiving an IP address from customer network 104A, programmable network platform 120 configures VLAN 200 on time synchronization server 202, associated with the subnet IP address of customer network 104A. Similarly, in response to receiving an IP address from customer network 104B, programmable network platform 120 also configures VLAN 201 on time synchronization server 202, associated with the subnet IP address of customer network 104B. Programmable network platform 120 can assign VLAN 200 and VLAN 201 to a single physical port of time synchronization server 202 (referred to as a "VLAN trunk"). To distinguish services on each VLAN, time synchronization services are labeled, for example, with a VLAN ID associated with the corresponding VLAN. For example, when sending a time synchronization service (e.g., a timestamp) to customer network 104A, time synchronization server 202 can add the VLAN ID associated with VLAN 200 to the time synchronization service. Similarly, when sending a time synchronization service to customer network 104B, time synchronization server 202 can add the VLAN ID associated with VLAN 201 to the time synchronization service.

[0065] In some examples, switch 226 can receive time-synchronized services from at least one customer device 108A in customer network 104A and add (or switch) the VLAN ID associated with VLAN 200 such as Figure 1 The existing VLAN ID of VLAN 100 is added to the time synchronization service to send the time synchronization service to a specific instance of the time synchronization service (e.g., the timing daemon 304A). Similarly, switch 226 can receive time synchronization services from at least one of the customer devices 108B in the customer network 104B and add (or switch) the VLAN ID associated with VLAN 201 to (or exchange) such VLAN IDs. Figure 1 The existing VLAN ID of VLAN 101 is used to send the time synchronization service to a specific instance of the time synchronization service (e.g., the timed daemon 304B).

[0066] The programmable network platform 120 can also configure a Virtual Routing and Forwarding (VRF) instance or network namespace for each VLAN to route time synchronization services between a specific customer network and a specific instance providing time synchronization services by the time synchronization server 202. For example, the programmable network platform 120 can use the VRF of VLAN 200 to configure the time synchronization server 202 to route time synchronization services between the customer network 104A and the first daemon of the first instance 246 providing time synchronization services by the time synchronization server 202 (e.g., ...). Figure 3A The time synchronization service is sent between the time synchronization daemon 304A and the time synchronization server 202. As an example, the programmable network platform 120 can be configured with a VRF that includes a virtual interface of VLAN 200 assigned to a physical port of the network interface card (NIC) 250 of the time synchronization server 202, wherein the virtual interface of VLAN 200 is mapped to a specific instance of the time synchronization service (e.g., ...). Figure 3A (The timed daemon 304A). Thus, when the first daemon processes the first instance 246 of the time synchronization service and causes the time synchronization server 202 to send a time synchronization message to the customer network 104A, the time synchronization server 202 can add the VLAN ID associated with VLAN 200 to the time synchronization service to send the service to the customer network 104A.

[0067] Similarly, the programmable network platform 120 can use the VRF of VLAN 201 to configure the time synchronization server 202 in the customer network 104B and the second daemon of the second instance 248 providing time synchronization services provided by the time synchronization server 202 (e.g., Figure 3AThe time synchronization service is sent between the time synchronization daemon 304B and the time synchronization server 202. In this example, the programmable network platform 120 can be configured with a VRF that includes a virtual interface of VLAN 201 assigned to a physical port of the (NIC) 250 of the time synchronization server 202, wherein the virtual interface of VLAN 201 is mapped to a specific instance of the time synchronization service (e.g., ...). Figure 3A (The timed daemon 304B). Thus, when the second daemon processes the second instance 246 of the time synchronization service and causes the time synchronization server 202 to send a time synchronization message to the customer network 104B, the time synchronization server 202 can add the VLAN ID associated with VLAN 201 to the time synchronization service to send the service to the customer network 104B.

[0068] Thus, when switch 226 receives time synchronization service 216 from at least one of customer devices 108A in customer network 104A, switch 226 can add a VLAN ID associated with VLAN 200 to send the service to the first daemon configured in time synchronization server 202. As an example, when switch 226 receives time synchronization service 216 with a VLAN ID associated with VLAN 100 from at least one of customer devices 108A in customer network 104A, switch 226 can convert the VLAN ID associated with VLAN 100 to a VLAN ID associated with VLAN 200 to send the service to the first daemon configured in time synchronization server 202. Similarly, when switch 226 receives time synchronization service 218 from at least one of customer devices 108B in customer network 104B, switch 226 can add a VLAN ID associated with VLAN 201 to send the service to the second daemon configured in time synchronization server 202. As an example, when switch 226 receives a time synchronization service 218 with a VLAN ID associated with VLAN 101 from at least one of the customer devices 108B in the customer network 104B, switch 226 can translate the VLAN ID associated with VLAN 101 into a VLAN ID associated with VLAN 201 to send the service to the first daemon configured within the time synchronization server 202. Services 216 and 218 may include PTP or NTP messages, more generally referred to as time synchronization messages, containing timestamp and / or time offset information.

[0069] As in Figure 3A and 3B As further described, the programmable network platform 120 can use a hybrid of VRF and network namespaces, depending on the type of virtualization (including containerization).

[0070] Figure 3A This is a block diagram illustrating an example of a time synchronization server, further illustrating one or more aspects of the technology described herein. Figure 3A In the example, time synchronization server 300 can represent Figure 1 Time synchronization server 110 or Figure 2 An example implementation of time synchronization server 202 is provided, which is configured with virtual routing and forwarding instances to route time synchronization services for VLANs associated with specific customer networks and specific instances of the time synchronization service.

[0071] exist Figure 3A In the example, the time synchronization server 300 includes network interface cards (“NIC”) 302A-302B (collectively referred to as “NIC 302”). Figure 3A The example provided is merely an example and can include any number of NICs. In this example, NIC 302A is configured with a first set of VLANs including VLANs 314A-314M (collectively referred to as "VLAN 314"). NIC 302B is configured with a second set of VLANs including VLANs 324A-324N (collectively referred to as "VLAN 324"). Each VLAN is associated with a specific customer network of the cloud switch requesting time synchronization services.

[0072] Programmable network platforms (e.g., Figure 1 and Figure 2 The programmable network platform 120 can utilize a time synchronization server 300 configured with VRFs for each VLAN, where each VRF includes one or more routes to send time synchronization services between a specific customer network and a specific instance of the time synchronization service. For example, VRF 314A includes a virtual interface for VLAN 312A, which is assigned to a physical port of NIC 302A and mapped to a timing daemon 304A of the first instance providing the time synchronization service. Similarly, VRF 314B includes a virtual interface for VLAN 312B, which is assigned to a physical port of NIC 302A and mapped to a timing daemon 304B of the second instance providing the time synchronization service. Likewise, VRF 314M includes a virtual interface for VLAN 312M, which is assigned to a physical port of NIC 302A and mapped to a timing daemon 304M of the Mth instance providing the time synchronization service.

[0073] The number of VLANs that can be configured on NIC 302A is limited by hardware constraints. As the number of customer networks connected to the cloud switch continues to expand, additional network interface cards (e.g., NIC 302B) can be added to the time synchronization server 300 and configured by the programmable network platform using one or more VLANs for the additional customer networks. For example, the programmable network platform can configure NIC 302B using VLANs for each additional customer network and configure VRFs for the VLANs to route time synchronization services between the time synchronization service instance and the additional customer networks. For example, VRF 324A includes a virtual interface for VLAN 322A, which is assigned to a physical port of NIC 302B and mapped to the timing daemon 306A of the instance providing the time synchronization service. Similarly, VRF 324B includes a virtual interface for VLAN 322B, which is assigned to a physical port of NIC 302B and mapped to the timing daemon 306B of another instance providing the time synchronization service. Similarly, VRF 324N includes a virtual interface for VLAN 322N, which is assigned to a physical port of NIC 302B and mapped to the timer daemon 306N, the Nth instance providing time synchronization services.

[0074] Timed processes, such as timed daemons 304A-304M (collectively referred to as "timed daemons 304"), can each provide an instance of time synchronization service for their respective VLAN in VLAN 312. Similarly, timed daemons 306A-306N (collectively referred to as "timed daemons 306") can each provide an instance of time synchronization service for their respective VLAN in VLAN 322. Each timed daemon 304 can use the same hardware clock (e.g., hardware clock 310A of NIC 302A) as a reference clock to provide a reference timing signal to a client computing device that synchronizes its system time with the system time of time synchronization server 300. Similarly, each timed daemon 306 can use the same hardware clock (e.g., hardware clock 310B of NIC 302B) as a reference clock to provide a reference timing signal to a client computing device that synchronizes its system time with the system time of time synchronization server 300. Although hardware clocks 310A and 310B are shown as NIC clocks, hardware clocks may include one or more of the following: network interface card (NIC) clock, graphics processing unit (GPU) clock, central processing unit (CPU) clock, or other hardware clocks.

[0075] Figure 3B This is a block diagram illustrating another example of a time synchronization server, further illustrating one or more aspects of the technology described herein. Figure 3B In the example, time synchronization server 350 can represent Figure 1 Time synchronization server 110 or Figure 2 An example implementation of time synchronization server 202 is provided, configured with namespaces to route time synchronization traffic for VLANs associated with specific customer networks and specific instances of the time synchronization service. Refer to VRF (e.g., such as...). Figure 3A The techniques described in this disclosure can also be applied to network namespaces, such as in the context of network containers (e.g., Kubernetes), as described below. Network namespaces can provide containers with network isolation, where each container includes its own IP address, routing table, etc.

[0076] exist Figure 3B In this example, the time synchronization server 350 is configured with namespaces, each namespace including one or more routes to send time synchronization traffic between a specific instance of the time synchronization service and a specific customer network. In this example, a programmable network platform (e.g., Figure 1 and Figure 2 The programmable network platform 120 can configure each NIC 302 of the time synchronization server 350 to include a namespace for each VLAN, where each VLAN is associated with a specific customer network. For example, namespace 334A includes a virtual interface for VLAN 312A, which is assigned to a physical port of NIC 302A and mapped to the timing daemon 304A of the first instance providing the time synchronization service. Similarly, namespace 334B includes a virtual interface for VLAN 312B, which is assigned to a physical port of NIC 302A and mapped to the timing daemon 304B of the second instance providing the time synchronization service. Likewise, namespace 334M includes a virtual interface for VLAN 312M, which is assigned to a physical port of NIC 302A and mapped to the timing daemon 304M of the Mth instance providing the time synchronization service.

[0077] The number of VLANs that can be configured on NIC 302A is limited by hardware constraints. As the number of customer networks connected to the cloud switch continues to expand, the programmable network platform can configure additional network interface cards (NICs), such as NIC 302B, for the time synchronization server 300 using one or more VLANs for the additional customer networks. For example, the programmable network platform can configure NIC 302B using VLANs for each additional customer network and configure namespaces for the VLANs to route time synchronization services between the time synchronization service instance and the additional customer networks. For example, namespace 344A includes a virtual interface for VLAN 322A, which is assigned to a physical port of NIC 302B and mapped to the timing daemon 306A of the instance providing the time synchronization service. Similarly, namespace 344B includes a virtual interface for VLAN 322B, which is assigned to a physical port of NIC 302B and mapped to the timing daemon 306B of another instance providing the time synchronization service. Similarly, namespace 344N includes a virtual interface for VLAN 322N, which is assigned to a physical port of NIC 302B and mapped to the timer daemon 306N, the Nth instance providing time synchronization services.

[0078] Figure 4 This is a conceptual diagram illustrating a redundant, precise timing system according to one or more techniques of the present invention. Figure 4 The interconnection system 400 may include two redundant time synchronization master devices, such as a primary time synchronization server 410A and a backup time synchronization server 410B. According to the technology described above, both the primary synchronization server 410A and the backup time synchronization server 410B can be configured to provide time synchronization services. Figure 4 In the example, backup time synchronization server 410B can provide time synchronization services in the event of a failure of primary time synchronization server 410A.

[0079] In this example, customer network 404A can be the primary network, while customer network 404B can be the backup network. Customer devices 408A1-408AN (collectively referred to as "customer devices 408A") of customer network 404A (e.g., the primary network) can access the primary synchronization server 410A and the backup synchronization server 410B via the primary cloud switch 402A. Similarly, customer devices 408B1-408BN (collectively referred to as "customer devices 408B") of customer network 404B (e.g., the backup customer network) can access the primary synchronization server 410A and the backup synchronization server 410B via the backup cloud switch 402B.

[0080] The interconnect system 400 may include two redundant VLANs (e.g., VLAN 411) belonging to the customer, providing primary and backup connections to the time synchronization service via the primary cloud switch 402A and backup cloud switch 402B, respectively. The customer may have ports assigned to each cloud switch (e.g., in...). Figure 4 It is shown as the "LAG port" and requires two virtual circuit connections to the time synchronization server.

[0081] exist Figure 4 In the example, when customer network 404A is onboard to cloud switch 402A, programmable network platform 120 can receive available IP addresses from the customer's subnet. These available IP addresses can be used to configure the primary time synchronization server 410A and the backup time synchronization server 410B. In this example, the administrator of customer network 404A can provide programmable network platform 120 with a first IP address of 10.10.20.20 / 24 and a second IP address of 10.10.20.21 / 24. In response, programmable network platform 120 can configure the primary time synchronization server 410A with the first IP address (e.g., 10.10.20.20 / 24) and the backup time synchronization server 410B with the second IP address (e.g., 10.10.20.21 / 24).

[0082] exist Figure 4 In the example, programmable network platform 120 can configure VLAN 400 associated with the primary time synchronization server 410A, backup time synchronization server 410B, and primary customer network 404A. Programmable network platform 120 can use VRF 412A associated with VLAN 400 to configure the primary time synchronization server 410A. Similarly, programmable network platform 120 can use VRF 412A associated with VLAN 400 to configure the backup time synchronization server 410B.

[0083] Because the routes to the primary time synchronization server 410A and the backup time synchronization server 410B are in the same VRF (e.g., VRF 412A) and associated with the same VLAN (e.g., VLAN 400), client devices can switch to the backup time synchronization server 410B in the event of a failure of the primary time synchronization server 410A. For example, if client device 408A is running the time synchronization protocol in multicast mode, the client device can automatically detect the backup time synchronization server 410B as the master device (e.g., using the best master clock algorithm). In this way, client device 408A can automatically switch to the backup time synchronization server.

[0084] In some examples, the programmable network platform 120 can configure VLAN 401 associated with the primary time synchronization server 410A, the backup time synchronization server 410B, and the backup customer network 404B for the backup connection. The programmable network platform 120 can configure the primary time synchronization server 410A using VRF 412B associated with VLAN 401. Similarly, the programmable network platform 120 can configure the backup time synchronization server 410B using VRF 412B associated with VLAN 401. In the event of a failure of the primary connection (e.g., a failure of cloud switch 402A), the customer network can use the backup connection to access the time synchronization service provided by either the primary time synchronization server 410A or the backup time synchronization server 410B.

[0085] Figure 5 This is a block diagram showing a more detailed view of a time synchronization server 540 configured to perform one or more technologies according to this disclosure. Figure 5 Time synchronization server 540 can represent Figure 1 Time synchronization server 110 Figure 2 Time synchronization server 202 Figure 3A Time synchronization server 300 Figure 3B Time synchronization server 350 and / or Figure 4 Example implementations of time synchronization servers 410A and 410B. Figure 5 This is just one example of a time synchronization server 540, and many other examples of synchronization server 540 can be used in other instances, and may include a subset of the components included in the example time synchronization server 540, or may include... Figure 5 Additional components not shown in the example time synchronization server 540.

[0086] like Figure 5 As shown in the example, time synchronization server 540 includes one or more processors 552, one or more input components 542, one or more communication units 544, one or more output components 546, and one or more storage components 548. The storage component 548 of time synchronization server 540 includes a time synchronization engine 554. Communication channel 550 interconnects each of components 542, 544, 546, 548, and 552 for inter-component communication (physical, communicative, and / or operational). In some examples, communication channel 550 may include a system bus, network connection, inter-process communication data structures, or any other method for transmitting data.

[0087] One or more communication units 544 of the time synchronization server 540 can communicate with external devices via one or more wired and / or wireless networks by sending and / or receiving network signals on one or more networks. Examples of communication units 544 include one or more network interface cards (e.g., NIC 556A-556N), such as Ethernet cards, optical transceivers, radio frequency transceivers, GPS receivers, or any other type of device capable of sending and / or receiving information. Other examples of communication units 544 may include shortwave radios, cellular data radios, wireless network radios, and Universal Serial Bus (USB) controllers.

[0088] One or more input components 542 of the time synchronization server 540 can receive input. Examples of input are haptic, audio, and video input. In one example, the input component 542 of the time synchronization server 540 includes a presence-sensitive input device (e.g., a touch-sensitive screen, PSD), a mouse, a keyboard, a voice response system, a camera, a microphone, or any other type of device for detecting input from a person or machine. In some examples, the input component 542 may include one or more sensor components, one or more position sensors (GPS components, Wi-Fi components, cellular components), one or more temperature sensors, one or more motion sensors (e.g., accelerometers, gyroscopes), one or more pressure sensors (e.g., barometers), one or more ambient light sensors, and one or more other sensors (e.g., microphones, cameras, infrared proximity sensors, hygrometers, etc.).

[0089] One or more output components 546 of the time synchronization server 540 can generate output. Examples of output are haptic, audio, and video output. In one example, the output component 546 of the time synchronization server 540 includes a PSD, sound card, video graphics adapter card, speaker, cathode ray tube (CRT) monitor, liquid crystal display (LCD), or any other type of device for generating output to a person or machine.

[0090] One or more processors 552 may implement functions and / or execute instructions associated with the time synchronization server 540. Examples of processors 552 include application processors, display controllers, auxiliary processors, one or more sensor hubs, and any other hardware configured to function as a processor, processing unit, or processing device. The time synchronization engine 554 may be operated by the processor 552 to perform various actions, operations, or functions of the time synchronization server 540. For example, the processor 552 of the time synchronization server 540 may retrieve and execute instructions stored in the storage unit 548, which cause the processor 552 to perform operations of the time synchronization engine 554. When executed by the processor 552, these instructions may cause the time synchronization server 540 to store information in the storage unit 548.

[0091] One or more storage components 548 within the time synchronization server 540 may store information processed during the operation of the time synchronization server 540 (e.g., the time synchronization server 540 may store data accessed by the time synchronization engine 554 during execution at the time synchronization server 540). In some examples, the storage component 548 is temporary storage, meaning that the primary purpose of the storage component 548 is not long-term storage. The storage component 548 on the time synchronization server 540 may be configured to store information temporarily as volatile memory, and therefore the stored contents are not retained if power is lost. Examples of volatile memory include random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), and other forms of volatile memory known in the art.

[0092] In some examples, storage component 548 also includes one or more computer-readable storage media. Storage component 548 in some examples includes one or more non-transitory computer-readable storage media. Storage component 548 may be configured to store a larger amount of information than is typically stored in volatile memory. Storage component 548 may be further configured to permanently store information in non-volatile memory space and retain the information after power-on or power-off cycles. Examples of non-volatile memory include magnetic hard disks, optical disks, floppy disks, flash memory, or electrically programmable memory (EPROM) or electrically erasable programmable memory (EEPROM). Storage component 548 may store program instructions and / or information (e.g., data) associated with synchronization engine 554. Storage component 548 may include memory configured to store data or other information associated with synchronization engine 554.

[0093] According to the technology of the present invention, the time synchronization engine 554 uses switching to provide time synchronization services. The time synchronization engine 554 can execute multiple time synchronization daemons or processes (e.g., PTP daemons and / or NTP daemons), each time synchronization process handling an instance of a time synchronization server corresponding to a client network. Each time synchronization process sends and receives timing packets via a corresponding VRF or namespace associated with a corresponding client network. As described above, the time synchronization server 540 also includes hardware network interface cards (NICs) 556A-556N through which all timing packets sent and received by the master device pass. Each hardware NIC includes a hardware clock and applies timestamps to sent or received packets, the timestamps indicating the time according to the hardware clock.

[0094] In this way, even if customer networks may have overlapping IP addresses, the time synchronization server 540 can manage time synchronization services to isolate each instance of the time synchronization service. Service providers can implement the techniques described herein as a service to customers to synchronize clocks on customer devices. The time synchronization server 540 can provide services to synchronize clocks on customer devices within a single data center, or it can synchronize customer devices located in different geographically distant regions. In some examples, aspects of the techniques described herein can be implemented as downloadable software plug-ins that execute on customer devices, for example, in a data center, and enable the computing system to perform the time synchronization process as described herein. In some examples, a device similar to the time synchronization server 540 (but performing client-side functions) can correspond to a customer device with software plug-ins and / or network interface cards (NICs) installed, which enable receiving and performing client-side time synchronization actions based on received timestamp offsets.

[0095] Figure 6 This is a flowchart illustrating an example operation of providing time synchronization services using cloud exchange according to the technology described in this disclosure. (Reference) Figure 1 To describe a programmable network platform Figure 6 The operation.

[0096] exist Figure 6 In the example, programmable network platform 120 receives IP addresses (602) of customer networks connected to cloud exchange 102 managed by the programmable network platform. For example, an administrator of customer network 104A can provide programmable network platform 120 with IP address 10.10.20.20 / 24 for customer network 104A. Similarly, an administrator of customer network 104B can provide programmable network platform 120 with IP address 10.10.20.20 / 24 for customer network 104B.

[0097] In response to receiving an IP address of a customer network connected to cloud switch 102, programmable network platform 120 configures time synchronization server 110 (604) connected to cloud switch 102 using VLANs associated with the customer network's IP address. For example, programmable network platform 120 can configure VLAN 200 for the IP address of customer network 104A and VLAN 201 for the IP address of customer network 104B. In this way, each VLAN can provide connectivity to the corresponding customer network while maintaining service isolation from other customer networks.

[0098] The programmable network platform 120 also utilizes VRFs or network namespaces for VLANs to configure a time synchronization server, wherein the VRFs or network namespaces include routes (606) for sending time synchronization services between a specific instance among multiple instances providing the time synchronization service in the customer network. For example, the programmable network platform 120 may configure VRF 112A associated with VLAN 200 and VRF 112B associated with VLAN 201. Each of VRFs 112A and 112B includes one or more routes for sending time synchronization services (e.g., PTP or NTP messages) for its associated VLAN. For example, the VRF may send time synchronization services for a VLAN associated with a specific customer network and a specific instance of the time synchronization service. For example, VRF 112A may include a first timing procedure for the first instance providing the time synchronization service (e.g., ...). Figure 3A The VRF112B may include a second timing procedure (e.g., a timing daemon 304A) for sending time synchronization services between the timing daemon 304A and the customer network 104A. Similarly, the VRF112B may include a second timing procedure for providing time synchronization services in a second instance (e.g., a timing daemon 304A). Figure 3A The time synchronization service route is sent between the timed daemon 304B and the customer network 104B.

[0099] Depending on the specific instance, certain actions or events of any of the techniques described herein may be performed in a different order, may be added, combined, or may be omitted entirely (e.g., not all described actions or events are necessary for practicing the techniques). Furthermore, in some examples, actions or events may be performed simultaneously, for example, through multithreading, interrupt handling, or multiple processors, rather than sequentially.

[0100] In one or more examples, the described functions can be implemented using hardware, software, firmware, or any combination thereof. If implemented in software, these functions can be stored or transmitted as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. A computer-readable medium can include a computer-readable storage medium, which corresponds to a tangible medium such as a data storage medium, or a communication medium that includes any medium that facilitates the transfer of a computer program from one place to another, for example, according to a communication protocol. In this way, a computer-readable medium can generally correspond to (1) a non-transitory tangible computer-readable storage medium or (2) a communication medium such as a signal or carrier wave. A data storage medium can be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementing the techniques described herein. A computer program product can include a computer-readable medium.

[0101] By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store the required program code in the form of instructions or data structures and is accessible by a computer. Furthermore, any connection is appropriately referred to as computer-readable media. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of media. Computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but rather refer to non-transient, tangible storage media. Disks and optical discs as used herein include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0102] Instructions can be executed by one or more processors including processing circuitry, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the term "processor" as used herein can refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. Additionally, in some aspects, the functionality described herein can be provided within dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into combined codecs. Furthermore, the techniques can be fully implemented within one or more circuit or logic elements.

[0103] The technology of this invention can be implemented in a wide variety of devices or apparatuses, including wireless handheld devices, integrated circuits (ICs), or a set of ICs (e.g., chipsets). Various components, modules, or units are described in this disclosure to emphasize functional aspects of a device configured to perform the disclosed technology, but they do not necessarily need to be implemented by separate hardware units. Rather, as described above, various units may be combined in a codec hardware unit or provided as a collection of cooperating hardware units including one or more processors as described above, along with appropriate software and / or firmware.

[0104] Various examples of this disclosure have been described. Any combination of the described systems, operations, or functions can be contemplated.

Claims

1. A method comprising: receiving, by a programmable network platform executed by a computing device, an Internet Protocol (IP) address of a customer network of a plurality of customer networks connected to a cloud exchange managed by the programmable network platform; configuring, by the programmable network platform, a time synchronization server connected to the cloud exchange with a virtualized local area network (VLAN) associated with the IP address of the customer network, the time synchronization server comprising one or more timing processes executed by the time synchronization server, each of the one or more timing processes providing a corresponding instance of a plurality of instances of a time synchronization service; and configuring, by the programmable network platform, the time synchronization server with a virtual routing and forwarding (VRF) or network namespace for the VLAN, wherein the VRF or network namespace comprises routes for sending time synchronization traffic between the customer network and a particular corresponding instance of the plurality of instances of the time synchronization service providing a particular timing process.

2. The method of claim 1, wherein the time synchronization service comprises a Network Time Protocol (NTP) service.

3. The method of claim 1, wherein the time synchronization service comprises a Precision Time Protocol (PTP) service.

4. The method of claim 1, wherein each of the one or more timing processes executed by the time synchronization server comprises a corresponding daemon.

5. The method of any of claims 1-4, wherein the customer network comprises a first customer network of the plurality of customer networks, wherein the particular corresponding instance comprises a first corresponding instance of the time synchronization service, wherein the VLAN comprises a first VLAN associated with the IP address of the first customer network, and wherein the VRF or the network namespace comprises a first VRF or a first network namespace for the first VLAN, the method further comprising: receiving, by the programmable network platform, an IP address of a second customer network of the plurality of customer networks connected to the cloud exchange, wherein the IP address of the first customer network and the IP address of the second customer network overlap; configuring, by the programmable network platform, the time synchronization server with a second VLAN associated with the IP address of the second customer network; and configuring, by the programmable network platform, the time synchronization server with a second VRF or a second network namespace for the second VLAN, wherein the second VRF or the second network namespace for the second VLAN comprises routes for sending time synchronization traffic between the second customer network and a second corresponding instance of the plurality of instances of the time synchronization service.

6. The method of any of claims 1-4, wherein the IP address comprises a first IP address of the customer network, wherein the VRF or network namespace comprises a first VRF or a first network namespace, wherein the time synchronization server comprises a first time synchronization server, wherein the multiple instances comprise a first plurality of instances of the time synchronization service, the method further comprising: receiving, by the programmable network platform, a second IP address of the customer network, wherein the first IP address of the customer network is used to configure the first time synchronization server and the second IP address of the customer network is used to configure a second time synchronization server for the customer network, the VLAN being associated with the first IP address and the second IP address of the customer network; configuring, by the programmable network platform, the second time synchronization server with the VLAN associated with the first IP address of the customer network and the second IP address of the customer network, the second time synchronization server comprising one or more second timing processes executed by the second time synchronization server, each of the one or more second timing processes providing a corresponding instance of a second plurality of instances of the time synchronization service; and configuring, by the programmable network platform, the second time synchronization server with a second VRF or a second network namespace for the VLAN, wherein the second VRF or the second network namespace comprises routing for sending time synchronization traffic between the customer network and a particular corresponding instance of the second plurality of instances of the time synchronization service in the event of a failure of the first time synchronization server.

7. A computing device comprising: one or more computer processors; and memory comprising instructions that, when executed by the one or more computer processors, cause the one or more computer processors to: receive an Internet Protocol (IP) address of a customer network of a plurality of customer networks connected to a cloud exchange; configure a time synchronization server connected to the cloud exchange with a virtualized local area network (VLAN) associated with the IP address of the customer network, the time synchronization server comprising one or more timing processes executed by the time synchronization server, each of the one or more timing processes providing a corresponding instance of a plurality of instances of a time synchronization service; and configure the time synchronization server with a virtual routing and forwarding (VRF) or network namespace for the VLAN, wherein the VRF or the network namespace comprises routing for sending time synchronization traffic between the customer network and a particular corresponding instance of the particular timing process of the plurality of instances of the time synchronization service.

8. The computing device of claim 7, wherein the time synchronization service comprises a Network Time Protocol (NTP) service.

9. The computing device of claim 7, wherein the time synchronization service comprises a Precision Time Protocol (PTP) service. ​ 10. The computing device of any of claims 7-9, wherein the customer network comprises a first customer network of the plurality of customer networks, wherein the VLAN comprises a first VLAN associated with an IP address of the first customer network, wherein the VRF or the network namespace comprises a first VRF or a first network namespace for the first VLAN, and wherein the particular corresponding instance comprises a first corresponding instance of the time synchronization service, the instructions further causing the one or more computer processors to: receive an IP address of a second customer network of the plurality of customer networks connected to the cloud exchange, wherein the IP address of the first customer network and the IP address of the second customer network overlap; configure the time synchronization server with a second VLAN associated with the IP address of the second customer network; and configure the time synchronization server with a second VRF or a second network namespace for the second VLAN, wherein the second VRF or the second network namespace comprises routes for sending time synchronization traffic between the second customer network and a second corresponding instance of the plurality of instances of the time synchronization service.

11. The computing device of any of claims 7-9, wherein the IP address comprises a first IP address of the customer network, wherein the VRF or the network namespace comprises a first VRF or a first network namespace, wherein the time synchronization server comprises a first time synchronization server, wherein the plurality of instances comprises a first plurality of instances of the time synchronization service, the instructions further causing the one or more computer processors to: receive a second IP address of the customer network, wherein the first IP address is used to configure the first time synchronization server and the second IP address is used to configure a second time synchronization server for the customer network, wherein the VLAN is associated with the first IP address and the second IP address; configure the second time synchronization server with the VLAN associated with the first IP address and the second IP address of the customer network, the second time synchronization server comprising one or more second timing processes executed by the second time synchronization server, each of the one or more second timing processes providing a corresponding instance of a second plurality of instances of the time synchronization service; and configure the second time synchronization server with a second VRF or a second network namespace for the VLAN, wherein the second VRF or the second network namespace comprises routes for sending time synchronization traffic between the customer network and a particular corresponding instance of the second plurality of instances of the time synchronization service in the event of a failure of the first time synchronization server.

12. An interconnection system comprising: a plurality of customer networks connected to a cloud exchange; and ​ a time synchronization server connected to the cloud exchange, the time synchronization server comprising: one or more timing processes executed by the time synchronization server, each of the one or more timing processes providing a corresponding instance of a plurality of instances of a time synchronization service; a virtualized local area network (VLAN) associated with an internet protocol (IP) address of a particular customer network of the plurality of customer networks; and a virtual routing and forwarding (VRF) or a network namespace for the VLAN, wherein the VRF or the network namespace comprises routes for sending time synchronization traffic between the particular customer network and a particular timing process of the particular corresponding instance of the plurality of instances of the time synchronization service.

13. The interconnection system of claim 12, wherein the time synchronization service comprises a network time protocol (NTP) service.

14. The interconnection system of claim 12, wherein the time synchronization service comprises a precision time protocol (PTP) service.

15. The interconnection system of any of claims 12 to 14, wherein the particular customer network comprises a first customer network of the plurality of customer networks, wherein the particular corresponding instance comprises a first corresponding instance of the time synchronization service, wherein the VLAN comprises a first VLAN associated with an IP address of the first customer network, wherein the VRF or the namespace comprises a first VRF or a first namespace for the first VLAN, and wherein the time synchronization server further comprises: a second VLAN associated with an IP address of a second customer network of the plurality of customer networks connected to the cloud exchange, wherein the IP address of the first customer network and the IP address of the second customer network overlap; and a second VRF or a second network namespace for the second VLAN, wherein the second VRF or the second network namespace for the second VLAN comprises routes for sending time synchronization traffic between the second customer network and a second instance of the plurality of instances of the time synchronization service.

16. The interconnection system of any of claims 12 to 14, wherein the IP address comprises a first IP address of the particular customer network, wherein the time synchronization server comprises a first time synchronization server, wherein the plurality of instances comprises a first plurality of instances of the time synchronization service, the interconnection system further comprising: a second time synchronization server comprising one or more second timing processes, each of the one or more second timing processes providing a corresponding instance of a second plurality of instances of the time synchronization service, wherein the second time synchronization server comprises: the VLAN, wherein the VLAN is associated with the first IP address of the particular customer network and a second IP address of the particular customer network; and the first VRF or the first namespace for the first VLAN, wherein the first VRF or the first namespace for the first VLAN comprises routes for sending time synchronization traffic between the first customer network and a first instance of the first plurality of instances of the time synchronization service. a second VRF or a second network namespace for the VLAN, wherein the second VRF or the second network namespace includes routes for sending time synchronization traffic between the particular customer network and a particular corresponding instance of the second plurality of instances providing the time synchronization service.

17. The interconnection system of claim 12, further comprising: a programmable network platform configured to: receive the IP address of the particular customer network; configure the time synchronization server with the VLAN associated with the IP address of the particular customer network; and configure the time synchronization server with the VRF or network namespace for the VLAN.

Images