Application-specific packet processing offload service

Abstract

A method for offloading network operations is described. The method includes receiving, from a first application, an offload service capability request message, the offload service capability request message used to request information about capabilities of an offload service that meet a set of requirements for the offload service; transmitting a response to the application, the response including a set of offload service templates that are (1) selected based on application requirements and (2) possible templates that will be modified for performing operations of the application; evaluating network resources for program code of the application to select a set of network resources for offloading operations of the first application to the network resources; and installing program code generated based on the set of offload service templates on the set of network resources to cause the set of network resources to process packets from a second application addressed to the first application.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of packet processing; and more specifically, to an offloading service representing an application that distributes packet processing to a set of network resources. Background Technology

[0002] Several techniques can be used to improve application performance, depending on the intended purpose of those applications. For example, for networking applications that provide packet switching and routing capabilities, techniques such as single root input / output virtualization (SR-IOV) and network interface cards (NICs) can be used to improve the processing performance of such applications. Although these techniques are designed to accelerate networking performance, they typically only provide general networking acceleration capabilities rather than application-specific packet processing logic acceleration.

[0003] Applications that directly benefit from application-specific offloading capabilities are typically implemented in a proprietary manner and tightly integrated with specialized software or hardware components that provide the necessary acceleration technologies for those specific applications. This tight integration with components that provide the desired acceleration capabilities can make applications entirely dependent on those technologies, which is undesirable in the context of cloud computing and virtualization, which involves a wide range of applications. Furthermore, such solutions are often difficult to scale and limited to a single application, thus preventing other applications from sharing (one or more) the same acceleration components. Summary of the Invention

[0004] A method for offloading network operations of a first application to network resources in a network system is described. The method includes: receiving an uninstallation service capability request message from the first application by an uninstallation service, wherein the uninstallation service capability request message requests information from the uninstallation service regarding the uninstallation service's capabilities to satisfy a set of requirements provided by the first application; and transmitting an uninstallation service capability request message response from the uninstallation service to the first application, wherein the uninstallation service capability request message response includes a set of uninstallation service templates, the uninstallation service templates being: (1) selected by the uninstallation service based on the set of requirements of the first application and (2) being possible templates to be modified for performing network operations of the first application, wherein each template in the set of uninstallation service templates includes one or more of the following: to be modified for performing network operations of the first application. A list of parameters for network operations of the first application, a set of code stubs for generating program code for performing network operations of the first application, and a set of libraries for generating program code; the uninstallation service evaluates network resources for the program code of the first application to select a set of network resources for installing the program and uninstalling the operations of the first application to the set of network resources; and the uninstallation service installs the program code of the first application generated based on the set of uninstallation service templates on the set of network resources such that the set of network resources processes one or more of the following: (1) packets addressed from the second application to the first application and (2) packets addressed from the first application to the second application.

[0005] It also describes a non-transitory machine-readable storage medium that provides instructions that, when executed by a processor of a device in a network system, will cause the processor to perform operations. These operations include: receiving an uninstallation service capability request message from a first application, wherein the uninstallation service capability request message requests information from an uninstallation service regarding the uninstallation service's ability to satisfy a set of requirements provided by the first application; transmitting an uninstallation service capability request message response to the first application, wherein the uninstallation service capability request message response includes a set of uninstallation service templates, the uninstallation service templates being: (1) selected by the uninstallation service based on the set of requirements of the first application and (2) possible templates to be modified for performing network operations of the first application, wherein each template in the set of uninstallation service templates includes one or more of the following: to be modified for performing network operations of the first application. The parameter list for performing network operations of the first application, the set of code stubs for generating program code for performing network operations of the first application, and the set of libraries for generating program code; evaluating network resources for program code of the first application to select a set of network resources for installing the program and uninstalling the operations of the first application to the set of network resources; and installing program code of the first application generated based on a set of uninstallation service templates on the set of network resources such that the set of network resources processes one or more of the following: (1) packets addressed from the second application to the first application and (2) packets addressed from the first application to the second application.

[0006] Also described is an apparatus for offloading network operations of a first application to network resources in a network system, the apparatus being configured to: receive an offload service capability request message from the first application, wherein the offload service capability request message requests information from an offload service regarding the offload service's capability to satisfy a set of requirements provided by the first application; and transmit an offload service capability request message response to the first application, wherein the offload service capability request message response includes a set of offload service templates, the offload service templates being: (1) selected by the offload service based on the set of requirements of the first application and (2) being possible templates to be modified for performing network operations of the first application, wherein each template in the set of offload service templates includes the following One or more of the following: a parameter list to be modified for performing network operations of the first application, a set of code stubs for generating program code for performing network operations of the first application, and a set of libraries for generating program code; evaluating network resources for the program code of the first application to select a set of network resources for installing the program and uninstalling the operations of the first application to the set of network resources; and installing the program code of the first application generated based on the set of uninstallation service templates on the set of network resources such that the set of network resources processes one or more of the following: (1) packets addressed from the second application to the first application and (2) packets addressed from the first application to the second application.

[0007] As described in this document, applications can use an offload service to dynamically request the offloading of certain application-specific packet processing logic implementations to P4 network resources available on the associated network infrastructure domain. The offload service has a holistic system view of the system-wide P4 network resources, allowing it to know the associated network topology and detailed information about all P4 network resources available on that network infrastructure, such as their location, the P4 architectures they support, their characteristics, etc. This holistic system view enables the offload service to find the most suitable P4 network resources(s) for deploying application-specific packet processing offload logic implementations, thereby ensuring the most efficient packet processing offload performance. Attached Figure Description

[0008] The invention can be best understood by referring to the following description and the accompanying drawings, which illustrate embodiments of the invention. In the drawings:

[0009] Figure 1 The diagram illustrates a logical representation of a set of functional blocks that can be used to manage the deployment and implementation of an unloading service according to an exemplary embodiment, including application-specific grouping unloading requirements and corresponding application-specific grouping unloading logic.

[0010] Figure 2 This illustrates a data flow between a pair of applications via a collection of P4 network resources and an offloading service, according to an exemplary embodiment.

[0011] Figure 3 An uninstallation service evaluation unit is shown according to an exemplary embodiment.

[0012] Figure 4A This illustrates the relationship between the unloading service template, the P4 target architecture, application-specific unloading service parameters, and the application-specific P4 code template, according to an exemplary embodiment.

[0013] Figure 4B This illustrates the relationship between application-specific uninstallation service parameters, application-specific P4 code templates, and application-specific P4 code, according to an exemplary embodiment.

[0014] Figure 5 This illustrates a method for verifying and testing application-specific P4 code according to an exemplary embodiment.

[0015] Figure 6 This illustrates the relationship between P4 network resources, application-specific offloading service data models, application-specific packet processing offloading logic, and offloading service templates according to an exemplary embodiment.

[0016] Figure 7This illustrates a data service flow between an application, non-P4 network resources, and P4 network resources according to an exemplary embodiment.

[0017] Figure 8 This illustrates a data service flow between an application, non-P4 network resources, and P4 network resources according to another exemplary embodiment.

[0018] Figure 9A A method for registering P4 network resources with an offloading service is shown according to an exemplary embodiment.

[0019] Figure 9B This diagram illustrates a data flow diagram for registering P4 network resources with an offloading service, according to an exemplary embodiment.

[0020] Figure 10A A method for an application to register with an uninstallation service is shown according to an exemplary embodiment.

[0021] Figure 10B This diagram illustrates a data flow diagram for an application to register with an uninstallation service, according to an exemplary embodiment.

[0022] Figure 11A This illustrates a method for terminating / unregistering an application from an uninstallation service, according to an exemplary embodiment.

[0023] Figure 11B This diagram illustrates a data flow diagram for terminating / unregistering an application from an uninstallation service, according to an exemplary embodiment.

[0024] Figure 12A This illustrates a method for terminating / unregistering an uninstallation service for P4 network resources, according to an exemplary embodiment.

[0025] Figure 12B This diagram illustrates a data flow diagram for terminating / unregistering a P4 network resource uninstallation service according to an exemplary embodiment.

[0026] Figure 13 An example is shown where an application utilizes a set of P4 network resources for packet parsing and header verification, according to an exemplary embodiment.

[0027] Figure 14 This illustrates another example of an application utilizing a set of P4 network resources for packet parsing and header verification, according to an exemplary embodiment.

[0028] Figure 15 This illustrates yet another example of an application utilizing a set of P4 network resources for packet parsing and header verification, according to an exemplary embodiment.

[0029] Figures 16A-16E This illustrates a method for offloading network operations of a first application to network resources in a network system, according to an exemplary embodiment.

[0030] Figure 17A The present invention illustrates connectivity between network devices (NDs) within an exemplary network and three exemplary implementations of the NDs, according to some embodiments thereof.

[0031] Figure 17B Exemplary methods for implementing a dedicated network device according to some embodiments of the present invention are shown.

[0032] Figure 17C Various exemplary methods of coupleable virtual network elements (VNEs) according to some embodiments of the present invention are shown.

[0033] Figure 17D This paper illustrates a network with a single network element (NE) on each ND in the ND according to some embodiments of the present invention, and compares in this straightforward approach a conventional distributed method (typically used by conventional routers) for maintaining reachability and forwarding information (also known as network control) with a centralized method for maintaining reachability and forwarding information.

[0034] Figure 17E The following simplified case is illustrated according to some embodiments of the invention: wherein each ND in the ND implements a single NE, but the centralized control plane has abstracted multiple NEs in different NDs into a single NE in one of (one or more) virtual networks.

[0035] Figure 17F The following situation is illustrated according to some embodiments of the invention: in which multiple VNEs are implemented on different NDs and these VNEs are coupled to each other, and in which a centralized control plane has abstracted these multiple VNEs so that they behave as a single VNE within one of the virtual networks.

[0036] Figure 18 A general control plane device with centralized control plane (CCP) software is shown according to some embodiments of the present invention. Detailed Implementation

[0037] The following description describes methods and apparatus for offloading services, whereby an application distributes packet processing to a set of network resources. Numerous specific details, such as logical implementation, opcodes, means of specifying operands, resource partitioning / sharing / copying implementation, types and interrelationships of system components, and logical partitioning / integration choices, are set forth in the following description to provide a more thorough understanding of the invention. However, those skilled in the art will recognize that the invention can be practiced without such specific details. In other instances, control structures, gate-level circuits, and complete sequences of software instructions have not been shown in detail to avoid obscuring the invention. Using the included description, those skilled in the art will be able to implement suitable functionality without excessive experimentation.

[0038] References to "an embodiment," "an embodiment," "an exemplary embodiment," etc., in this specification indicate that the described embodiment may include a particular feature, structure, or characteristic, but each embodiment may not necessarily include that particular feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same embodiment. Additionally, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is assumed that those skilled in the art will understand that such a feature, structure, or characteristic can be implemented in conjunction with other embodiments, whether or not it is explicitly described.

[0039] Parenthesized text and boxes with dashed borders (e.g., large dashes, small dashes, dotted lines, and dots) may be used herein to illustrate optional operations that add additional features to embodiments of the invention. However, such notation should not be construed as meaning that these are the only options or optional operations, and / or that boxes with solid borders are not optional in some embodiments of the invention.

[0040] In the following description and claims, the terms “coupled” and “connected” and their derivatives may be used. It should be understood that these terms are not intended to be synonyms with each other. “Coupled” is used to indicate that two or more elements cooperate or interact with each other, and they may or may not be in direct physical or electrical contact with each other. “Connected” is used to indicate that communication is established between two or more elements coupled to each other.

[0041] As will be described in more detail herein, a P4 packet processing offloading service (sometimes simply referred to as the offloading service) is proposed. Applications can use the P4 packet processing offloading service to dynamically specify their own application-specific packet processing offloading requirements for the allocation of P4 network resources. Application-specific packet processing offloading requirements (sometimes referred to as offloading requirements) are specified by the application utilizing P4 technology. As used herein, P4 is a technology designed to program the data plane functionality of network devices / resources and partially define the interface between the control plane and the data plane. P4 devices / resources are available in software virtual devices and in hardware devices such as network interface cards (NICs) and Ethernet switches. Although P4 technology and corresponding P4 network devices are described herein, the technology described herein can be similarly applied to other technologies that provide a language for expressing how packets are processed by the data plane of programmable forwarding elements such as hardware or software switches, network interface cards, routers, or network devices. Therefore, the use of P4 technology is for illustrative purposes.

[0042] The offloading service described in this document fulfills application offloading requirements by deploying the corresponding application-specific packet processing offloading logic, which can be implemented using P4 programs / code, to available P4 network resources in the associated network infrastructure domain. As used herein, the term "offloading" is intended to reflect that certain operations that could be performed by the application itself are instead performed on behalf of the application by other system components (e.g., P4 network resources), with the aim of increasing overall system efficiency.

[0043] Figure 1 This diagram illustrates a logical representation of a set of functional blocks for managing the deployment and implementation of an unloading service 100, according to an exemplary embodiment, including application-specific grouping unloading requirements and corresponding application-specific grouping unloading logic. For example... Figure 1As shown, the offloading service 100 may include a resource domain 102, which includes network topology information 104 and P4 resource information 106. The network topology information 104 (sometimes referred to as a network model) includes information associated with the network topology, including information about the interconnections between networking elements of the associated network infrastructure (e.g., P4 network resources and non-P4 network resources) and the interconnections between these networking elements and the compute / storage resources in which applications are deployed. The network topology information 104 may be provided directly to the offloading service 100 or may be dynamically discovered by the offloading service 100. In some embodiments, the visibility of the network topology information 104 to the offloading service 100 may be limited in scope. For example, the network topology information 104 provided to the offloading service 100 may be limited to the interconnections between a set of servers and networking resources directly connected to the set of servers. The offloading service 100 may use the network topology information 104 to help determine the most appropriate location for deploying application-specific packet processing offloading logic implementations.

[0044] P4 resource information 106 (sometimes referred to as the network resource information model) includes information about all P4 network resources available on the associated network infrastructure domain and the visibility of the offloading service 100 within the network topology (e.g., as specified by network topology information 104). Detailed information about each P4 network resource is useful for the offloading service 100 to make appropriate decisions regarding the deployment and monitoring of application-specific packet processing logic implementations. For example, it is expected that P4 network resources will provide, at a minimum, their unique identity, their capabilities (e.g., processing and memory availability), their system characteristics (e.g., the associated architecture corresponding to a specific compiler), and their location within the network infrastructure domain. If necessary, each P4 network resource may also be requested to provide the necessary development environment and other P4 network resource-specific information.

[0045] In some embodiments, each P4 network resource is continuously monitored to report the amount of available and allocated resources, as well as all relevant analytical information. For example, offloading service 100 can use this information to determine whether the application-specific packet processing offloading logic implementation is working as expected (e.g., whether all application-specific packet processing offloading requirements are being met).

[0046] like Figure 1As shown, the offloading service 100 may include a development domain 108, which includes an offloading service template 110. The offloading service template 110 allows applications to configure their specific offloading service specifications. The offloading service template 110 also includes, or represents, a P4 architecture, a P4 library, and development tools that allow application developers to write their own application-specific packet processing offloading logic implementations as P4 programs / code. In some embodiments, all P4 network resources in an associated network infrastructure domain share the same P4 architecture to allow the deployment of any application-specific packet processing offloading logic implementation on any available P4 network resource without recompilation or specific program adaptation. Also as Figure 1 As shown, the uninstallation service 100 may include an application domain 112, which includes application uninstallation service information 114, an application uninstallation service enabler 116, and an application uninstallation service agent 118. The application uninstallation service information 114 includes information about each application requesting uninstallation services from the uninstallation service 100, including information about all submitted application-specific packet-scraping uninstallation requests. For example, since each application provides its application-specific packet-scraping uninstallation requests and its application-specific packet-scraping uninstallation logic implementation, the uninstallation service 100 can use this information to evaluate the best packet-scraping uninstallation opportunity for each application. In this configuration, interactions between applications 202 can be performed via the application uninstallation service agent 118 of the uninstallation service 100, which allows applications 202 to manage, configure, provision, and communicate with the uninstallation service 100. In some embodiments, packets addressed to application 202 are either (1) processed by the corresponding network resource 204 and subsequently forwarded to application uninstallation service agent 118, and then reach the target application 202; or (2) transmitted to application uninstallation service agent 118 and forwarded to a set of network resources 204 for processing, and then reach the target application 202.

[0047] Application uninstallation service enabler 116 provides applications with information and software utilities for integration with uninstallation service 100. Specifically, a corresponding enabler 116 may be provided for each application to fully utilize uninstallation service 100. For example, enabler 116 may include automatically generated application programming interfaces (APIs) and client-server stubs, and potentially provide specific management and monitoring tools.

[0048] In some embodiments, the offloading service 100 provides application developers with test implementations of their application-specific offloading service logic, which can be used to verify the proper integration of their application(s) with the offloading service 100. Such test implementations may reflect the application-specific packet processing offloading logic implementation during the initial verification phase, or they may reflect a more integrated test implementation during the deployment phase. Such integrated test implementations may be designed to more realistically reflect a real P4 implementation running on one or more P4 network resources allocated for the deployment of the offloading logic.

[0049] Application uninstallation service proxy 118 provides a proxy for service management and provisioning requests between the application and the application-specific packet-processing uninstallation logic implementation. If it is necessary to dynamically provision the application-specific packet-processing uninstallation logic implementation, uninstallation service 100 may act as a proxy service for this purpose. For example, Figure 2 This illustrates the communication between a pair of applications 2021 and 2022 via P4 network resources 2041-204. N The collection and data flow of the uninstallation service 100. Specifically, application 2021 transmits an application uninstallation service management / providement request to the uninstallation service 100. This request can be made through the application uninstallation service agent 118, which can either be provided within the uninstallation service 100 or is running on the P4 network resource 204 itself. In this configuration, the uninstallation service 100 can act as a proxy and provide services to P4 network resources 2041-204. N The collection of committed application 2021 uninstallation service tables. Subsequently, P4 network resources 2041-204... N The set can represent the processing of application 2021 that may be transferred from application 2022 via non-P4 network resources 2061-206. M The collection is routed to P4 network resources 2041-204. N The grouping of the collection. As described, managing and supplying requests is separate and independent from the groups exchanged between applications 202. Furthermore, only the application 202 that owns the application uninstaller is allowed to manage / supply the groups exchanged between applications 202.

[0050] From a security perspective, the configuration described above allows the offloading service 100 to specify its own security requirement level to protect application-specific packet processing offloading logic implementations and isolate P4 network resources 204 from direct access by application 202. However, for trusted applications and services, the offloading service 100 may optionally allow application 202 to connect directly to their application-specific packet processing offloading logic implementations via an interface.

[0051] During the verification phase of the application-specific grouping and unloading logic implementation, a test provisioning agent can be provided to application developers to verify the integration of their application 202 with the unloading service 100. During the deployment phase, and once the application-specific grouping and unloading logic implementation has been fully deployed, a fully integrated provisioning agent can be provided to application 202 to manage its own application-specific grouping and unloading logic implementation.

[0052] For example Figure 1 As shown, the unloading service 100 may include core service logic 120, which includes an unloading service evaluation unit 122 and an unloading service deployment unit 124. Figure 3 As shown, the offload service evaluation unit 122 can verify application 202 requests for offload service 100 and allow application developers to integrate and verify their applications 202 with offload service 100, and then formally deploy the entire application-specific packet processing offload logic implementation on the associated network infrastructure domain. Specifically, the offload service evaluation unit 122 includes basic verification 302, which provides application certificate verification 302A, application offload service request verification 302B, and application service level agreement (SLA) verification 302C. Furthermore, the offload service evaluation unit 122 includes basic integration 304, which provides application offload service enabler 304A (e.g., application offload service basic test service 306) and application offload service basic test provisioning agent 304B.

[0053] The evaluation of the application request for the uninstallation service 100 includes the analysis of information provided by the application 202 when registering with the uninstallation service 100. During the application uninstallation service request process, it is assumed that, in addition to the P4 program that implements the application-specific packet processing uninstallation logic expected by the application 202, the application 202 will provide the uninstallation service 100 with a minimum amount of information about the SLA specification it requests via the uninstallation service template 110. The uninstallation service 100 verifies the information received during the registration of the application 202 with the uninstallation service 100. Verification criteria may be based on: (1) the result of compiling the application 202's P4 program using the specified uninstallation service parameters; and / or (2) the amount of P4 network resources 204 required to enable the implementation of the application-specific packet processing uninstallation logic. These verification checks may be performed automatically by the uninstallation service 100.

[0054] Once the basic validation checks are complete, application uninstallation service enablers 304A are provided to application developers to access and test the application-specific grouping and uninstallation logic implementations they request. Developers can use these application uninstallation service enablers 304A to develop and validate the proper integration of their application 202 with the uninstallation service 100.

[0055] As discussed earlier, for application 202 (which requires dynamically provisioned application-specific packet processing offloading logic implementations), a basic test provisioning agent 304B for the application offloading service can also be provided. Once the evaluation process is complete, the offloading service 100 demonstrates that application 202's request to the offloading service 100 meets all its evaluation criteria for potential deployment by the offloading service 100 on P4 network resource 204.

[0056] The offloading service deployment unit 124 identifies a P4 network resource 204 suitable for implementing an application-specific packet processing offloading implementation. In some embodiments, the most suitable P4 network resource 204 is the P4 network resource 204 closest to the requesting application 202. This could be a P4 network resource 204 located on the same server as application 202 (e.g., the P4 network resource 204 is a NIC) or a P4 network resource 204 directly connected to the server where application 202 is deployed (e.g., the P4 network resource 204 is an Ethernet switch). The most suitable P4 network resource 204 for the application-specific offloading service logic implementation may depend minimally on information associated with network topology and proximity, the availability and capabilities of the P4 network resource 204, and potentially on application-specific deployment requirements.

[0057] When the offloading service 100 allocates at least one P4 network resource 204 to run an application-specific packet processing offloading logic implementation, the offloading service 100 can implement the application-specific offloading service logic implementation on the allocated P4 network resource 204 by installing it on the allocated P4 network resource 204. Where the application 202 is deployed on multiple P4 network resources 204, the offloading service 100 supports evaluating and deploying the application-specific packet processing offloading logic implementation across the multiple P4 network resources 204.

[0058] To develop applications using P4 technology, a P4 target, along with its specific P4 architecture and development environment, is required. More specifically, the P4 target is a P4 network resource 204, which can be a software device or a hardware device. Given that each P4 network resource 204 may have its own specific capabilities and characteristics, each P4 target is required to provide at least one P4 architecture describing the packet pipelines and packet paths it supports, as well as detailing all necessary P4 libraries for any additional supported function blocks and data types.

[0059] Typically, P4 programs / code are not expected to be portable across different P4 architectures. However, given sufficient resources, a P4 program written for a given architecture should be portable across all P4 targets that faithfully implement the corresponding model / architecture. The P4 architecture can be viewed as a contract between P4 programs and P4 targets.

[0060] Although the P4 Portable Switch Architecture (PSA) represents the standard P4 architecture for P4 targets, each P4 target can also have its own specific P4 architecture. Assuming that the offloading service 100 will have to manage many different types of P4 network resources 204, the set of P4 network resources 204 in the network should use the minimum number of P4 architectures, and ideally, should use the same standard P4 architecture. It should also be noted that even if all P4 targets support the same PSA, they may have different amounts of resources (e.g., processing power and memory) to provide, which will also have to be specifically managed by the offloading service 100.

[0061] Uninstall service 100 provides application developers with an uninstall service template 110, allowing them to specify their expected requirements regarding the services provided by uninstall service 100. For example... Figure 4A As shown, each offload service template 110 specifies the P4 target architecture 402 and offload service-specific offload service parameters 404. The application-specific offload service parameters 404 can be used to determine the number of table entries required by application 202, the expected distribution scheme, and fault tolerance, etc. Application developers can select and populate the most suitable offload service template 110 that meets their needs, and the offload service 100 generates an application-specific P4 code template 406 to allow application developers to begin writing their own application-specific P4 code 408 (i.e., application-specific offload service logic implementation), such as... Figure 4B As shown in the image.

[0062] Once application developers have completed their application-specific P4 code 408 implementation, they use tools provided by the uninstallation service 100 to verify and test their implementation. For example, Figure 5 As shown in method 500, in operation 502, the filled application-specific P4 code template 406 (including application-specific P4 code 408) is first checked against the provided unloading service architecture (e.g., P4 target architecture 402) and the specified unloading service parameters (e.g., application-specific unloading service parameters 404). If the filled application-specific P4 code template 406 is invalid, then in operation 504, the filled application-specific P4 code template 406 is rejected. Conversely, if the filled application-specific P4 code template 406 is valid, then in operations 506-520, the filled application-specific P4 code template 406 can be appropriately tested, and then in operation 522, it is deployed on one or more P4 targets.

[0063] Specifically, method 500 may first determine whether the filled application-specific P4 code template 406 has been successfully tested (e.g., application-specific P4 code 408 has passed all tests). In response to determining that the filled application-specific P4 code template 406 has not yet been tested, method 500 moves to operation 508 to generate and deploy a test implementation of the application-specific P4 code 408. Specifically, to test the application-specific P4 code 408, the necessary capabilities are provided to application developers for testing their application-specific P4 code 408 before deploying it on a live network infrastructure. A first test implementation is proposed to test the application-specific P4 code 408 on a selected offload service architecture using specified offload service parameters in operation 508. This first test setup tests the application-specific P4 code 408 as a standalone application.

[0064] Assuming the applied-specific P4 code template 406 is valid and operation 508 is working as expected, and after operation 506 determines that the applied-specific P4 code template 406 has been successfully tested, operation 510 uninstalls service 100, identifying the specific P4 network resource 204 where the applied-specific P4 code template 406 should be deployed. Depending on the provided uninstallation service parameters and network infrastructure, it may be necessary to deploy the applied-specific P4 code template 406 on multiple different P4 network resources 204. When multiple applied-specific P4 code templates 406 share a single P4 network resource 204 (as determined in operation 512), a second test implementation is proposed to test multiple applied-specific P4 code templates 406 on a system similar to the allocated P4 network resource 204. Specifically, in operation 514, uninstallation service 100 determines whether the merging of operations involving multiple applied-specific P4 code templates 406 was successful. If the merging is unsuccessful, in operation 516, uninstallation service determines to reject the request from application 202. If (1) no shared P4 network resource 204 is selected in operation 512 or (2) operation 514 confirms that everything is working as expected, then in operation 518, the unload service determines whether testing has been performed on a system similar to the selected P4 network(s) 204. If testing has not been performed, method 500 moves to operation 520 to generate and deploy a test implementation on a system similar to the selected P4 network resource 204. Conversely, if testing has been performed, method 500 moves to operation 522 to deploy the newly generated P4 implementation to the allocated P4 resource.

[0065] like Figure 6As shown, an offloading service template 110 is provided for application 202 to specify application-specific packet processing offloading requirements to offloading services 100. In one embodiment, the deployment parameterization of application-specific packet processing offloading logic 602 and data model 604 can specify certain deployment and service level expectations, including application P4 offloading service implementation 606 and application offloading service parameters 608. For example, such deployment parameters can be specifically tailored to performance, resilience, and sizing design aspects or required capabilities and deployment constraints.

[0066] Regarding the development environment, a suitable development environment would be available to allow applications 202 to specify their own application-specific grouping unloading logic 602 and data model 604 to the unloading service 100. Ideally, such a development environment would minimally propose an unloading service P4 architecture 610 and an unloading service P4 library 612, along with corresponding unloading service parameters 614 supported by the unloading service 100. The applications would then use those unloading service P4 architectures 610 and P4 libraries 612 to write P4 programs that implement their application-specific grouping unloading logic 602 and data model 604.

[0067] like Figure 7 As shown, application 202 may be interconnected through several different networking elements (e.g., non-P4 network resources 206 and P4 network resources 204), which is true when application 202 is deployed on different computing resources of a public cloud network infrastructure. This is despite the fact that some of those networking elements may be P4 capable (e.g., P4 network resources 2041-204). N However, some other networking components may not have P4 capabilities (e.g., non-P4 network resources 2061-206). Y ).exist Figure 7 In the example shown, data traffic is exchanged between application 2021 and application 2022, where the data traffic potentially passes through several P4 network resources 204 and non-P4 network resources 206.

[0068] In one embodiment, at least one P4 network resource 204 is available on the intended service path between applications 202, while there may or may not be any non-P4 network resources 206. The availability of at least one P4 network resource 204 within the associated network infrastructure domain enables the offloading service 100 to provide services to applications 202 within the associated system domain. The requirement of having at least one P4 network resource 204 is imposed by the fact that the offloading service 100 expects applications 202 to use P4 technologies that can only be deployed on P4 network resources 204 to provide their application-specific offloading service logic implementations.

[0069] Given that traffic between Application 2021 and Application 2022 will always require traversing at least one or more P4 network resources 204, Application 2021 and Application 2022 can utilize any of those P4 network resources 204 to perform tasks on behalf of Application 202 itself. For example, as Figure 8 As shown, when sending packets from application 2021 to application 2022, application 2022 may potentially benefit from offloading certain packet processing tasks to P4 network resource 204, thereby allowing it to perform its preprocessing operations on its destination packets before they arrive at application 2022 itself. However, in one embodiment, the most suitable location should be kept as close as possible to application 2022, whenever possible.

[0070] In some embodiments, the offloading service 100 may allow application 202 to manage, provision, and communicate with specific P4 network resources 204 allocated to it by an application-specific offloading service logic implementation. When deploying an application-specific offloading service logic implementation, the offloading service 100 may generate new APIs to manage and provision that specific logic implementation. For example, APIs may be generated to populate table entries used by the application's logic. P4 technology assumes that the compilation of a P4 program produces two main artifacts: (1) a data plane configuration that implements the forwarding logic of the P4 program; and (2) an API for managing the state of data plane objects from the control plane.

[0071] Since the offloading service 100 is expected to provide the necessary logic implementation for accessing each application-specific offloading service logic implementation on each P4 target, the offloading service 100 can provide the necessary proxy functionality to allow each application 202 to remotely access those allocated P4 network resources 204. The interface between the P4 pipeline and the offloading service 100 can be implemented using a proprietary solution, or alternatively, can be automatically generated by the offloading service 100 using P4 runtime technology. As discussed above, the interface to the proxy service can be part of the application-specific offloading service enabler 116.

[0072] In some embodiments, assuming that each offloading service 100 manages different P4 network resources 204 or manages at least different slices of network infrastructure, multiple instances of offloading service 100 may be available on the network infrastructure. For example, each offloading service 100 may have its own visibility to the network infrastructure, in which case different applications 202 may request to use different instances of offloading service 100.

[0073] Considering the scenario where offloading service 100 is not available on the deployed system or where offloading service 100 cannot allocate any P4 network resources 204 to application 202, application 202 may assume that offloading service 100 cannot provide its application-specific packet processing offloading logic implementation. Therefore, it is assumed that application 202 should always maintain the ability to fully process its entire packet processing logic to account for situations where offloading service 100 cannot fulfill its requests.

[0074] Therefore, application 202 can implement some means to confirm whether packets have passed their intended application-specific packet offloading logic implementation (i.e., proof of work) when packets arrive at application 202 itself. For example, in cases where the application-specific packet offloading logic implementation adds a preamble header, the proof of work could be the presence of the preamble header itself.

[0075] Now go to Figure 9A This document describes a method 900A for registering a P4 network resource 204 with an offloading service 100 according to an exemplary embodiment. The operations in the flowcharts will be described with reference to exemplary embodiments of other figures. However, it should be understood that the operations in the flowcharts can be performed by embodiments of the invention other than those discussed with reference to other figures, and that the embodiments of the invention discussed with reference to these other figures can perform operations different from those discussed in the flowcharts. Figure 9B Data flow diagram 900B describes method 900A, in Figure 9B In this embodiment, P4 network resource 2041 registers with the offloading service 100. In other embodiments, other P4 network resources 204 may also register with the offloading service 100 using method 900A.

[0076] like Figure 9A As shown, method 900A may begin with operation 902A, where P4 network resource 2041 transmits an uninstallation service discovery message 902B for discovering the uninstallation service 100. Specifically, the uninstallation service discovery message 902B may be a multicast or broadcast message, or P4 network resource 2041 may have been statically configured to transmit the uninstallation service discovery message 902B to a specific address and / or port associated with the uninstallation service 100. Therefore, the uninstallation service 100 can detect / receive the uninstallation service discovery message 902B.

[0077] In operation 904A, in response to the offload service discovery message 902B, offload service 100 transmits an offload service discovery message acknowledgment 904B to P4 network resource 2041. Therefore, P4 network resource 2041 can receive the offload service discovery message acknowledgment 904B, which indicates that the offload service discovery message 902B has been received and that offload service 100 is present.

[0078] In operation 906A, P4 network resource 2041 transmits a P4 network resource registration request 906B to the offloading service 100 discovered based on operations 902A and 904A. The P4 network resource registration request 906B may indicate various characteristics associated with P4 network resource 2041, including the architecture of P4 network resource 2041, the available resources of P4 network resource 2041 (e.g., processing and memory resources), and / or the compiler associated with the architecture of P4 network resource 2041 for compiling P4 code that will be offloaded to P4 network resource 2041 by application 202.

[0079] In operation 908A, the offloading service 100 sets / updates the P4 network resource information model 908B based on the P4 network resource registration request 906B received from the P4 network resource 2041. Specifically, the offloading service 100 indicates the characteristics of the P4 network resource 2041 in the P4 resource information 106 based on the P4 network resource registration request 906B.

[0080] In operation 910A, the offloading service sets up / updates the network model 910B based on the P4 network resource registration request 906B received from the P4 network resource 2041. Specifically, the offloading service 100 updates the network topology information / model 104 based on the P4 network resource registration request 906B to include links between the P4 network resource 2041 and other components of the network infrastructure.

[0081] In operation 912A, the offloading service 100 sends a P4 network resource registration request confirmation 912B to the P4 network resource 2041. The P4 network resource registration request confirmation 912B indicates that the P4 network resource 2041 has registered with the offloading service 100. Therefore, upon receiving the P4 network resource registration request confirmation 912B, the P4 network resource 2041 can now expect to potentially receive an offloading request for processing network data on behalf of the remote application 202.

[0082] Figure 10A A method 1000A for registering application 202 with uninstallation service 100 according to an exemplary embodiment is shown. The operations in the flowcharts will be described with reference to exemplary embodiments of other figures. However, it should be understood that the operations in the flowcharts can be performed by embodiments of the invention other than those discussed with reference to other figures, and that the embodiments of the invention discussed with reference to these other figures can perform operations different from those discussed in the flowcharts. The following will be discussed regarding... Figure 10B The data flow diagram 1000B is used to describe method 1000A. Figure 10B In this embodiment, application 2021 registers with uninstallation service 100. In other embodiments, other applications 202 may also register with uninstallation service 100.

[0083] like Figure 10A As shown, method 1000A may begin with operation 1002A, in which application 2021 transmits an uninstallation service discovery message 1002B for discovering the uninstallation service 100. Specifically, the uninstallation service discovery message 1002B may be a multicast or broadcast message, or application 2021 may have been statically configured to transmit the uninstallation service discovery message 1002B to a specific address and / or port associated with the uninstallation service 100. Therefore, the uninstallation service 100 can detect / receive the uninstallation service discovery message 1002B.

[0084] In operation 1004A, in response to the uninstall service discovery message 1002B, uninstall service 100 sends an uninstall service discovery message acknowledgment 1004B to application 2021. Therefore, application 2021 can receive the uninstall service discovery message acknowledgment 1004B, which indicates that the uninstall service discovery message 1002B has been received and therefore uninstall service 100 exists.

[0085] In operation 1006A, application 2021 sends an uninstallation service capability request message 1006B to uninstallation service 100. The uninstallation service capability request message 1006B requests information from uninstallation service 100 regarding the capabilities of uninstallation service 100. Specifically, application 2021 requests the capability of P4 network resources 204 available to uninstallation service 100, so that uninstallation service 100 can make these P4 network resources 204 available to application 2021.

[0086] In operation 1008A, the offloading service 100 transmits an offloading service capability request message response / acknowledgment 1008B to the application 2021. The offloading service capability request message response / acknowledgment 1008B indicates the capabilities of the offloading service 100, including the processing power, memory capacity, and / or architecture of various P4 network resources 204 registered with the offloading service 100. For example, the offloading service 100 may indicate one or more P4 architectures to the developer of the application 2021, enabling the developer to write their own P4 program specifying their needs for packet processing offloading. In some embodiments, the offloading service 100 may also suggest an application-specific offloading implementation or offloading service template 110 to the application 2021. More specifically, via the offloading service capability request message response / acknowledgment 1008B, the offloading service 100 may provide multiple application-specific implementations / templates that can be considered to meet the common needs of many different applications 202. In this case, an application-specific implementation / template can be instantiated for Application 2021 based on the requirements of Application 2021 and the input / parameters provided by the developer, in order to generate P4 program code that performs the uninstallation logic of Application 2021.

[0087] In operation 1010A, application 2021 sends an application uninstallation service request 1010B to uninstallation service 100. Application uninstallation service request 1010B indicates the desired operation / service to be uninstalled to P4 network resource 204. For example, application uninstallation service request 1010B may include a set of parameters and a set of processing / memory requirements and / or an indication of the application-specific implementation / template to be instantiated by uninstallation service 100 on behalf of application 2021.

[0088] In operation 1012A, the uninstallation service 100 evaluates application uninstallation service request 1010B. Specifically, when the uninstallation service 100 receives the application uninstallation service request 1010B, the uninstallation service 100 performs analysis to verify the application uninstallation service request 1010B and determine the most suitable P4 network resource(s)204 that can be allocated to the application-specific implementation / template of the request.

[0089] In operation 1014A, uninstallation service 100 sends an application uninstallation service request confirmation 1014B to application 2021. Specifically, uninstallation service 100, via application uninstallation service request confirmation 1014B, instructs application 2021 whether a set of P4 network resources 204 for hosting application 2021 has been found and whether the submitted P4 program code has been appropriately modified to meet the requirements of the allocated P4 network resources 204.

[0090] In operation 1016A, application 2021 sends an application uninstallation deployment request 1016B to uninstallation service 100. Specifically, in response to application 2021 receiving application uninstallation service request confirmation 1014B (application uninstallation service request confirmation 1014B indicating that uninstallation service 100 has found the appropriate location for the associated logic used to host application 2021 and the submitted P4 program code has been appropriately modified to meet the requirements of the allocated P4 network resource 204), application 2021 then submits application uninstallation deployment request 1016B to uninstallation service 100, application uninstallation deployment request 1016B being used to initiate the uninstallation operation.

[0091] In operation 1018A, the uninstallation service 100 transmits an uninstallation service deployment request 1018B to the set of allocated P4 network resources 204. Specifically, in response to the application uninstallation deployment request 1016B, the uninstallation service 100 transmits an uninstallation service deployment request 1018B to the set of allocated P4 network resources 204, which includes an application-specific implementation that implements the logic of associated P4 program code for processing by the set of allocated P4 network resources 204, thereby enabling the set of allocated P4 network resources 204 to process / deploy the logic of the associated P4 code in operation 1020A.

[0092] In operation 1022A, the set of allocated P4 network resources 204 sends an uninstallation service deployment request confirmation 1022B to the uninstallation service 100. The uninstallation service deployment request confirmation 1022B is used to confirm the deployment of the application-specific implementation / template.

[0093] In operation 1024A, the uninstallation service 100 sends an application uninstallation deployment request confirmation 1024B to application 2021. The application uninstallation deployment request confirmation 1024B is used to confirm the deployment of the application-specific implementation / template.

[0094] Figure 11A A method 1100A for terminating / unregistering from uninstall service 100 according to an exemplary embodiment is shown. The operations in the flowcharts will be described with reference to exemplary embodiments of other figures. However, it should be understood that the operations in the flowcharts can be performed by embodiments of the invention other than those discussed with reference to other figures, and that the embodiments of the invention discussed with reference to these other figures can perform operations different from those discussed in the flowcharts. The following will be discussed regarding... Figure 11B The data flow diagram 1100B is used to describe method 1100A. Figure 11B In this embodiment, application 2021 deregisters with uninstall service 100. In other embodiments, other applications 202 may also deregister with uninstall service 100, so that uninstall service 100 removes the application-specific uninstall service logic implementation (if any) from the corresponding P4 network resource 204 in which they are deployed.

[0095] like Figure 11A and Figure 11B As shown, method 1100A can begin at operation 1102A, wherein application 2021 sends an application uninstallation service termination request 1102B to uninstallation service 100. The application uninstallation service termination request 1102B requests uninstallation service 100 to deregister application 2021, so that uninstallation service 100 removes the application-specific uninstallation service logic implementation from the corresponding P4 network resource 204.

[0096] In operation 1104A, the uninstallation service 100 evaluates the application uninstallation service termination request 1102B in operation 1104B. Specifically, the uninstallation service 100 determines which P4 network resources 204 are associated with / assigned to application 2021. For illustrative purposes, method 1100A will be described with respect to the uninstallation service 100 determining in operation 1104A that P4 network resources 2041 are associated with / assigned to application 2021.

[0097] In operation 1106A, the uninstallation service 100 transmits an uninstallation service termination request 1106B to the P4 network resource 204 determined in operation 1104A. That is, in operation 1106A, the uninstallation service 100 transmits the uninstallation service termination request 1106B to the P4 network resource 2041.

[0098] In operation 1108A, P4 network resource 2041 processes service termination request 1106B (1108B). Specifically, in operation 1108A, P4 network resource 2041 locates and removes the application-specific service termination logic implementation associated with application 2021.

[0099] In operation 1110A, P4 network resource 2041 transmits an uninstallation service termination request acknowledgment 1110B to uninstallation service 100. The uninstallation service termination request acknowledgment 1110B indicates that P4 network resource 2041 has received and successfully processed the uninstallation service termination request 1106B to remove the application-specific uninstallation service logic implementation associated with application 2021.

[0100] In operation 1112A, uninstallation service 100 sets / updates the network resource information model 1112B based on uninstallation service termination request 1106B. That is, in response to uninstallation service termination request confirmation 1110B and based on uninstallation service termination request 1106B, uninstallation service 100 removes the association between application 2021 and P4 network resource 2041.

[0101] In operation 1114A, in response to receiving the uninstall service termination request acknowledgment 1110B, uninstall service 100 transmits the application uninstall service termination request acknowledgment 1114B to application 2021. The application uninstall service termination request acknowledgment 1114B indicates that uninstall service 100 and P4 network resource 2041 have received and successfully processed the application uninstall service termination request 1102B to remove the application-specific uninstall service logic implementation associated with application 2021.

[0102] Figure 12A A method 1200A for terminating / unregistering from a P4 network resource 204 and unloading service 100 is illustrated according to an exemplary embodiment. The operations in the flowcharts will be described with reference to exemplary embodiments of other figures. However, it should be understood that the operations in the flowcharts can be performed by embodiments of the invention other than those discussed with reference to other figures, and that the embodiments of the invention discussed with reference to these other figures can perform operations different from those discussed in the flowcharts. The following will discuss… Figure 12B The data flow diagram 1200B is used to describe method 1200A. Figure 12BIn this embodiment, P4 network resource 2041 deregisters with the uninstallation service 100. In other embodiments, other P4 network resources 204 may also deregister with the uninstallation service 100, so that the uninstallation service 100 removes the application-specific uninstallation service logic implementation from the corresponding P4 network resources 204 in which they are deployed.

[0103] like Figure 12A and Figure 12B As shown, method 1200A can begin at operation 1202A, where P4 network resource 2041 sends a P4 network resource deregistration request 1202B to the uninstallation service 100. The P4 network resource deregistration request 1202B requests the uninstallation service 100 to deregister P4 network resource 2041, so that the uninstallation service 100 removes the application-specific uninstallation service logic implementation from P4 network resource 2041.

[0104] In operation 1204A, the uninstallation service 100 transmits an unsolicited application uninstallation service termination request 1204B to the application 202 associated with the P4 network resource 2041. Specifically, the uninstallation service 100 transmits the unsolicited application uninstallation service termination request 1204B to any application 202 running an application-specific uninstallation service logic implementation on the P4 network resource 2041. For illustrative purposes, the uninstallation service 100 transmits the unsolicited application uninstallation service termination request 1204B to application 2021 in operation 1204A.

[0105] In operation 1206A, application 2021 processes 1206B an unsolicited application offload service termination request 1204B. That is, application 2021 performs a compensation / switching process in response to P4 network resource 2041 no longer processing application 2021's application-specific offload service logic implementation.

[0106] In operation 1208A, application 2021 sends an unsolicited application uninstallation service termination request acknowledgment 1208B to uninstallation service 100. The unsolicited application uninstallation service termination request acknowledgment 1208B indicates that application 2021 has received and successfully processed the unsolicited application uninstallation service termination request 1204B.

[0107] In operation 1210A, in response to receiving an unsolicited application uninstallation service termination request 1204B, uninstallation service 100 updates the network resource information model 1210B. That is, uninstallation service 100 updates the network model to indicate that P4 network resource 2041 is no longer available.

[0108] In operation 1212A, the offloading service 100 transmits a P4 network resource deregistration request confirmation 1212B to the P4 network resource 2041. The P4 network resource deregistration request confirmation 1212B indicates that the P4 network resource deregistration request 1202B is received and processed so that the P4 network resource 2041 is removed from the network resource information model 1210B and is no longer available for offloading allocation by the offloading service 100.

[0109] Now go to Figure 13 An example will be described where application 202 utilizes a set of P4 network resources 204 for packet parsing and header verification. Specifically, packet parsing and header verification can be considered recurring processing overhead; it remains necessary, but may represent a relatively heavy burden for application 202. Such tasks could potentially be performed more efficiently on behalf of application 202 by other system components (e.g., P4 network resources 204).

[0110] In this use case, it is assumed that application 202 uses P4 technology to specify its packet parsing and header verification processing requirements, thereby submitting its offloading service specifications and P4-based service logic implementation to offloading service 100. Offloading service 100 is responsible for verifying and evaluating the offloading service request from application 202, as well as the deployment and appropriate implementation of the offloading capabilities of the carefully planned request.

[0111] Based on the application request to the uninstallation service 100 (the portion of its specific grouping processing logic used to uninstall it), the uninstallation service 100 may determine that the delegated task will be executed more efficiently at a strategic location (or more) within the system infrastructure domain, potentially by specialized software and / or hardware components to increase performance.

[0112] like Figure 13 As shown, once application developers have developed and tested their application-specific packet processing offloading logic implementation, the corresponding application 202 requests the offloading service 100 to deploy it on the most suitable P4 network resource 204 of the associated network infrastructure. In this example, application 2021 sends a request to the offloading service 100. Along with the request, application 2021 sends the corresponding P4 program code that implements the requested application-specific offloading service logic using application-specific offloading service configuration parameters. Upon receiving the application offloading service request, the offloading service 100 verifies and evaluates the request. Assuming the provided P4 program code is deemed valid and the request satisfies the terms of the corresponding SLA with the offloading service 100, the offloading service 100 determines that the application-specific packet processing offloading logic implementation can be deployed on P4 network resources 2041 and 2042, which are directly connected to compute / storage resources 13021-1302. PThe collection includes 13021 compute / storage resources currently running in 2021. Figure 13 In this context, the application-specific uninstallation service implementation corresponds to uninstallation service 100 installing the application-specific packet processing uninstallation logic implementation on P4 network resources 2041 and 2042.

[0113] Go to Figure 14 As shown in the diagram, application 2021 has requested preprocessing of packets destined for it. When packets are sent from client application 1402, they traverse the network infrastructure until they reach one of the P4 network resources 204 directly connected to the compute / storage resource 13021 in which application 2021 is running. In this case, it is P4 network resource 2041.

[0114] As mentioned earlier, Application 2021 has requested that some of its packet parsing and header validation tasks be offloaded. For example, all packets destined for Application 2021 are validated and / or filtered according to some application-specific rules before reaching Application 2021 itself. That means that each packet will be examined more or less thoroughly, which assumes that at least a certain number of packet headers will be parsed and validated.

[0115] Although the application of specific packet processing offloading logic may still be limited to packet header verification, Figure 14 The example shown suggests providing additional information to Application 2021. For instance, this additional information could be provided via a pre-packet header added by the application-specific offloading service implementation. Application 2021 can assume that the information contained in this pre-packet header is information already parsed by its offloading functionality, with the intent to make more efficient decisions within Application 2021 itself. This information could also represent information obtained from the data model associated with the application-specific offloading service implementation and based on packet header information, existing service flows, or network data analysis. This example assumes that Application 2021 will use this pre-packet header information to more quickly jump to its core logic, rather than having to perform deeper packet parsing operations and multiple header validations and table lookups before starting to execute its core logic.

[0116] While this type of offloading functionality used for packet parsing and header verification can be seen as very useful for Virtual Network Functions (VNF) applications to minimize their recurring packet header overhead, several other offloading functions are conceivable, such as those that provide application-specific load balancing functionality, or those that provide application-specific routing, switching, service linking, or many other types of packet processing functionality.

[0117] Now go to Figure 15This will describe a use case where application 202 can dynamically manage table entries for its application-specific offload service implementation deployed on P4 network resource 204. For example... Figure 15 As shown, packets are sent from client application 1502 to application 2021. Assuming an application-specific offloading service implementation is deployed on the left-hand P4 network resource 2041 directly connected to compute / storage resource 13021, packets destined for application 2021 are specifically preprocessed to identify existing service flows. When a packet destined for application 2021 is not associated with any existing service flow, the packet is sent to application 2021 on compute / storage resource 13021. Furthermore, a pre-header may be added to each packet destined for application 2021.

[0118] When application 2021 receives a packet, it can extract information from the front header, which can be used to identify that no service flow is currently associated with the packet. Additional information can also be provided in the front header, potentially helping application 2021 make appropriate decisions about creating a new service flow. In this example, application 2021 can use network analysis information from P4 network resource 2041 to assign the newly created service flow to a new server. In some embodiments, application 2021 updates its application-specific offload service implementation's table entries to create a service flow on P4 network resource 2041. As previously mentioned, application 202 can optionally provision tables associated with its application-specific offload service implementation.

[0119] In one embodiment, considering that application 2021 has already created table entries for the new service flow in the application-specific offload service implementation deployed on the allocated P4 network resource 2041, packets initially destined for application 2021 on compute / storage resource 13021 can now be redirected toward compute / storage resource 13022 (i.e., when a new service flow would require this, packets matching the newly created service flow are now redirected to compute / storage resource 13022 instead of compute / storage resource 13021). It can also be assumed that service flow information can be provided via packet pre-headers added by the application-specific offload service implementation on P4 network resource 2041.

[0120] As described herein, application 202 can use offloading service 100 to dynamically request the offloading of certain application-specific packet processing logic implementations to P4 network resources 204 available on the associated network infrastructure domain. Offloading service 100 has a system-wide holistic system view of the P4 network resources 204, which allows it to know the associated network topology and detailed information about all P4 network resources 204 available on that network infrastructure, such as their location, the P4 architectures they support, their characteristics, etc. This holistic system view enables offloading service 100 to find the most suitable P4 network resources 204(s) for deploying application-specific packet processing offloading logic implementations, thereby ensuring the most efficient packet processing offloading performance.

[0121] Unloading service 204 provides an unloading service template 110 to application developers. Applications 202 can use the unloading service template 110 to specify their packet processing unloading requirements, including unloading service-specific deployment specifications. Furthermore, as part of the selected unloading service template 110, applications 202 are provided with a P4 architecture, P4 libraries, and P4 program stubs, which application developers can use to implement their own application-specific packet processing unloading logic. Once completed, the application-specific packet processing unloading logic implementation is provided as a P4 program to unloading service 100, which can compile and deploy the P4 program to the most suitable P4 network resource 204.

[0122] For application-specific packet processing offloading logic implementations (which define the data model used to provision their packet processing offloading logic), offloading service 100 allows application 202 to manage and provision their own offloading service logic implementations using a provisioning agent provided by offloading service 100. Before deploying application-specific packet processing offloading logic implementations on the P4 target, offloading service 100 allows application 202 to test their packet processing offloading implementations in a test environment, which aims to verify the proper integration of application 202 with offloading service 100.

[0123] Figures 16A-16E A method 1600 for offloading network operations of a first application 2021 to network resources 204 in a network system 1300, according to an exemplary embodiment, is shown. The operations in the flowcharts will be described with reference to exemplary embodiments of other figures. However, it should be understood that the operations in the flowcharts can be performed by embodiments of the invention other than those discussed with reference to other figures, and that the embodiments of the invention discussed with reference to these other figures can perform operations different from those discussed in the flowcharts.

[0124] like Figure 16AAs shown, method 1600 may begin at operation 1602, wherein uninstall service 100 receives uninstall service discovery message 1002B from first application 2021.

[0125] In operation 1604, in response to the uninstall service discovery message 1002B, uninstall service 100 sends uninstall service discovery message confirmation 1004B to the first application 2021, wherein uninstall service discovery message confirmation 1004B indicates that uninstall service discovery message 1002B has been received and uninstall service 100 exists in network system 1300.

[0126] Alternatively, method 1600 may begin at operation 1606, wherein unloading service 100 receives unloading service discovery message 902B from each of the set of network resources 2041 and 2042.

[0127] In operation 1608, in response to the unload service discovery message 1002B, the unload service 100 transmits an unload service discovery message acknowledgment 904B to each network resource 204 in the set of network resources 2041 and 2042.

[0128] In operation 1610, the uninstallation service 100 receives a network resource registration request 906B from each network resource 204 in the set of network resources 2041 and 2042.

[0129] In operation 1612, the unloading service 100 updates the network resource information model 908B and the network model 910B based on the network resource registration request 906B from each network resource 204 in the set of network resources 2041 and 2042. In some embodiments, the network resource information model 908B describes the characteristics of each network resource 204 in the set of network resources 2041 and 2042, and the network model 910B describes the interconnections between elements in the network system 1300, including the set of network resources 2041 and 2042 and the computing node 13021 on which the first application 2021 is operating.

[0130] In operation 1614, the uninstallation service 100 sends a network resource registration request confirmation 912B to each network resource 204 in the set of network resources 2041 and 2042. The network resource registration request confirmation 912B is used to indicate that each corresponding network resource 204 in the set of network resources 2041 and 2042 has been successfully registered.

[0131] In operation 1616, uninstallation service 100 receives uninstallation service capability request message 1006B from first application 2021. In some embodiments, uninstallation service capability request message 1006B requests uninstallation service 100 for information about the set of capabilities of uninstallation service 100 to satisfy the requirements provided by first application 2021.

[0132] In operation 1618, the uninstallation service 100 transmits an uninstallation service capability request message response 1008B to the first application 2021, wherein the uninstallation service capability request message response 1008B includes a set of uninstallation service templates 110, which are: (1) selected by the uninstallation service 100 based on a set of requirements of the first application 2021 and (2) are possible templates that will be modified to perform network operations of the first application 2021. In some embodiments, each template in the set of uninstallation service templates 110 includes one or more of the following: a list of parameters that will be modified to perform network operations of the first application 2021, a set of code stubs for generating program code for performing network operations of the first application 2021, and a set of libraries for generating program code. In one embodiment, network resource 204 is P4 network resource 204, and program code is P4 program code.

[0133] In operation 1620, uninstallation service 100 receives application uninstallation service request 1010B from first application 2021, which includes templates from the set of templates 110 that are populated for performing network operations of first application 2021.

[0134] In operation 1622, the uninstallation service 100 evaluates network resources 204 for the program code of the first application 2021 to select a set of network resources 2041 and 2042 for installing the program code and uninstalling the operation of the first application 2021 to the set of network resources 2041 and 2042.

[0135] In operation 1624, uninstallation service 100 transmits application uninstallation service request confirmation 1014B, which indicates that a set of network resources 204 for hosting program code of the first application 2021 has been found and that the program code has been appropriately modified to fit the set of network resources 2041 and 2042.

[0136] In operation 1626, uninstallation service 100 receives application uninstallation deployment request 1016B from first application 2021. Application uninstallation deployment request 1016B is used to start the uninstallation operation.

[0137] In operation 1628, the uninstallation service 100 installs program code of the first application 2021 generated based on the set of uninstallation service template 110 on the set of network resources 2041 and 2042, such that the set of network resources 2041 and 2042 processes one or more of the following: (1) packets addressed to the first application 2021 from the second application 2022 and (2) packets addressed to the second application 2022 from the first application 2021. Specifically, the installation program code on the set of network resources 2041 and 2042 may include the uninstallation service 100: (1) in sub-operation 1628A, transmitting an uninstallation service deployment request 1018B including the program code to the set of network resources 2041 and 2042 so that each network resource 204 in the set of network resources 2041 and 2042 installs the program code; and (2) in sub-operation 1628B, after the program code is successfully installed by the set of network resources 2041 and 2042, receiving an uninstallation service deployment request acknowledgment 1022B from each network resource 204 in the set of network resources 2041 and 2042.

[0138] In operation 1630, the uninstallation service 100 transmits an application uninstallation deployment request confirmation 1024B to the first application 2021. The application uninstallation deployment request confirmation 1024B is used to confirm the deployment of program code. In one embodiment, the interaction with the first application 2021 and the second application 2022 is performed via the application uninstallation service agent 118 of the uninstallation service 100. The application uninstallation service agent 118 allows the first application 2021 to manage, configure, provision, and communicate with the uninstallation service 100. In some embodiments, packets addressed to the first application 2021 are either (1) processed by a set of network resources 2041 and 2042 and then forwarded to the application uninstallation service agent 118 before reaching the first application 2021; or (2) transmitted to the application uninstallation service agent 118 and forwarded to a set of network resources 2041 and 2042 for processing before reaching the first application 2021.

[0139] In operation 1632, the uninstallation service 100 receives from the first application 2021 a set of control packets for one or more of the following: managing, configuring, and providing program code installed on the set of network resources 2041 and 2042.

[0140] In operation 1634, uninstall service 100 receives application uninstall service termination request 1102B from first application 2021.

[0141] In operation 1636, the unloading service 100 determines that the set of network resources 2041 and 2042 is allocated to the first application 2021.

[0142] In operation 1638, the uninstall service 100 sends an uninstall service termination request 1106B to each of the network resources 204 in the set of network resources 2041 and 2042.

[0143] In operation 1640, uninstall service 100 receives uninstall service termination request confirmation 1110B, which indicates that the set of network resources 2041 and 2042 will no longer process program code.

[0144] In operation 1642, service 100 updates the network resource information model to indicate the termination of the set of network resources 2041 and 2042.

[0145] In operation 1644, uninstallation service 100 sends an application uninstallation service termination request 1112B to the first application. The application uninstallation service termination request 1112B is used to indicate that the operation of uninstalling the first application 2021 by uninstallation service 100 has been terminated.

[0146] In operation 1646, the uninstallation service 100 receives a network resource deregistration request 1202B from network resource 2041 in the set of network resources 2041 and 2042.

[0147] In operation 1648, uninstallation service 100 sends an unsolicited application uninstallation service termination request 1204B to first application 2021.

[0148] In operation 1650, uninstallation service 100 receives an unsolicited application uninstallation service termination request confirmation 1208B. The unsolicited application uninstallation service termination request confirmation 1208B is used to indicate that: the first application 2021 has performed a switching process to deal with network resources 2041 no longer processing program code.

[0149] In operation 1652, the service 100 is uninstalled to update the network model of the network system 1300 in order to address the fact that network resources 2041 are no longer available for the uninstallation operation.

[0150] In operation 1654, the uninstallation service 100 sends a network resource deregistration request confirmation 1212B to network resource 2041.

[0151] Electronic devices use machine-readable storage media (e.g., disks, optical discs, solid-state drives, read-only memory (ROM), flash memory devices, phase-change memory) and machine-readable transmission media (also known as carrier waves) (e.g., electrical, optical, radio, acoustic, or other forms of propagated signals, such as carrier waves, infrared signals) to store and (internally and / or via a network with other electronic devices) code (consisting of software instructions, and sometimes referred to as computer program code or computer program) and / or data. Therefore, an electronic device (e.g., a computer) includes hardware and software such as a collection of one or more processors coupled to one or more machine-readable storage media (e.g., where the processor is a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application-specific integrated circuit, field-programmable gate array, other electronic circuitry, or a combination of one or more of the foregoing), with one or more machine-readable storage media used to store code for execution on the collection of processors and / or for storing data. For example, an electronic device may include non-volatile memory containing code, because even when the electronic device is off (when power is off), the non-volatile memory can still hold the code / data, while when the electronic device is on, the portion of the code that will be executed by one or more processors of the electronic device is typically copied from the slower non-volatile memory to the volatile memory of the electronic device (e.g., dynamic random access memory (DRAM), static random access memory (SRAM)). A typical electronic device also includes a set of one or more physical network interfaces (NIs) for establishing network connections with other electronic devices (to transmit and / or receive code and / or data using propagation signals). For example, the set of physical NIs (or the set of one or more physical NIs combined with a set of processors executing code) can perform any formatting, encoding, or translation to allow the electronic device to send and receive data, whether via wired and / or wireless connections. In some embodiments, physical NIs may include radio circuitry capable of receiving data from other electronic devices via a wireless connection and / or transmitting data to other devices via a wireless connection. Such a radio circuit may include one or more transmitters, one or more receivers, and / or one or more transceivers suitable for radio frequency communication. The radio circuit can convert digital data into radio signals with suitable parameters (e.g., frequency, timing, channel, bandwidth, etc.). The radio signals can then be transmitted via an antenna to one or more suitable receivers. In some embodiments, the collection of one or more physical NIs may include one or more network interface controllers (NICs) (also known as network interface cards, network adapters, or local area network (LAN) adapters).One or more NICs facilitate the connection of electronic devices to other electronic devices, allowing them to communicate via wires by plugging a cable into a physical port connected to the NIC. One or more portions of embodiments of the invention may be implemented using different combinations of software, firmware, and / or hardware.

[0152] A network device (ND) is an electronic device that is communicatively interconnected with other electronic devices on a network (e.g., other network devices, end-user devices). Some network devices are "multi-service network devices," which support multiple networking functions (e.g., routing, bridging, switching, Layer 2 aggregation, session boundary control, quality of service, and / or subscriber management) and / or multiple application services (e.g., data, voice, and video).

[0153] Figure 17A The present invention illustrates connectivity between network devices (NDs) within an exemplary network and three exemplary implementations of the NDs, according to some embodiments thereof. Figure 17A The diagram shows NDs 1700A-H, and their connectivity is illustrated by lines between 1700A-1700B, 1700B-1700C, 1700C-1700D, 1700D-1700E, 1700E-1700F, 1700F-1700G, and 1700A-1700G, as well as lines between 1700H and each of 1700A, 1700C, 1700D, and 1700G. These NDs are physical devices, and the connectivity between them can be wireless or wired (often referred to as links). The additional lines extending from NDs 1700A, 1700E, and 1700F illustrate that these NDs act as entry and exit points for the network (and therefore, these NDs are sometimes called edge NDs; while the others may be called core NDs).

[0154] Figure 17A Two exemplary ND implementations in the example are: 1) a dedicated network device 1702 using a custom application-specific integrated circuit (ASIC) and a dedicated operating system (OS); and 2) a general-purpose network device 1704 using a general-purpose off-the-shelf (COTS) processor and a standard OS.

[0155] The dedicated network device 1702 includes networking hardware 1710, which includes: a collection of one or more processors 1712, one or more forwarding resources 1714 (typically including one or more ASICs and / or network processors), and one or more physical network interfaces (NIs) 1716 (through which network connections are established, such as those shown in the connectivity diagram between ND 1700A-H), and a non-transitory machine-readable storage medium 1718 therein storing networking software 1720. During operation, the networking software 1720 can be executed by the networking hardware 1710 to instantiate a collection of one or more networking software instances 1722. Each of the one or more networking software instances 1722 and the portion of the networking hardware 1710 that executes that networking software instance (whether it is hardware dedicated to that networking software instance and / or a time slice of hardware shared in time by that networking software instance with other networking software instances in the one or more networking software instances 1722) form separate virtual network elements 1730A-R. Each of the (one or more) virtual network elements (VNEs) 1730A-R includes a control communication and configuration module 1732A-R (sometimes referred to as a local control module or control communication module) and (one or more) forwarding tables 1734A-R, such that a given virtual network element (e.g., 1730A) includes: a collection of control communication and configuration modules (e.g., 1732A), one or more forwarding tables (e.g., 1734A), and the portion of the networking hardware 1710 that executes the virtual network element (e.g., 1730A).

[0156] The dedicated network device 1702 is often considered, physically and / or logically, to include: 1) the ND control plane 1724 (sometimes referred to as the control plane), which includes one or more processors 1712 that perform control communication and configuration modules 1732A-R; and 2) the ND forwarding plane 1726 (sometimes referred to as the forwarding plane, data plane, or media plane), which includes one or more forwarding resources 1714 and physical NI 1716 that utilize forwarding tables 1734A-R. For example, when the ND is a router (or is implementing routing functionality), the ND control plane 1724 (one or more processors 1712 that perform control communication and configuration modules 1732A-R) is typically responsible for participating in controlling how data (e.g., packets) will be routed (e.g., the next hop of the data and the physical NI that the data goes out to) and storing that routing information in one or more forwarding tables 1734A-R, and the ND forwarding plane 1726 is responsible for receiving the data on physical NI 1716 and forwarding the data out to the appropriate physical NI in physical NI 1716 based on one or more forwarding tables 1734A-R.

[0157] Figure 17B Exemplary methods for implementing a dedicated network device 1702 according to some embodiments of the present invention are shown. Figure 17B A dedicated network device including card 1738 (typically hot-swappable) is shown. Although in some embodiments, card 1738 has two types (one or more cards operating as ND forwarding plane 1726 (sometimes called line cards) and one or more cards operating to implement ND control plane 1724 (sometimes called control cards)), alternative embodiments may combine functionality onto a single card and / or include additional card types (e.g., an additional type of card called a service card, resource card, or multi-application card). Service cards can provide specialized processing (e.g., Layer 4 through Layer 7 services such as firewalls, Internet Protocol Security (IPsec), Secure Sockets Layer (SSL) / Transport Layer Security (TLS), Intrusion Detection Systems (IDS), Peer-to-Peer (P2P) networks, Voice over IP (VoIP) session border controllers, mobile wireless gateways (Gateway General Packet Radio Service (GPRS) Support Node (GGSN), Evolved Packet Core (EPC) gateways)). For example, service cards can be used to terminate IPsec tunnels and perform accompanying authentication and encryption algorithms. These cards are coupled together via one or more interconnect mechanisms shown as backplane 1736 (e.g., a first full-mesh coupling line card, and a second full-mesh coupling all cards).

[0158] Return to Figure 17AThe general-purpose network device 1704 includes hardware 1740, which includes: a collection of one or more processors 1742 (often COTS processors) and physical NI 1746, and a non-transitory machine-readable storage medium 1748 therein storing software 1750 and / or offloading services 100. During operation, the processors 1742 execute software 1750 to instantiate one or more collections of one or more applications 1764A-R. Although one embodiment does not implement virtualization, alternative embodiments may use different forms of virtualization. For example, in one such alternative embodiment, virtualization layer 1754 represents the kernel of an operating system (or a pad that executes on a base operating system) that allows the creation of multiple instances 1762A-R called software containers, each of which can be used to execute one or more sets of applications 1764A-R; wherein the multiple software containers (also known as virtualization engines, virtual private servers, or prisons) are separate from each other and from the kernel space in which the operating system runs; and wherein, unless explicitly permitted, the set of applications running in a given user space cannot access the memory of other processes. In another such alternative embodiment, virtualization layer 1754 represents a hypervisor (sometimes called a virtual machine monitor (VMM)) or a hypervisor that executes on top of a host operating system, and each set of applications 1764A-R runs on top of a guest operating system within an instance 1762A-R called a virtual machine (in some cases, it can be viewed as a strictly isolated form of software container)—the guest operating system and applications may not be aware that they are running on a virtual machine compared to running on a "bare metal" host electronic device, or through paravirtualization, the operating system and / or applications may be aware that virtualization exists for optimization purposes. In other alternative embodiments, one, some, or all of the applications are implemented as one or more single kernels, which can be generated by utilizing a limited set of libraries that provide the specific OS services required by the application, which are only directly compiled by the application (e.g., from a Library Operating System (LibOS) that includes drivers / libraries for OS services). Since a single kernel can be implemented to run directly on hardware 1740, directly on a hypervisor (in this case, the single kernel is sometimes described as running within a LibOS virtual machine), or in a software container, embodiments can be implemented entirely using a single kernel running directly on a hypervisor represented by virtualization layer 1754, a single kernel running within a software container represented by instances 1762A-R, or as a combination of a single kernel and the above techniques (e.g., a single kernel and a virtual machine both running directly on a hypervisor, a single kernel and a collection of applications running in different software containers).

[0159] The instantiation and virtualization (if implemented) of one or more sets of applications 1764A-R are collectively referred to as one or more software instances 1752. Each set of applications 1764A-R, its corresponding virtualization construct (e.g., instance 1762A-R) (if implemented), and the portion of hardware 1740 that executes them (whether it is dedicated hardware for that execution and / or a time slice of hardware shared in time) form one or more separate virtual network elements 1760A-R.

[0160] One or more virtual network elements 1760A-R perform functionality similar to (one or more) virtual network elements 1730A-R (e.g., similar to (one or more) control communication and configuration modules 1732A and (one or more) forwarding tables 1734A) (this virtualization of hardware 1740 is sometimes referred to as Network Functions Virtualization (NFV)). Therefore, NFV can be used to integrate many network device types into industry-standard high-capacity server hardware, physical switches, and physical storage devices, which may reside in data centers, NDs, and customer premises equipment (CPEs). Although embodiments of the invention are shown with each instance 1762A-R corresponding to a VNE 1760A-R, alternative embodiments may implement this correspondence with a finer level of granularity (e.g., line card virtualization of line cards, control card virtualization of control cards, etc.); it should be understood that the techniques described herein with reference to the correspondence between instance 1762A-R and VNE are also applicable to embodiments in which such finer-level granularity and / or a single core are used.

[0161] In some embodiments, the virtualization layer 1754 includes a virtual switch that provides forwarding services similar to those of a physical Ethernet switch. Specifically, the virtual switch forwards traffic between instances 1762A-R and(one or more) physical NI 1746 and optionally between instances 1762A-R; additionally, the virtual switch may implement network isolation between VNE 1760A-R, disallowing these VNE 1760A-R from communicating with each other by policy (e.g., by honoring Virtual Local Area Networks (VLANs)).

[0162] Figure 17A The third exemplary ND implementation is a hybrid network device 1706, which includes a custom ASIC / dedicated OS and a COTS processor / standard OS in a single card within a single ND or ND. In some embodiments of such a hybrid network device, a platform VM (i.e., a VM that implements the functionality of the dedicated network device 1702) can provide paravirtualization to the networking hardware present in the hybrid network device 1706.

[0163] Regardless of the above exemplary implementations of the ND, when considering a single VNE among multiple VNEs implemented by the ND (e.g., only one VNE is part of a given virtual network), or in cases where the ND is currently implementing only a single VNE, the shortened term Network Element (NE) is sometimes used to refer to that VNE. And in all the above exemplary implementations, each VNE (e.g., the VNE in (one or more) VNEs 1730A-R, VNE 1760A-R, and the hybrid network device 1706) receives data on a physical NI (e.g., 1716, 1746) and forwards that data outward to the appropriate physical NI in the physical NI (e.g., 1716, 1746). For example, a VNE that implements the functionality of an IP router forwards IP packets based on a portion of the IP header information in the IP packet; wherein the IP header information includes the source IP address, destination IP address, source port, destination port (where “source port” and “destination port” refer to protocol ports in this document, not the physical ports of the ND), transport protocol (e.g., User Datagram Protocol (UDP), Transmission Control Protocol (TCP)) and Differential Service Code Point (DSCP) value.

[0164] Figure 17C Various exemplary methods of coupling a VNE according to some embodiments of the present invention are shown. Figure 17C The diagram shows VNE 1770A.1-1770A.P (and optional VNE 1770A.Q-1770A.R) implemented in ND 1700A and VNE 1700H.1 implemented in ND1700H. Figure 17C In this context, VNEs 1770A.1-P are separate from each other in the sense that they can receive packets from outside ND 1700A and forward packets to outside ND 1700A; VNE 1770A.1 is coupled to VNE 1770H.1, and therefore, they pass packets between their respective NDs; VNEs 1770A.2-1770A.3 may optionally forward packets between themselves without forwarding them to outside ND 1700A; and VNE 1770A.P may optionally be the first VNE in a VNE chain (sometimes called a dynamic service link, where each VNE in this series of VNEs provides a different service, such as one or more Layer 4-7 network services), the VNE chain including: VNE 1770A.Q, followed by VNE 1770A.R. Although Figure 17C Various exemplary relationships between VNEs are shown, but alternative embodiments may support other relationships (e.g., more / fewer VNEs, more / fewer dynamic service chains, multiple different dynamic service chains with some common VNEs and some different VNEs).

[0165] Figure 17A The ND may, for example, form part of the Internet or a private network; and other electronic devices (not shown; such as end-user devices, including workstations, laptops, netbooks, tablets, handheld computers, mobile phones, smartphones, tablet phones, multimedia phones, Voice over Internet Protocol (VoIP) phones, terminals, portable media players, GPS units, wearable devices, gaming systems, set-top boxes, and Internet-enabled home appliances) may be coupled to the network (directly or via other networks, such as access networks) to communicate with each other (directly or via servers) and / or access content and / or services through the network (e.g., the Internet or a Virtual Private Network (VPN) laid on the Internet (e.g., tunneling the Internet)). Such content and / or services are typically provided by one or more servers (not shown) belonging to a service / content provider or one or more end-user devices (not shown) participating in a peer-to-peer (P2P) service, and may include, for example, public web pages (e.g., free content, storefronts, search services), private web pages (e.g., web pages accessed via username / password for email services), and / or corporate networks via VPNs. For example, end-user devices can be coupled (e.g., via (wired or wireless) to customer premises equipment accessing the network) to edge NDs, which in turn are coupled (e.g., via one or more core NDs) to other edge NDs, which in turn are coupled to electronic devices acting as servers. However, through compute and storage virtualization, as Figure 17A One or more electronic devices operating in the ND can also host one or more such servers (e.g., in the case of general network device 1704, one or more software instances 1762A-R can operate as servers; the same is true for hybrid network device 1706; in the case of dedicated network device 1702, one or more such servers can also run on a virtualization layer executed by one or more processors 1712); in this case, the server is alleged to be located in the same location as the VNE of the ND.

[0166] A virtual network is a physical network (such as an L2 and / or L3 network) that provides network services (e.g., L2 and / or L3 services). Figure 17A Virtual networks are logical abstractions of physical networks. Virtual networks can be implemented as overlay networks (sometimes called network virtualization overlays), which provide network services (e.g., Layer 2 (L2, data link layer) and / or Layer 3 (L3, network layer) services) through underlying networks (e.g., L3 networks, such as Internet Protocol (IP) networks that use tunnels (e.g., Generic Routing Encapsulation (GRE), Layer 2 Tunneling Protocol (L2TP), IPSec) to create overlay networks).

[0167] A Network Virtualization Edge (NVE) is located at the edge of the underlying network and participates in implementing network virtualization. The network-facing side of an NVE uses the underlying network to transmit frames to and from other NVEs via tunnels. The NVE also sends data to and receives data from systems outside the network. A Virtual Network Instance (VNI) is a specific instance of a virtual network on an NVE (e.g., a NE / VNE on the ND, or a portion of a NE / VNE on the ND, where the NE / VNE is divided into multiple VNEs through emulation). One or more VNIs can be instantiated on an NVE (e.g., as different VNEs on the ND). A Virtual Access Point (VAP) is a logical connection point on an NVE used to connect external systems to the virtual network; a VAP can be a physical port or a virtual port identified by a logical interface identifier (e.g., a VLAN ID).

[0168] Examples of network services include: 1) Ethernet LAN emulation services (Ethernet-based multipoint services similar to Internet Engineering Task Force (IETF) Multiprotocol Label Switching (MPLS) or Ethernet VPN (EVPN) services), where external systems interconnect across the underlying network via a LAN environment (e.g., NVE provides separate L2 VNIs (virtual switching instances) for different such virtual networks and provides L3 (e.g., IP / MPLS) tunnel encapsulation across the underlying network); and 2) virtualized IP forwarding services (from a service definition perspective, this is similar to IETF IP VPN (e.g., Border Gateway Protocol (BGP) / MPLS IPVPN)), where external systems interconnect across the underlying network via an L3 environment (e.g., NVE provides separate L3 VNIs (forwarding and routing instances) for different such virtual networks and provides L3 (e.g., IP / MPLS) tunnel encapsulation across the underlying network). Network services may also include quality of service capabilities (e.g., traffic classification and labeling, traffic conditioning and scheduling), security capabilities (e.g., filters to protect customer premises from network-initiated attacks, to prevent malformed route advertisements), and management capabilities (e.g., comprehensive detection and handling).

[0169] Figure 17D The invention illustrates some embodiments of the invention. Figure 17A The network consists of a single network element on each ND in the ND, and within this direct approach, the traditional distributed method (typically used by conventional routers) for maintaining reachability and forwarding information (also known as network control) is compared with a centralized method for maintaining reachability and forwarding information. Specifically, Figure 17D The network element (NE) 1770A-H is shown, which has the same connectivity as the ND 1700A-H of 17A.

[0170] Figure 17D As shown, distributed method 1772 distributes the responsibilities for generating reachability and forwarding information across NE 1770A-H; in other words, the neighbor discovery and topology discovery processes are distributed.

[0171] For example, when using a dedicated network device 1702, the control communication and configuration modules 1732A-R of the ND control plane 1724 typically include a reachability and forwarding information module, which is used to implement one or more routing protocols (e.g., Exterior Gateway Protocols such as Border Gateway Protocol (BGP), Interior Gateway Protocols (IGPs) (e.g., Open Shortest Path First (OSPF), Intermediate System to Intermediate System (IS-IS), Routing Information Protocol (RIP), Label Distribution Protocol (LDP), Resource Reservation Protocol (RSVP) (including RSVP-Service Engineering (TE): an extension of RSVP to LSP tunnels and General Multiprotocol Label Switching (GMPLS) signaling RSVP-TE)), one or more routing protocols communicate with other NEs to exchange routes, and then select those routes based on one or more routing metrics. Therefore, the NE... The 1770A-H (e.g., one or more processors 1712 that execute one or more control communication and configuration modules 1732A-R) perform their duties of participating in control of how data (e.g., packets) will be routed (e.g., the next hop of the data and the physical NI of the data's outgoing traffic) by distributedly determining reachability within the network and calculating their respective forwarding information. Routes and adjacency relationships are stored in one or more routing structures (e.g., a routing information library (RIB), a label information library (LIB), one or more adjacency structures) on the ND control plane 1724. The ND control plane 1724 utilizes routing structures (one or more) as a basis. Information (e.g., adjacency and route information) is programmed into the ND forwarding plane 1726. For example, the ND control plane 1724 programs adjacency and route information into one or more forwarding tables 1734A-R (e.g., a forwarding information library (FIB), a label forwarding information library (LFIB), and one or more adjacency structures) on the ND forwarding plane 1726. For Layer 2 forwarding, the ND may store one or more bridging tables for forwarding the data based on the Layer 2 information in the data. Although the above example uses a dedicated network device 1702, the same distributed approach 1772 can also be implemented on a general-purpose network device 1704 and a hybrid network device 1706.

[0172] Figure 17DThe diagram illustrates a centralized approach 1774 (also known as Software-Defined Networking (SDN)) that decouples the system that makes decisions about where to send traffic from the underlying system that forwards traffic to the chosen destination. The centralized approach 1774 has the responsibility of generating reachability and forwarding information in a centralized control plane 1776 (sometimes referred to as an SDN control module, controller, network controller, OpenFlow controller, SDN controller, control plane node, network virtualization authority, or management control entity), and therefore, the neighbor discovery and topology discovery processes are centralized. The centralized control plane 1776 has a southbound interface 1782 with the data plane 1780 (sometimes referred to as the infrastructure layer, network forwarding plane, or forwarding plane (which should not be confused with the ND forwarding plane)), which includes NE 1770A-H (sometimes referred to as switches, forwarding elements, data plane elements, or nodes). The centralized control plane 1776 includes a network controller 1778, which includes a centralized reachability and forwarding information module 1779 and / or offloading service 100. The centralized reachability and forwarding information module 1779 determines reachability within the network and distributes forwarding information to NE 1770A-H of the data plane 1780 via a southbound interface 1782 (which can use the OpenFlow protocol). Therefore, network intelligence is centralized in the centralized control plane 1776, which is executed on an electronic device typically separate from the ND.

[0173] For example, in the case of using a dedicated network device 1702 in the data plane 1780, each of the control communication and configuration modules 1732A-R in the ND control plane 1724 typically includes a control agent that provides the VNE side of the southbound interface 1782. In this context, the ND control plane 1724 (which executes one or more processors 1712 of control communication and configuration modules 1732A-R) communicates with the centralized control plane 1776 via a control agent to receive forwarding information from the centralized reachability and forwarding information module 1779 (and in some cases, reachability information) to perform its duties of participating in how data (e.g., packets) will be routed (e.g., the next hop of the data and the physical NI of the data's outgoing path). (It should be understood that in some embodiments of the invention, in addition to communicating with the centralized control plane 1776, one or more control communication and configuration modules 1732A-R may also play a role in determining reachability and / or calculating forwarding information—albeit less than in the case of a distributed approach; such embodiments are generally considered to be part of the centralized approach 1774, but they may also be considered hybrid approaches.)

[0174] Although the above example uses a dedicated network device 1702, the same centralized approach 1774 can be implemented using a general-purpose network device 1704 (e.g., each VNE in VNE 1760A-R performs its duties of controlling how data (e.g., packets) is routed (e.g., the next hop of the data and the physical NI of the data's outgoing path) by communicating with a centralized control plane 1776 to receive forwarding information from a centralized reachability and forwarding information module 1779 (and in some cases, reachability information); it should be understood that in some embodiments of the invention, VNE 1760A-R may also play a role in determining reachability and / or calculating forwarding information in addition to communicating with the centralized control plane 1776—albeit less than in the case of a distributed approach) and a hybrid network device 1706. In fact, the use of SDN technology can enhance the NFV technology typically used in implementations of general-purpose network device 1704 or hybrid network device 1706, because NFV is able to support SDN by providing an infrastructure on which SDN software can run, and both NFV and SDN are designed to utilize commercial server hardware and physical switches.

[0175] Figure 17D It is also shown that the centralized control plane 1776 has a northbound interface 1784 to the application layer 1786 in which one or more applications 1788 reside. The centralized control plane 1776 has the capability to form virtual networks 1792 (sometimes referred to as logical forwarding planes, network services, or overlay networks, where NEs 1770A-H of the data plane 1780 are the underlying networks) for the applications 1788. Therefore, the centralized control plane 1776 maintains a global view of all NDs and configured NEs / VNEs, and it effectively maps virtual networks to the underlying NDs (including maintaining these mappings when the physical network changes due to hardware (ND, link, or ND component) failures, additions, or removals).

[0176] although Figure 17DThe distributed approach 1772 and the centralized approach 1774 are shown separately, but in some embodiments of the invention, network control efforts may be distributed differently or a combination of the two may be used. For example: 1) Embodiments may generally use the centralized approach (SDN) 1774, but delegate certain functions to the NE (e.g., the distributed approach may be used to implement one or more of the following: fault monitoring, performance monitoring, protection switching, and primitives for neighbor and / or topology discovery); or 2) Embodiments of the invention may perform neighbor discovery and topology discovery via a centralized control plane and distributed protocols, and compare the results to raise anomalies if they are inconsistent. Such embodiments are generally considered to be part of the centralized approach 1774, but they may also be considered hybrid approaches.

[0177] although Figure 17D This illustrates a simplified case where each ND in ND 1700A-H implements a single NE 1770A-H, but it should be understood that reference... Figure 17D The described network control method also works for networks in which one or more NDs in ND 1700A-H implement multiple VNEs (e.g., VNE 1730A-R, VNE 1760A-R, VNE in hybrid network device 1706). Alternatively or additionally, network controller 1778 can also emulate the implementation of multiple VNEs in a single ND. Specifically, instead of (or supplementing) implementing multiple VNEs in a single ND, network controller 1778 can present the implementation of VNE / NE in a single ND as multiple VNEs in virtual network 1792 (all VNEs in the same virtual network in one or more virtual networks 1792, each VNE in a different virtual network in one or more virtual networks 1792, or some combination thereof). For example, network controller 1778 may enable ND to implement a single VNE (NE) in the underlying network and then logically partition the resources of that NE within the centralized control plane 1776 to present different VNEs in one or more virtual networks 1792 (wherein, these different VNEs in the overlay network are sharing the resources of the single VNE / NE implementation on the ND in the underlying network).

[0178] on the other hand, Figure 17E and Figure 17F Exemplary abstractions of NE and VNE, which can be presented by network controller 1778 as part of different virtual networks in virtual network 1792, are shown respectively. Figure 17E The following simplified case illustrates some embodiments of the invention: wherein each ND in ND 1700A-H implements a single NE 1770A-H (see Figure 17DHowever, the centralized control plane 1776 has abstracted multiple NEs (NE 1770A-C and GH) in different NDs into (abstracted to represent) Figure 17D A single NE 1770I in one of (one or more) virtual networks 1792. Figure 17E As shown, in this virtual network, NE1770I is coupled to NE 1770D and 1770F, while NE 1770D and 1770F are still coupled to NE 1770E.

[0179] Figure 17F The following situation is illustrated according to some embodiments of the invention: wherein multiple VNEs (VNE 1770A.1 and VNE 1770H.1) are implemented on different NDs (ND 1700A and ND 1700H) and these VNEs are coupled to each other, and wherein the centralized control plane 1776 has abstracted these multiple VNEs so that they behave as Figure 17D A single VNE 1770T within one of the virtual networks 1792. Therefore, the abstraction of an NE or VNE can span multiple NDs.

[0180] While some embodiments of the invention implement the centralized control plane 1776 as a single entity (e.g., a single software instance running on a single electronic device), alternative embodiments may distribute this functionality across multiple entities (e.g., multiple software instances running on different electronic devices) for redundancy and / or scalability purposes.

[0181] Similar to network device implementations, the electronic devices(s) running the centralized control plane 1776 and thus the network controller 1778 including the centralized reachability and forwarding information module 1779 can be implemented in a variety of ways (e.g., dedicated devices, general-purpose (e.g., COTS) devices, or hybrid devices). These electronic devices will similarly include one or more processors(s), a collection of one or more physical NIs, and a non-transitory machine-readable storage medium on which the centralized control plane software is stored. For example, Figure 18 A general-purpose control plane device 1804 is shown, including hardware 1840, which includes: a collection of one or more processors 1842 (which are often COTS processors) and physical NI 1846, and a non-transitory machine-readable storage medium 1848 in which centralized control plane (CCP) software 1850 and / or offloading services 100 are stored.

[0182] In embodiments using compute virtualization, processor(s) 1842 typically execute software to instantiate virtualization layer 1854 (e.g., in one embodiment, virtualization layer 1854 represents the kernel of an operating system (or a pad executing on a base operating system), which allows the creation of multiple instances 1862A-R called software containers (representing separate user spaces and also known as virtualization engines, virtual private servers, or prisons), each of which can be used to execute a collection of one or more applications; in another embodiment, virtualization layer 1854 represents a hypervisor (sometimes called a virtual machine monitor (VMM)) or a hypervisor executing on top of a host operating system, and applications run on top of guest operating systems within instances 1862A-R called virtual machines (in some cases, which can be viewed as a strictly isolated form of software container) run by the hypervisor; in yet another embodiment, applications are implemented as a single kernel, which can be achieved by leveraging applications that are only directly compiled (e.g., from drivers including OS services / The Library Operating System (LibOS) of the library provides a limited set of libraries for specific OS services required by the application to generate the single kernel, which may run directly on hardware 1840, directly on a hypervisor represented by virtualization layer 1854 (in which case, the single kernel is sometimes described as running within a LibOS virtual machine), or in a software container represented by one of instances 1862A-R. Similarly, in embodiments where compute virtualization is used, during operation, an instance of CCP software 1850 (shown as CCP instance 1876A) is executed on virtualization layer 1854 (e.g., within instance 1862A). In embodiments where compute virtualization is not used, CCP instance 1876A is executed on a "bare metal" general control plane device 1804, either as a single kernel or on top of the host operating system. The instantiation of CCP instance 1876A, along with virtualization layer 1854 and instances 1862A-R (if implemented), are collectively referred to as software instances 1852(one or more).

[0183] In some embodiments, CCP instance 1876A includes network controller instance 1878. Network controller instance 1878 includes centralized reachability and forwarding information module instance 1879 (which is a middleware layer that provides the operating system with the context of network controller 1778 and communicates with various NEs) and above this middleware layer, CCP application layer 1880 (sometimes referred to as the application layer) (providing the intelligence required for various network operations, such as protocols, network situational awareness, and user interface). At a more abstract level, the CCP application layer 1880 within the centralized control plane 1776 works with one or more virtual network views (one or more logical views of the network), and the middleware layer provides the transformation from virtual network to physical view.

[0184] The centralized control plane 1776 computes and middleware layer maps relevant messages to the data plane 1780 based on the CCP application layer 1880 for each flow. A flow can be defined as a set of packets whose headers match a given bit pattern; in this sense, traditional IP forwarding is also flow-based forwarding, where the flow is defined by, for example, a destination IP address; however, in other implementations, the given bit pattern used for flow definition can include more fields (e.g., 10 or more) in the packet header. Different ND / NE / VNEs of the data plane 1780 can receive different messages and therefore different forwarding information. The data plane 1780 processes these messages and programs the appropriate flow information and corresponding actions into the forwarding table (sometimes called a flow table) of the appropriate NE / VNE. The NE / VNE then maps incoming packets to the flows represented in the forwarding table and forwards packets based on the matches in the forwarding table.

[0185] Standards such as OpenFlow define the protocols used for these messages and the models used to process these packets. The models for processing packets include header parsing, packet classification, and making forwarding decisions. Header parsing describes how packets are interpreted based on a set of well-known protocols. Several protocol fields are used to construct a matching structure (or keywords) that will be used for packet classification (e.g., the first keyword field could be the source Media Access Control (MAC) address, and the second keyword field could be the destination MAC address).

[0186] Packet classification involves performing a lookup in memory to determine which entry in the forwarding table (also known as a forwarding table entry or flow entry) best matches the packet based on the matching structure or keywords of the forwarding table entry. It's possible that many flows represented in a forwarding table entry can correspond to / match a packet; in this case, the system is typically configured to determine a forwarding table entry from among numerous forwarding table entries according to a defined scheme (e.g., selecting the first matching forwarding table entry). A forwarding table entry includes a set of specific matching criteria (a set of values ​​or wildcards, or an indication of what part of the packet should be compared with one or more specific values / wildcards, such as those defined by matching capabilities—for a specific field in the packet header or for some other packet content) and a set of one or more actions the data plane takes upon receiving a matching packet. For example, actions could include: pushing a header onto the packet, forwarding the packet using a specific port, flooding the packet, or simply discarding the packet. Therefore, a forwarding table entry for an IPv4 / IPv6 packet with a specific Transmission Control Protocol (TCP) destination port might contain actions specifying that these packets should be discarded.

[0187] Forwarding decisions and actions are made based on the forwarding table entries identified during group classification by performing a set of actions identified in the matching forwarding table entries on the group.

[0188] However, when an unknown packet (e.g., a "missing packet" or "match not found" as used in OpenFlow terminology) arrives at data plane 1780, the packet (or a subset of its header and content) is typically forwarded to centralized control plane 1776. Centralized control plane 1776 then programs forwarding table entries into data plane 1780 to accommodate packets belonging to the unknown packet flow. Once a specific forwarding table entry has been programmed into data plane 1780 by centralized control plane 1776, the next packet with a matching certificate will be matched against that forwarding table entry and will take the set of actions associated with that matching entry.

[0189] A network interface (NI) can be physical or virtual; and in the context of IP, an interface address is an IP address assigned to an NI, regardless of whether it is a physical or virtual NI. A virtual NI can be associated with a physical NI, can be associated with another virtual interface, or can exist independently (e.g., a loopback interface, a point-to-point protocol interface). NIs (physical or virtual) can be numbered (NIs with IP addresses) or unnumbered (NIs without IP addresses). A loopback interface (and its loopback address) is a specific type of virtual NI (and IP address) frequently used for management purposes for NEs / VNEs (physical or virtual); such IP addresses are called node loopback addresses. The IP addresses assigned to one or more NIs on an ND are called the IP addresses of that ND; at a finer granular level, the IP addresses assigned to one or more NIs on an NE / VNE implemented on an ND may be called the IP addresses of that NE / VNE.

[0190] A routing system's next-hop selection for a given destination can be resolved as a single path (i.e., a routing protocol can generate a next hop on the shortest path); however, if the routing system determines that multiple feasible next hops exist (i.e., the forwarding solution generated by the routing protocol provides more than one next hop on the shortest path—multiple equal-cost next hops), then additional criteria are used—for example, in connectionless networks, equal-cost multipathing (ECMP) (also known as equal-cost multipathing, multipath forwarding, and IP multipathing) can be used (e.g., typical implementations use specific header fields as criteria to ensure that packets in a particular packet flow are always forwarded on the same next hop, thus maintaining packet flow ordering). For the purposes of multipath forwarding, a packet flow is defined as a set of packets sharing ordering constraints. As an example, the set of packets in a particular TCP transmission sequence needs to arrive in order; otherwise, TCP logic will interpret out-of-order delivery as congestion and slow down TCP transmission rates.

[0191] Layer 3 (L3) Link Aggregation (LAG) links are links that directly connect two NDs to multiple IP-addressed link paths (assigning different IP addresses to each link path), and perform load distribution decisions across these different link paths in the ND forwarding plane; in this case, load distribution decisions are made between link paths.

[0192] Some NDs include functionality for authentication, authorization, and accounting (AAA) protocols, such as RADIUS (Remote Authentication Dial-Up Subscriber Service), Diameter, and / or TACACS+ (Terminal Access Controller Access Control System+). AAA can be provided via a client / server model, where the AAA client is implemented on the ND, and the AAA server can be implemented locally on the ND or on a remote electronic device coupled to the ND. Authentication is the process of identifying and verifying a subscriber. For example, a subscriber might be identified by a combination of username and password or by a unique key. Authorization determines what a subscriber can do after being authenticated, such as gaining access to certain informational resources on the electronic device (e.g., through the use of access control policies). Accounting is the recording of user activity. (Summary) For example, the end-user device can be coupled (e.g., via an access network) to an edge ND (supporting AAA processing) coupled to the core ND, which in turn couples to an electronic device that implements the service / content provider's server. AAA processing is performed to identify the subscriber's subscriber record stored in the AAA server. The subscriber record includes a set of attributes used during the processing of the subscriber's business (e.g., subscriber name, password, authentication information, access control information, rate limiting information, regulatory information).

[0193] Some NDs (e.g., some edge NDs) use subscriber circuits to internally represent end-user devices (or sometimes customer premises equipment (CPEs), such as residential gateways (e.g., routers, modems)). A subscriber circuit uniquely identifies a subscriber session within the ND and typically exists for the lifetime of the session. Therefore, when a subscriber connects to an ND, the ND typically assigns a subscriber circuit, and when the subscriber disconnects, the ND correspondingly deassigns the subscriber circuit. Each subscriber session represents a distinguishable stream of packets passed between the ND and the end-user device (or sometimes the CPE, such as a residential gateway or modem) using protocols such as Point-to-Point Protocol (PPPoX) based on another protocol (e.g., where X is Ethernet or Asynchronous Transfer Mode (ATM)), Ethernet, 802.1Q Virtual LAN (VLAN), Internet Protocol, or ATM. Subscriber sessions can be initiated using various mechanisms, such as manually provisioned Dynamic Host Configuration Protocol (DHCP), DHCP / Clientless Internet Protocol Service (CLIPS), or Media Access Control (MAC) address tracking. For example, Point-to-Point Protocol (PPP) is commonly used for Digital Subscriber Line (DSL) services and requires the installation of a PPP client, which allows subscribers to enter a username and password, which in turn are used to select subscriber records. When using DHCP (e.g., for cable modem services), a username is typically not provided; however, in such cases, other information is provided (e.g., information including the MAC address of the hardware in the end-user device (or CPE)). DHCP and CLIPS on the ND use captured MAC addresses and use these addresses to distinguish subscribers and access their subscriber records.

[0194] Virtual circuits (VCs), synonymous with virtual connections and virtual channels, are connection-oriented communication services delivered via packet-switched communication. VC communication is similar to circuit switching because both are connection-oriented, meaning that data is delivered in the correct order in both cases, and signaling overhead is required during the connection establishment phase. VCs can exist at different layers. For example, at Layer 4, connection-oriented transport layer data link protocols such as Transmission Control Protocol (TCP) can rely on connectionless packet-switched network layer protocols such as IP, where different packets can be routed on different paths and therefore delivered out of order. In the case of establishing reliable VCs using TCP over the underlying unreliable and connectionless IP protocol, the VC is identified by the source and destination network socket address pairs (i.e., sender and receiver IP addresses and port numbers). However, VCs are possible because TCP includes segment numbering and reordering on the receiver side to prevent out-of-order delivery. Virtual circuits can also exist at Layer 3 (Network Layer) and Layer 2 (Data Link Layer); such virtual circuit protocols are based on connection-oriented packet switching, meaning that data is always delivered along the same network path (i.e., through the same NE / VNE). In these protocols, packets are not routed individually, and complete addressing information is not provided in the header of each data packet; only a small Virtual Channel Identifier (VCI) is needed in each packet; and routing information is transmitted to the NE / VNE during the connection establishment phase; the switching only involves looking up the VCI in a table, rather than analyzing the complete address. Examples of network layer and data link layer virtual circuit protocols that always deliver data on the same path include: X.25, where VCs are identified by Virtual Channel Identifiers (VCIs); Frame Relay, where VCs are identified by VCIs; Asynchronous Transfer Mode (ATM), where circuits are identified by Virtual Path Identifier (VPI) and Virtual Channel Identifier (VCI) pairs; General Packet Radio Service (GPRS); and Multiprotocol Label Switching (MPLS), which can be used for virtual circuit-based IP (each circuit is identified by a label).

[0195] Some NDs (e.g., some edge NDs) use a circuit hierarchy. The leaf nodes of the circuit hierarchy are subscriber circuits. Subscriber circuits in the hierarchy typically represent aggregations of multiple subscriber circuits, and therefore have network segments and elements for providing access network connectivity to those end-user devices from the ND. These parent circuits can represent physical or logical aggregations of subscriber circuits (e.g., Virtual LANs (VLANs), Permanent Virtual Circuits (PVCs) (e.g., for Asynchronous Transfer Mode (ATM)), circuit groups, channels, pseudowires, physical NIs of the ND, and link aggregation groups). Circuit groups are virtual constructs that allow various sets of circuits to be grouped together for configuration purposes (e.g., aggregation rate control). Pseudowires are simulations of point-to-point, connection-oriented services at Layer 2. Link aggregation groups are virtual constructs that combine multiple physical NIs for bandwidth aggregation and redundancy purposes. Thus, parent circuits physically or logically encapsulate subscriber circuits.

[0196] Each VNE (e.g., a virtual router, a virtual bridge (which can act as a virtual switch instance in Virtual Private LAN Services (VPLS)) is typically managed independently. For example, in the case of multiple virtual routers, each virtual router within a virtual router may share system resources but is separate from the others in terms of its management domain, AAA (Authentication, Authorization, and Accounting) namespace, IP address, and one or more routing databases. Multiple VNEs can be employed in an edge ND to provide direct network access and / or different categories of services to subscribers of service providers and / or content providers.

[0197] Within certain NDs, an "interface" independent of the physical NI can be configured as part of a VNE to provide higher-layer protocol and service information (e.g., Layer 3 addressing). In addition to other subscriber configuration requirements, the subscriber record in the AAA server identifies which context (e.g., which VNE / NE within a VNE / NE) the corresponding subscriber should be bound to within the ND. As used herein, an association is formed between the interface bound to a physical entity (e.g., physical NI, channel) or logical entity (e.g., circuit, such as a subscriber circuit or a logical circuit (a collection of one or more subscriber circuits)) and the context on which a network protocol (e.g., routing protocol, bridging protocol) is configured for that context. When a higher-layer protocol interface is configured and associated with a physical entity, subscriber data flows over that physical entity.

[0198] Some NDs provide support for implementing VPNs (Virtual Private Networks) (e.g., Layer 2 VPNs and / or Layer 3 VPNs). For example, an ND that couples a provider's network and a customer's network is called a PE (Provider Edge) and a CE (Customer Edge), respectively. In a Layer 2 VPN, forwarding is typically performed on one or more CEs at either end of the VPN, and traffic is sent across the network (e.g., via one or more PEs coupled by other NDs). Layer 2 circuits (e.g., Ethernet ports, ATM Permanent Virtual Circuits (PVCs), Frame Relay PVCs) are configured between the CEs and PEs. In a Layer 3 VPN, routing is typically performed by the PEs. For example, an edge ND supporting multiple VNEs can be deployed as a PE; and the VNEs can be configured with VPN protocols, and thus the VNE is called a VPNVNE.

[0199] Some NDs support VPLS (Virtual Private LAN Service). For example, in a VPLS network, end-user devices access content / services offered through the VPLS network by coupling to a CE (Customer Equipment), which in turn is coupled to a PE (Personal Equipment) coupled to other NDs. VPLS networks can be used to implement triple-play network applications (e.g., data applications such as high-speed internet access), video applications (e.g., television services such as IPTV (Internet Protocol Television), VoD (Video on Demand) services), and voice applications (e.g., VoIP (Voice over Internet Protocol) services), VPN services, etc. VPLS is a type of Layer 2 VPN that can be used for multipoint connectivity. VPLS networks also allow end-user devices coupled to CEs in separate geographical locations to communicate with each other across a wide area network (WAN) as if they were directly attached to each other in a local area network (LAN) (called an emulated LAN).

[0200] In a VPLS network, each CE (Content Provider) is typically attached to a PE's bridge module via an access network (wired and / or wireless) through an attached circuit (e.g., a virtual link or connection between the CE and PE). The PE's bridge module is attached to an emulated LAN via an emulated LAN interface. Each bridge module acts as a "virtual switch instance" (VSI) by maintaining a forwarding table that maps MAC addresses to pseudowires and attached circuits. The PE forwards frames to their destinations (e.g., other CEs, other PEs) based on the MAC destination address field included in the frames received from the CEs.

[0201] Although the invention has been described with reference to several embodiments, those skilled in the art will recognize that the invention is not limited to the described embodiments, but can be practiced with modifications and variations within the spirit and scope of the appended claims. Therefore, this description is to be regarded as illustrative rather than restrictive.

Claims

1. A method (1600) for offloading network operations of a first application (2021) to network resources (204) in a network system (1300), the method comprising: The uninstallation service (100) receives (1616) an uninstallation service capability request message (1006B) from the first application, wherein the uninstallation service capability request message requests information from the uninstallation service regarding the uninstallation service's ability to satisfy a set of requirements provided by the first application; The uninstallation service transmits an uninstallation service capability request message response (1008B) to the first application (1618), wherein the uninstallation service capability request message response includes a set of uninstallation service templates (110), wherein the uninstallation service templates are: (1) selected by the uninstallation service based on the set of requirements of the first application and (2) are templates to be modified for performing network operations of the first application, wherein each template in the set of uninstallation service templates includes one or more of the following: a list of parameters to be modified for performing network operations of the first application, a set of code stubs for generating program code for performing network operations of the first application, and a set of libraries for generating the program code; The uninstallation service evaluates (1622) the network resources for the program code of the first application to select a set of network resources (2041 and / or 2042) for installing the program and uninstalling the network operations of the first application to the set of network resources; and The uninstallation service installs (1628) the program code of the first application generated based on the set of uninstallation service templates on the set of network resources, such that the set of network resources processes one or more of the following: (1) packets addressed to the first application from the second application and (2) packets addressed to the second application from the first application.

2. The method of claim 1, further comprising: The uninstallation service receives (1602) an uninstallation service discovery message (1002B) from the first application. as well as In response to the uninstall service discovery message, the uninstall service sends an uninstall service discovery message acknowledgment (1004B) to the first application (1604), wherein the uninstall service discovery message acknowledgment indicates that the uninstall service discovery message has been received and that the uninstall service exists in the network system.

3. The method of claim 2, further comprising: The uninstallation service receives (1620) an application uninstallation service request (1010B) from the first application, the application uninstallation service request including templates from the set of templates that are filled with templates for performing the network operation of the first application; The uninstallation service transmits (1624) an application uninstallation service request confirmation (1014B), which indicates that: a set of network resources for hosting the program code of the first application has been found and the program code has been modified to fit the set of network resources. as well as The uninstallation service receives (1626) an application uninstallation deployment request (1016B) from the first application, which is used to start the uninstallation operation.

4. The method of claim 3, wherein, Installing the program code on the set of network resources includes: The uninstallation service transmits (1628A) an uninstallation service deployment request (1018B) including the program code to the set of network resources, so that each network resource in the set of network resources installs the program code; and After the program code is successfully installed from the set of network resources, the uninstallation service receives (1628B) uninstallation service deployment request confirmation (1022B) from each network resource in the set of network resources. The method further includes: The uninstallation service sends an application uninstallation deployment request confirmation (1024B) to the first application (1630), which is used to confirm the deployment of the program code.

5. The method of claim 1, further comprising: The uninstallation service receives (1606) an uninstallation service discovery message (902B) from each network resource in the set of network resources. In response to the uninstall service discovery message, the uninstall service sends an uninstall service discovery message acknowledgment (904B) to each network resource in the set of network resources (1608). The uninstallation service receives (1610) a network resource registration request (906B) from each network resource in the set of network resources. The uninstallation service updates (1612) the network resource information model (908B) and the network model (910B) based on the network resource registration request from each network resource in the set of network resources, wherein the network resource information model describes the characteristics of each network resource in the set of network resources, and wherein the network model describes the interconnections between elements in the network system, including the set of network resources and the computing node on which the first application is operating; and The uninstallation service transmits (1614) a network resource registration request confirmation (912B) to each network resource in the set of network resources, the network resource registration request confirmation indicating that each corresponding network resource in the set of network resources has been successfully registered.

6. The method of claim 5, wherein, The characteristics of each network resource in the set of network resources include one or more of the following: (1) the architecture of each network resource in the set of network resources, (2) the processing power of each network resource in the set of network resources, and (3) the memory capacity of each network resource in the set of network resources.

7. The method of claim 1, further comprising: The uninstallation service receives (1634) an application uninstallation service termination request (1102B) from the first application. The uninstallation service determines (1636) that the set of network resources is allocated to the first application; The uninstallation service sends an uninstallation service termination request (1106B) to each network resource in the set of network resources (1638). The uninstallation service receives (1640) an uninstallation service termination request confirmation (1110B), which indicates that the set of network resources will no longer process the program code. The uninstallation service updates (1642) the network resource information model (1112B) to indicate the termination of the set of network resources; and The uninstallation service sends an application uninstallation service termination request confirmation (1114B) to the first application (1644), which indicates that the operation of uninstalling the first application by the uninstallation service has been terminated.

8. The method of claim 1, further comprising: The uninstallation service receives (1646) a network resource deregistration request (1202B) from the network resource (2041) in the set of network resources. The uninstallation service sends an unsolicited application uninstallation service termination request (1204B) to the first application (1648). The uninstallation service receives (1650) an unsolicited application uninstallation service termination request confirmation (1208B), which indicates that the first application has performed a switching process in response to the network resources no longer processing the program code. The network resource information model (1210B) of the network system is updated (1652) by the uninstallation service to address the situation where the network resources are no longer available for the uninstallation operation; and The uninstallation service sends a network resource deregistration request confirmation (1212B) to the network resource (1654).

9. The method of claim 1, wherein, The network resource is a P4 network resource, and the program code is P4 program code.

10. The method of claim 1, further comprising: The uninstallation service receives (1632) a set of control packets from the first application for one or more of the following: managing, configuring, and supplying program code installed on the set of network resources. The interaction with the first application and the second application is performed via the application uninstallation service agent (118) of the uninstallation service. The application uninstallation service agent allows the first application to manage, configure, provision the uninstallation service, and communicate with the uninstallation service.

11. The method of claim 10, wherein the packet addressed to the first application: either (1) is processed by the set of network resources and subsequently forwarded to the application uninstallation service agent, and then reaches the first application; or (2) is transmitted to the application uninstallation service agent and forwarded to the set of network resources for processing, and then reaches the first application.

12. A non-transitory machine-readable storage medium (1848) that provides instructions, when executed by a processor (1842) of a device (1804) in a network system (1300), to cause the processor to perform operations including: Receive (1616) Uninstall service capability request message (1006B) from the first application (2021), where, The uninstall service capability request message requests information from the uninstall service (100) regarding the uninstall service's ability to satisfy the set of requirements provided by the first application; Send (1618) Uninstall service capability request message response (1008B) to the first application, wherein the uninstall service capability request message response includes a set of uninstall service templates (110), wherein the uninstall service templates are: (1) selected by the uninstall service based on the set of requirements of the first application and (2) are templates to be modified for performing network operations of the first application, wherein each template in the set of uninstall service templates includes one or more of the following: a list of parameters to be modified for performing network operations of the first application, a set of code stubs for generating program code for performing network operations of the first application, and a set of libraries for generating program code; The network resources are evaluated (1622) for the program code of the first application to select a set of network resources (2041 and 2042) for installing the program and unloading the network operations of the first application to the set of network resources; and Install (1628) the program code of the first application generated based on the set of uninstall service templates on the set of network resources such that the set of network resources processes one or more of the following: (1) packets addressed to the first application from the second application and (2) packets addressed to the second application from the first application.

13. The non-transitory machine-readable storage medium of claim 12, wherein, The operation also includes: Receive (1602) Uninstall service discovery message (1002B) from the first application; In response to the uninstall service discovery message, an uninstall service discovery message acknowledgment (1004B) is transmitted to the first application (1604), wherein the uninstall service discovery message acknowledgment indicates that the uninstall service discovery message has been received and that the uninstall service exists in the network system; Receive (1620) an application uninstallation service request (1010B) from the first application, the application uninstallation service request including templates from the set of templates that are populated for performing the network operation of the first application; The uninstallation service transmits (1624) an application uninstallation service request confirmation (1014B), the application uninstallation service request confirmation indicating: a set of network resources for hosting the program code of the first application has been found and the program code has been modified to fit the set of network resources; and The uninstallation service receives (1626) an application uninstallation deployment request (1016B) from the first application, which is used to start the uninstallation operation.

14. The non-transitory machine-readable storage medium of claim 13, wherein, Installing the program code on the set of network resources includes: A request to deploy an uninstallation service for the program code is sent (1628A) to the set of network resources, including the program code, so that each network resource in the set of network resources installs the program code; and After the program code is successfully installed from the set of network resources, an uninstallation service deployment request confirmation (1022B) is received (1628B) from each network resource in the set of network resources. The operation further includes: The uninstallation service sends an application uninstallation deployment request confirmation (1024B) to the first application (1630), which is used to confirm the deployment of the program code.

15. The non-transitory machine-readable storage medium of claim 12, wherein, The operation also includes: Receive (1606) an offload service discovery message (902B) from each of the network resources in the set of network resources. In response to the uninstall service discovery message, an uninstall service discovery message acknowledgment (904B) is sent (1608) to each network resource in the set of network resources. Receive (1610) a network resource registration request (906B) from each network resource in the set of network resources. Based on the network resource registration request from each network resource in the set of network resources, update (1612) the network resource information model (908B) and the network model (910B), wherein the network resource information model describes the characteristics of each network resource in the set of network resources, and wherein the network model describes the interconnections between elements in the network system, including the set of network resources and the computing node on which the first application is operating; and The uninstallation service transmits (1614) a network resource registration request confirmation (912B) to each network resource in the set of network resources. The network resource registration request confirmation indicates that each corresponding network resource in the set of network resources has been successfully registered. The characteristics of each network resource in the network resources include one or more of the following: (1) the architecture of each network resource in the set of network resources, (2) the processing power of each network resource in the set of network resources, and (3) the memory capacity of each network resource in the set of network resources.

16. The non-transitory machine-readable storage medium of claim 12, wherein, The operation also includes: Receive (1634) Application uninstallation service termination request (1102B) from the first application. Determine (1636) that the set of network resources is allocated to the first application; Send (1638) an unload service termination request (1106B) to each network resource in the set of network resources. Receive (1640) Uninstall service termination request confirmation (1110B), the uninstall service termination request confirmation being used to indicate that: the set of network resources will no longer process the program code; Update (1642) the network resource information model (1112B) to indicate the termination of the set of said network resources; and Send (1644) an application uninstallation service termination request confirmation (1114B) to the first application, the application uninstallation service termination request confirmation indicating that the operation of uninstalling the first application by the uninstallation service has been terminated.

17. The non-transitory machine-readable storage medium of claim 12, wherein, The operation also includes: Receive (1646) a network resource deregistration request (1202B) from the network resource (2041) in the set of network resources. Send (1648) an unsolicited application uninstallation service termination request (1204B) to the first application. Receive (1650) Unsolicited Application Uninstall Service Termination Request Confirmation (1208B), the Unsolicited Application Uninstall Service Termination Request Confirmation being used to indicate that: the first application has performed a switching process in response to the network resources no longer processing the program code; Update (1652) the network resource information model (1210B) of the network system to address the situation where the network resources are no longer available for offloading operations; and Send (1654) Network resource deregistration request confirmation (1212B) to the network resource.

18. The non-transitory machine-readable storage medium of claim 12, wherein, The network resource is a P4 network resource, and the program code is P4 program code.

19. An apparatus (1804) for offloading network operations of a first application (2021) to network resources (204) in a network system (1300), the apparatus being used to: Receive (1616) uninstall service capability request message (1006B) from the first application, wherein, The uninstall service capability request message requests information from the uninstall service (100) regarding the uninstall service's ability to satisfy the set of requirements provided by the first application; Send (1618) Uninstall service capability request message response (1008B) to the first application, wherein the uninstall service capability request message response includes a set of uninstall service templates (110), wherein the uninstall service templates are: (1) selected by the uninstall service based on the set of requirements of the first application and (2) are templates to be modified for performing network operations of the first application, wherein each template in the set of uninstall service templates includes one or more of the following: a list of parameters to be modified for performing network operations of the first application, a set of code stubs for generating program code for performing network operations of the first application, and a set of libraries for generating program code; The network resources are evaluated (1622) for the program code of the first application to select a set of network resources (2041 and 2042) for installing the program and unloading the network operations of the first application to the set of network resources; and Install (1628) the program code of the first application generated based on the set of uninstall service templates on the set of network resources such that the set of network resources processes one or more of the following: (1) packets addressed to the first application from the second application and (2) packets addressed to the second application from the first application.

20. The apparatus of claim 19, further comprising: Receive (1602) uninstall service discovery message (1002B) from the first application; and In response to the uninstall service discovery message, an uninstall service discovery message acknowledgment (1004B) is sent to the first application (1604), wherein, The uninstall service discovery message confirmation indicates that the uninstall service discovery message has been received and that the uninstall service exists in the network system.

Images