Serverless packet processing service with isolated virtual network integration

Abstract

The present disclosure relates to serverless packet processing services with isolated virtual network integration. A program to be executed to perform a packet processing operation on a packet associated with a resource group and security settings for the resource group are received. The program is transmitted to a set of fast path nodes based on metadata for the group being assigned to the resource group. Security operations based on the settings are performed with respect to a particular packet and the program is executed at a fast path node. Packet routing operations corresponding to the received packet are performed based at least in part on results of the program.

Description

[0001] This application is a divisional application of Inventive Patent Application No. 202080045348.6, filed 17 June 2020, entitled “Serverless Packet Processing Service with Isolated Virtual Network Integration”. TECHNICAL FIELD

[0002] The present disclosure relates to a serverless packet processing service with isolated virtual network integration. BACKGROUND

[0003] Many companies and other organizations operate computer networks that interconnect many computing systems to support their operations, whether the computing systems are co-located (e.g., as part of a local network) or instead located in multiple different geographic locations (e.g., connected by one or more private or public intermediate networks). For example, data centers that house large numbers of interconnected computing systems have become commonplace, such as private data centers by and for a single organization, and public data centers operated as entities that provide computing resources to customers. Some public data center operators provide network access, power, and secure installation facilities for hardware owned by various customers, while other public data center operators provide “full service” facilities that also include hardware resources available for use by the operator’s customers.

[0004] The advent of virtualization technology for commodity hardware has provided benefits in managing large-scale computing resources for many customers with different needs, allowing multiple customers to efficiently and securely share various computing resources. For example, virtualization technology can allow a single physical virtualization host to be shared among multiple users by providing each user with one or more “guest” virtual machines hosted by the single virtualization host. Each such virtual machine can represent a software simulation that acts as a different logical computing system, providing the user with the illusion that they are the sole operator of a given hardware computing resource, while also providing application isolation and security among the various virtual machines. Instantiating several different virtual machines on the same host can also help increase the overall hardware utilization level at a data center, leading to higher return on investment. As the demand for virtualization-based services of provider networks continues to grow, it can be necessary to add more and more features related to networking and interconnectivity to meet the needs of applications implemented using the services. SUMMARY

[0005] A computer-implemented method is provided that includes obtaining an indication of an unauthorized transfer of data to or from a first set of compute instances of a cloud environment; allocating a first set of packet processing resources to detect unauthorized transfers of data to or from the first set of compute instances; and automatically increasing a number of packet processing resources in the first set in response to detecting that a compute instance has been added to the first set of compute instances. BRIEF DESCRIPTION OF DRAWINGS

[0006] Figure 1 An example system environment in which a packet processing service can be implemented is illustrated in accordance with at least some embodiments.

[0007] Figure 2 Example sub-components of a packet processing service are illustrated in accordance with at least some embodiments.

[0008] Figure 3 An example customer view of operations performed at a packet processing service is illustrated in accordance with at least some embodiments.

[0009] Figure 4 An example overview of fast path and slower path operations at a packet processing service is illustrated in accordance with at least some embodiments.

[0010] Figure 5 An example overview of business source based dynamic provisioning of nodes at a packet processing service is illustrated in accordance with at least some embodiments.

[0011] Figure 6 An example distribution of nodes of a packet processing service across availability containers of a provider network is illustrated in accordance with at least some embodiments.

[0012] Figure 7 An example packet processing pipeline including multiple stages is illustrated in accordance with at least some embodiments.

[0013] Figure 8 An example mapping of packet processing pipeline stages to nodes of a packet processing service is illustrated in accordance with at least some embodiments.

[0014] Figure 9 Example context information that can be maintained and modified as packets are processed at various stages of a pipeline is illustrated in accordance with at least some embodiments.

[0015] Figure 10 , Figure 11 and Figure 12 Collectively illustrate example stages of a multi-stage pipeline at which isolated virtual network configuration related packet processing operations can be performed in accordance with at least some embodiments.

[0016] Figure 13FIG. illustrates an example implementation platform for a packet processing node, in accordance with at least some embodiments.

[0017] Figure 14 FIG. illustrates example categories of packet processing applications that can be implemented at a serverless packet processing service using pipelines, in accordance with at least some embodiments.

[0018] Figure 15 FIG. illustrates example types of entities that can implement application programming interfaces at a packet processing service, in accordance with at least some embodiments.

[0019] Figure 16 and Figure 17 FIG. illustrates example packet processing pipeline related programmatic interactions, in accordance with at least some embodiments.

[0020] Figure 18 FIG. illustrates an example exception path packet processing scenario, in accordance with at least some embodiments.

[0021] Figure 19 FIG. illustrates an example system environment in which a data plane extension of a packet processing service of a provider network can be implemented at a premises outside of the provider network, in accordance with at least some embodiments.

[0022] Figure 20 FIG. illustrates example types of premises that can implement a packet processing service extension, in accordance with at least some embodiments.

[0023] Figure 21 FIG. illustrates example programming interactions related to a packet processing service extension, in accordance with at least some embodiments.

[0024] Figure 22 is a flow diagram illustrating aspects of operations that can be performed to implement a packet processing pipeline integrated with isolated virtual network configurations, in accordance with at least some embodiments.

[0025] Figure 23 is a flow diagram illustrating aspects of operations that can be performed to implement a customized exception path for a packet processing pipeline, in accordance with at least some embodiments.

[0026] Figure 24 is a flow diagram illustrating aspects of operations that can be performed to implement an extension of a packet processing service outside of a data center of a provider network, in accordance with at least some embodiments.

[0027] Figure 25 is a block diagram illustrating an example computing device that can be used in at least some embodiments.

[0028] While embodiments are described herein by way of example with respect to several embodiments and illustrative figures, those skilled in the art will recognize that embodiments are not limited to the embodiments or illustrative figures described. As should be understood, the figures and detailed description are not intended to limit the embodiments to the form disclosed, but rather, are intended to cover all modifications, equivalents and alternatives falling within the spirit and scope of the appended claims. The headings used herein are for organizational purposes only and are not intended to be used to limit the scope of this description or the claims. As used throughout this application, the word "may" is used in a permissive sense (i.e., meaning having the potential to), rather than the mandatory sense (i.e., meaning must). Similarly, the words "include," "including," and "includes" mean including, but not limited to. The term "or" is used in its inclusive sense (i.e., meaning and / or) unless the context clearly indicates otherwise. For example, the phrase "at least one of x, y or z" means any one of x, y, or z, as well as any combination thereof. DETAILED DESCRIPTION

[0029] The present disclosure relates to methods and apparatus for implementing a packet processing service at which a customer-designed pipeline for packet processing operations associated with a traffic source or destination configured within an isolated virtual network of a provider network can be executed using dynamically provisioned and programmed service nodes. A network established by an entity such as a company or public sector organization to provide one or more network-accessible services (such as various types of cloud-based computing, storage, or analytics services) accessible to a set of distributed clients via the Internet and / or other networks can be referred to as a provider network. A provider network can sometimes be referred to as a "public cloud" environment. Within a provider network, in some embodiments, groups of resources can be configured as isolated virtual networks on behalf of respective clients. As the name indicates, an isolated virtual network (IVN) can be logically isolated from the rest of the provider network (and other networks outside the provider network) with respect to various aspects of network configuration— e.g., internal or "private" IP addresses can be assigned to resources of a given IVN independently of addresses assigned within other networks, security settings of a given IVN can limit the sources and destinations with which resources of the IVN can communicate, etc.

[0030] A packet processing service as described herein can be used to implement a variety of applications, including load balancing, policy-based routing, network address translation, and intrusion detection. The disclosed packet processing service can include a plurality of nodes that are divided into two high-level categories in various embodiments: control plane nodes and data plane nodes. Control plane nodes can be responsible for administrative tasks such as provisioning, propagation of changes to metadata used for packet processing operations, and the like, whereas data plane nodes can be primarily responsible for processing packets received at the service. Data plane nodes can be further subdivided into fast path processing nodes and slower path processing nodes, where fast path nodes are (ideally) used for the vast majority of received packets, and slower path nodes are used for less frequent operations, as discussed in further detail below. Slower path nodes can be referred to as "exception path" nodes in at least some embodiments, as they can be expected to be used only relatively infrequently (i.e., in scenarios that can be considered exceptions rather than common). In at least some embodiments, at least some data plane nodes of the service can be provisioned or assigned to a business endpoint dynamically, e.g., without the client submitting a request to establish the nodes. Thus, from the perspective of a client of the packet processing service, the service can be considered "serverless" in such embodiments, as the client can not have to concern themselves with details regarding service node establishment, health monitoring of service nodes, scaling up or down of the fleet of nodes used on behalf of the client, and the like. In different embodiments, a serverless packet processing service can transparently support a desired level of scalability, availability, and reliability for various types of packet processing tasks.

[0031] At a high level, in various embodiments at least two categories of clients can utilize a serverless packet processing service (SPPS). A first category can include application vendors who wish to utilize the SPPS to present various types of sophisticated networking-related applications to their own end customers, while a second category can include such end customers themselves. For example, a packet processing application vendor can design sophisticated packet processing pipelines that are capable of very quickly detecting attempted cyber attacks (e.g., using sophisticated pattern matching or machine learning algorithms), and use the SPPS to implement virtual appliances that are capable of implementing such pipelines. End users, such as business entities that have established their own networks within a provider network and / or at premises outside, can request that instances of such virtual appliances be configured to help protect their networks. Similarly, other vendors can provide high-level load balancing appliances, appliances for network address translation, appliances that can be used by telecommunications organizations to manage per-user bandwidth usage, etc., again using SPPS nodes. In many cases, end users of such appliances can have established corresponding isolated virtual networks (IVNs) within a provider network that implements the SPPS, and these appliances can be used to generate or direct traffic at or to compute instances within such IVNs. By design, in at least some embodiments the SPPS can be tightly integrated with IVN infrastructure - for example, packet processing operations used to implement and enforce IVN configuration requirements (e.g., security rules selected by a customer on whose behalf an IVN has been established, routing rules for an IVN, etc.) can be implemented transparently with packet processing pipelines that implement vendor-supplied appliance logic. In various embodiments, packet processing appliance vendors and their end users can be able to benefit from the seamless scalability, availability, and reliability of the SPPS, however the deep integration of the SPPS with IVN configuration management can ensure that all IVN-related security and routing rules will also be automatically enforced (without the vendor having to explicitly integrate security and routing rules with their appliances). When IVN configuration changes to a source and / or destination of traffic, the SPPS can be automatically notified of the change, and in various embodiments corresponding changes can be made to packet processing programs implemented at SPPS nodes. In some embodiments, packet processing application vendors as well as end users can not even have to be aware that IVN configuration-related packet processing tasks are being performed at the SPPS.

[0032] As will be appreciated by those skilled in the art in light of the present disclosure, certain embodiments can be capable of realizing various advantages and practical applications / benefits, including some or all of the following: (a) enabling very large amounts of message traffic to be efficiently processed pursuant to customizable packet processing application pipelines using an automatically scaled fleet of packet processing nodes, (b) enhancing the security of packet processing applications, e.g., by automatically implementing security-related settings for isolated virtual networks established on behalf of end users of such applications, without requiring the provider / vendor of the application pipelines to incorporate security-related settings into their pipelines, and / or (c) enhancing the user experience of administrators and packet processing application providers by eliminating the need for such users to request provisioning of packet processing resources or to manually incorporate isolated network configuration settings into packet processing applications.

[0033] According to some embodiments, a system can include one or more computing devices of a“serverless” packet processing service (SPPS). The devices can include instructions that, when executed on or across one or more processors, cause the devices to obtain, via one or more programmatic interfaces, a request for a packet processing pipeline to be established for a set of resources of an isolated virtual network (IVN) of a virtualized computing service (VCS). The packet processing pipeline can include one or more stages, and in at least some embodiments can expose respective instances of the pipeline in the form of virtual appliances. For at least one stage of the pipeline, the request can indicate at least one rule. Any combination of one or more different types of rules can be used at the pipeline stage— e.g., some rules can involve using a key derived from a header of a received packet to look up and perform an action (e.g., an executable program or function) within a table, other rules can simply involve specifying execution of a program, etc. For example, the programs and / or rules can be expressed using an efficient low-level programming language or framework designed for packet processing tasks, such as eBPF (extended Berkeley Packet Filter). In various embodiments, the request can not specify particular resources to be used to implement the pipeline. The instructions, when executed, can identify a plurality of fast-path packet processing nodes (FPPNs) for the set of resources of network traffic based at least in part on a number of sources of traffic within the isolated virtual network. In various embodiments, each such (FPPN) can be implemented using a combination of hardware and software.

[0034] The programs and / or rules indicated in the pipeline establishment request can be deployed or pushed to one or more FPPNs, where they can be stored in respective caches for use in processing subsequent packets belonging to the same flow (where a given flow can be distinguished from other flows, e.g., based on a tuple including source and destination addresses and ports). In at least some embodiments, one or more network configuration operations can be performed, e.g., in response to the programming request, to establish the traffic ingress point and / or egress point of the pipeline. In one embodiment, for example, one or more virtual network interfaces can be configured as the ingress point or egress point. In response to a particular packet arriving at the ingress point, one or more packet processing operations can be performed in accordance with the security-related configuration settings of the IVN in some embodiments; such packet processing operations can not necessarily be included as part of the pipeline for which the request was received earlier. Rules corresponding to respective stages of the deployed pipeline can be applied to the received packet, e.g., one or more eBPF programs can be executed with respect to the received packet at one or more stages. Based at least in part on the results of the application of the rules and / or the routing configuration settings of the IVN, one or more routing-related actions can be implemented with respect to the received packet in various embodiments. Any of several different types of routing actions can be implemented based on the logic of the pipeline and the configuration settings of the IVN— e.g., in some cases at least one packet corresponding to the received packet (e.g., a rewritten or modified version of the received packet) can be transmitted via the egress point of the pipeline to a destination, whereas in other cases the received packet can simply be discarded without any corresponding packet being sent toward the destination.

[0035] In at least some embodiments, control plane elements of the SPPS can detect when a source or destination of traffic (e.g., a compute instance such as a guest virtual machine) is added to or removed from an IVN. Responsive to configuration of some number of additional sources or destinations of traffic, one or more additional SPPS nodes (e.g., FPPNs and / or other slower path packet processing nodes) can be automatically configured in such embodiments. Similarly, if / when a source or destination of traffic is decommissioned or removed from an IVN, the number of SPPS nodes assigned to the IVN can be reduced in some embodiments. In at least some embodiments, as noted above, the count of SPPS nodes can be determined based at least in part on the number of sources of traffic rather than destinations. Thus, unlike some conventional load balancers or other packet processing applications, the SPPS node count can be scaled up or down based on the number of sources from which traffic can have to be processed rather than the number of destinations of traffic. In one embodiment, a shuffle sharding algorithm can be used to map sources of traffic (e.g., compute instances or virtualized hosts from which traffic is to be processed) to SPPS nodes such that fundamentally different but potentially overlapping subsets of SPPS nodes available at an IVN are assigned to process traffic from respective sources. In some embodiments, one set of SPPS nodes can be assigned to process inbound traffic directed to a given set of compute instances or other resources, and a different (but potentially overlapping) set of SPPS nodes can be assigned to process outbound traffic originating from the same set of compute instances or resources.

[0036] In some embodiments, location considerations can be used to select particular SPPS nodes to assign to particular sources or destinations of traffic. As the term "fast path" implies, one of the goals of SPPS can include performing the desired type of packet processing very efficiently, resulting in minimal achievable latency for the transmission of traffic. Thus, in embodiments in which sources / destinations of traffic are distributed among many racks of a data center, respective sets of FPPNs can be pre-configured (e.g., prior to actual requests to establish packet processing pipelines) at computing devices located at a number of selected racks that are placed in a number of approximately evenly distributed locations within the data center. When packet processing pipelines are to be established for traffic of one or more compute instances (e.g., as instances of virtual appliances) and the compute instances are located on hosts of racks Rl, R2, and R3, FPPNs that happen to be configured on racks R4 and R5 that are in close proximity to Rl, R2, and R3 can be assigned to these compute instances. Thus, in some embodiments when assigning FPPNs to a pipeline, proximity of the devices to which the FPPNs have been pre-configured to represent sources / destinations of packets to be processed can be considered. Note that the assignment of SPPS nodes (including FPPNs) to sources / destinations of traffic can be dynamically modified over time in some embodiments, e.g., based on workload changes and other factors.

[0037] Any of a wide variety of execution platforms can be used for FPPNs and / or other SPPS nodes in different embodiments, including, e.g., compute instances running at virtualization hosts of a virtualized computing service, non-virtualized hosts, dedicated hardware servers with one or more cards attached via a peripheral interconnect (e.g., a PCIe or Peripheral Component Interconnect Express interconnect) at which packet processing programs / rules are executed, etc. According to some embodiments, SPPS nodes for pipeline instances established for traffic generated at or directed to one or more end user IVNs can themselves be configured within another IVN - that is, an IVN can be used for SPPS nodes as well as for sources and destinations of traffic. In at least some embodiments, SPPS nodes groups can be configured as respective independently-updated and independently-configured cells, such that if a failure event occurs within a given cell, the effects of the failure cannot cross cell boundaries and affect other cells. When software, firmware, or hardware updates are to be applied in such cell-based embodiments, the update can be propagated only once to the nodes (e.g., FPPNs) of one cell. Prior to propagating the update to another cell, the SPPS control plane can attempt to verify that the cell to which the update was applied meets stability criteria (e.g., there has been no increase in dropped packets or other undesirable metrics for packet processing pipelines implemented using the updated cell).

[0038] As noted earlier, in at least some embodiments, programs and / or rules associated with packet processing stages can be cached at FPPNs. If a packet is received at an FPPN such that the FPPN cannot find an entry in its cache that is applicable to the packet, the FPPN can submit a query to a slow path packet processing node (SPPN) indicating a cache miss. Several different types of SPPNs can be configured as part of the SPPS data plane in various embodiments, including nodes that are primarily responsible for IVN configuration related packet processing, nodes that make vendor supplied decisions based on packet processing pipelines in response to cache misses, and nodes that include repositories for storing persistent state information associated with the processing of various network flows. One or more SPPNs can decide on particular packet processing actions to be taken at the FPPN for the packet that caused the cache miss (and in some cases for subsequent packets of the same flow). If possible, entries that can be cached at the FPPN (e.g., for use with respect to subsequent packets of the same flow) can be communicated from the SPPN to the FPPN in some embodiments. In some cases, state information stored in a repository accessible from the SPPN (e.g., state information generated in response to earlier cache misses from different FPPNs) can be used in responding to a given FPPN cache miss. In some embodiments, in response to detecting a failure at a given FPPN, the workload of the failed FPPN can be assigned to one or more replacement FPPNs, and state information related to packet processing operations previously performed at the failed FPPN can be obtained from a repository at the replacement FPPN.

[0039] The configuration settings (and / or the logic for the vendor-supplied pipeline) for determining actions at slower-path nodes can be modified over time in various embodiments, in which case it can be necessary to invalidate cache entries at the FPPNs. In at least one embodiment, a timeout-based cache entry revalidation technique can be employed at one or more FPPNs. For at least some cache entries, a respective timeout can be set in such embodiments. When the timeout expires, the FPPN can initiate a revalidation operation for the corresponding cache entry, e.g., by issuing a revalidation query to the SPPN, effectively asking whether the SPPN entry remains valid. If the SPPN fails to revalidate the entry, the entry can be removed from the FPPN cache in some embodiments. In at least one embodiment, if a cache entry is no longer valid, and the SPPN is able to generate a valid replacement entry, the replacement entry can be provided to the FPPN. In some embodiments, a multi-tiered timeout-based scheme can be used for FPPN cache entry revalidation or replacement. According to such a scheme, both soft and hard timeouts can potentially be used for a given cache entry, with the soft timeout interval set to a value smaller than the hard timeout interval. When a soft timeout STOl for a cache entry CE1 for a packet flow PF1 expires at the FPPN, an indication of the expiration of STOl can be provided to the SPPN (so that the SPPN can revalidate CE1 and start a new set of soft and hard timeout periods, or indicate that CE1 is no longer valid), but the FPPN can continue to use CE1 to process other received packets for FP1. In contrast, if / when a hard timeout HTOl corresponding to CE1 expires, the FPPN can stop using CE1, effectively invalidating or discarding CE1 from the cache. In some embodiments, multiple soft timeout expiration indications (corresponding to several different cache entries) can be batched and sent to the SPPN as a single message to amortize the communication cost of the scheme. In embodiments in which a timeout-based technique is implemented for FPPN cache management, the SPPS client on behalf of which a cache entry is stored can not have to explicitly request that the cache entry be invalidated. For example, after traffic for a given packet flow stops, the corresponding cache entry can eventually be invalidated based on timeout expiration.

[0040] As indicated earlier, slower path packet processing operations such as those taken in response to cache misses at FPPNs can be referred to in some embodiments as exception path operations (as they are primarily intended for less frequent or "exceptional" scenarios for which higher latency can be acceptable), and SPPNs can also be referred to as exception path nodes. According to one embodiment, an SSPN client such as a supplier of a packet processing appliance can use one or more programming interfaces to specify a target set of resources to be used for exception path processing of at least some packet flows arriving at an instance of the appliance. For example, such custom target resources can include a collection of the client's compute instances or other computing devices at which custom packet processing operations that would not be performed at the FPPNs of the SPPS can be implemented (e.g., due to resource constraints of the FPPNs, security reasons, etc.). In some embodiments, the target resources for exception path processing can be indicated in the same programming interaction that requests establishment of a packet processing pipeline to be implemented using some number of FPPNs; in other embodiments, a different programming request to customize the pipeline and configuration of the exception path can be used. In various embodiments, a determination of whether a given packet should be transmitted to such target resources can be made using rules indicated in the pipeline, e.g., at the FPPNs or SPPNs. In some embodiments, the target exception path packet processing resources specified by the client can be configured within IVNs separate from the IVNs at which the sources / destinations of the packets are configured and / or separate from the IVNs in which a subset of the FPPNs / SPPNs used for the pipeline are configured. In one embodiment, at least some of the target resources for exception path processing can be located at premises outside of the provider network at which most of the SPPS resources are located.

[0041] Even in scenarios where custom exception path processing is configured for one or more stages of the pipeline, IVN configuration related packet processing can still be performed when needed for a particular packet flow, and a set of FPPNs can be identified for traffic to be processed, just as FPPNs are identified when custom exception path processing is not needed. When a packet of a flow is received at the SPPS, the SPPS can determine whether the packet satisfies one or more IVN related constraints, such as security requirements, and reject the packet if such requirements are not satisfied. If the packet satisfies IVN configuration related requirements, a determination can be made using rules or criteria specified for the stages of the pipeline whether the packet is to be processed using fast path operations or whether the packet requires exception path processing. If the packet can be processed using fast path operations (e.g., indicated via eBPF programs specified earlier in the pipeline definition), fast path processing can be completed using FPPNs. If the packet requires non-custom exception path processing, one or more SPPS resources can be used; if custom exception path processing is required, in at least some embodiments the packet can be sent to a target exception path resource specified by the SPPS client.

[0042] A wide variety of packet processing applications can be supported at the SPPS via the pipeline based approach discussed above in different embodiments. Such applications can include (but can not be limited to) implementing respective algorithms for custom or policy driven routing, network address translation (e.g., with source address or source port substitution), multicast, cross-IVN packet transfer (transfer from one IVN to another IVN), secure transfer of data to or from a virtual private network, intrusion detection, per user or per group bandwidth usage tracking, load balancing, secure transfer of data between a provider network data center and a premises outside of the provider network, data exfiltration detection, or business logic implementing applications designed by a vendor. In some embodiments, an online marketplace (e.g., supported by an operator of a provider network implementing the SPPS) implementing pre-designed pipelines of various types of packet processing pipelines can be supported, and vendors can assemble or construct new pipelines by combining pre-designed pipelines obtained from such a marketplace.

[0043] According to some embodiments, portions of the SPPS data plane can be established at premises external to the data centers of the provider network that implements the control plane of the SPPS. Metadata associated with a resource group (e.g., an IVN) of the provider network can be stored at one or more data centers of the provider network. A set of packet processing nodes of the SPPS can be configured at a premises external to the data centers, e.g., at a co-located facility where resources of the provider network are located with resources owned by other entities, or at a facility owned by a customer. At a management or control plane component at one of the data centers of the provider network, an indication of a packet processing pipeline to be implemented for at least a subset of packets associated with the resource group can be received in various embodiments, e.g., via one or more programmatic interfaces. A representation of the packet processing pipeline can be transmitted from the data center of the provider network to the external premises along with one or more configuration settings for the resource group. In response to detecting a particular packet at the external premises, one or more security-related operations based on the configuration settings for the resource group can be performed. Packet processing operations of the pipeline can then be performed at one or more of the packet processing nodes established at the external premises. At least one packet corresponding to the detected packet (e.g., a rewritten or transformed version of the packet) can be transmitted from the one or more packet processing nodes toward a destination based on the results of the packet processing in some cases; in other cases, the detected packet can simply be discarded. When a configuration change is detected at the SPPS control plane resources within the provider network that would affect packet processing operations (or would require additional SPPS nodes at the external premises), appropriate actions can be initiated from the SPPS control plane— e.g., the configuration change can be propagated to the external premises, a new version of the pipeline stage program can be transmitted to the external premises, etc. For applications implemented at least partially at such external premises, the ability to logically extend the capabilities of the SPPS data plane can be extremely beneficial, as SPPS data plane functionality can be implemented in practice very close to the resources at which the rest of the application is running.

[0044] Example System Environment

[0045] Figure 1An example system environment in which a packet processing service can be implemented is illustrated in accordance with at least some embodiments. As shown, system 100 can include resources and artifacts of a serverless packet processing service (SPPS) 102 implemented as a component of a virtualized compute service 105 at a provider network 101. As mentioned earlier, a network providing one or more network-accessible services (such as various types of cloud-based computing, storage, or analytics services) accessible to a set of distributed clients via the Internet and / or other networks can be referred to as a provider network in various embodiments. A provider network can sometimes be referred to as a "public cloud" environment. In some cases, resources of a provider network are distributed across multiple data centers, which in turn can be distributed across a number of geographic regions (e.g., each region corresponding to one or more cities, states, or countries).

[0046] In the depicted embodiment, SPPS 102 can include a set of fast-path resources 122, exception or slower-path resources 123, and isolated virtual network (IVN) integration resources 124. Source-based dynamic provisioning of data plane resources can be implemented at SPPS 102 in the depicted embodiment: that is, for a given packet processing pipeline instance, the number of fast-path and / or slower-path resources assigned can be selected (and dynamically changed as needed) based on the number of traffic sources for the pipeline instance. Note that while data plane components of a SPPS are shown in Figure 1 In the depicted embodiment, SPPS 102 can include a set of fast-path resources 122, exception or slower-path resources 123, and isolated virtual network (IVN) integration resources 124. Source-based dynamic provisioning of data plane resources can be implemented at SPPS 102 in the depicted embodiment: that is, for a given packet processing pipeline instance, the number of fast-path and / or slower-path resources assigned can be selected (and dynamically changed as needed) based on the number of traffic sources for the pipeline instance. Note that while data plane components of a SPPS are shown in

[0047] In the depicted embodiment, at least some of the compute instances and other resources of VCS 105 can be organized into a series of isolated virtual networks (IVNs) 120. A given isolated virtual network, such as IVNs 120A, 120B, or 120C, can include a series of networking resources (including, e.g., compute instances, virtualized and / or physical networking devices, hosts, etc.) allocated to a given client that are logically isolated from (and, by default, inaccessible from) resources allocated to other clients in other isolated virtual networks. A client on whose behalf an IVN is established can be granted substantial flexibility with respect to network configuration of resources for the IVN— e.g., a dedicated IP address for a compute instance can be selected by the client without having to consider the possibility that other resources within other IVNs can have been assigned the same IP address, a client's chosen subnets can be established within the IVN, security rules can be set by the client for incoming and outgoing traffic with respect to the IVN, etc. Multiple IVNs can be established on behalf of the same client in some embodiments. In the depicted example scenario, IVNs 120A and 120B have been established for client Cl of VCS 105, whereas IVN 120C has been established for a different client C2. Note that, although virtualization hosts 150 are shown in Figure 1 Figure 1 In the depicted example scenario, IVNs 120A and 120B have been established for client Cl of VCS 105, whereas IVN 120C has been established for a different client C2. Note that, although virtualization hosts 150 are shown in

[0048] ​In various embodiments, clients of the VCS can utilize compute instances 160 to run various types of applications. In some cases, applications running at a given VH can communicate with resources elsewhere within the provider network 101 and / or resources external to the provider network 101, such as external network 150. In the depicted embodiment, individual ones of the compute instances can be programmatically attached to one or more virtual network interfaces (VNIs), whereas individual ones of the virtualization hosts (and resources external to the VCS) can have configured physical network interface cards (NICs). A given VNI can have one or more network addresses (e.g., Internet Protocol (IP) version 4 or version 6 addresses) assigned to it, and a NIC can also have one or more network addresses assigned to it. To convey packet content from a CI 160 at a VH 150 to a destination external to the VH, an encapsulation protocol can be implemented at the VCS, which can be used to map VNI network addresses to physical NIC addresses (among other functions). Further, for at least some sequences of network packets sent from or directed to a compute instance 160, packet processing actions such as load balancing, source address substitution, etc. can be performed, e.g., with the assistance of the SPPS 102. For example, some number of CIs 160 can collectively be used to implement a particular application App1 of client Cl, and can have to load balance App1 service requests among the CIs.

[0049] A sequence of network packets, referred to as a packet flow, can be conveyed between a given CI 160 and one or more traffic destinations or sources using a packet processing pipeline implemented at least in part at the SPPS 102. In the depicted embodiment, the SPPS 102 can be configured to implement a packet processing pipeline that includes a plurality of packet processing stages, such as a packet classification stage 202, a packet modification stage 204, a packet load balancing stage 206, and a packet forwarding stage 208. The packet classification stage 202 can be configured to classify packets of a packet flow based on one or more packet classification criteria, such as a source IP address, a destination IP address, a source port number, a destination port number, a protocol type, etc. The packet modification stage 204 can be configured to modify packets of a packet flow based on one or more packet modification criteria, such as a source IP address, a destination IP address, a source port number, a destination port number, a protocol type, etc. The packet load balancing stage 206 can be configured to load balance packets of a packet flow based on one or more packet load balancing criteria, such as a source IP address, a destination IP address, a source port number, a destination port number, a protocol type, etc. The packet forwarding stage 208 can be configured to forward packets of a packet flow based on one or more packet forwarding criteria, such as a source IP address, a destination IP address, a source port number, a destination port number, a protocol type, etc. Figure 1The middle diagram illustrates many different example flow types; for example, each flow can be distinguished from other flows based on an N-tuple that includes an identifier of a network protocol being used, a source network address, a source network port, a destination network address, and / or a destination network port. In some embodiments, the N-tuple used to uniquely identify a network packet flow can also include other elements, such as an identifier of a virtual network interface for the packet or an indication of a direction of packet transmission. Example flow 192A can include packets transmitted between CIs 160A and 160C within the same IVN 120A. Flow 192B can include packets transmitted between CI 160A and storage service 140 of provider network 101, e.g., via a fleet of VCS border networking devices 130. Flow 192C can include packets sent between CI 160B at IVN 120A and external network 150 (e.g., a customer data center or co-location facility). Flow 192D can include packets transmitted between CI 160T and database service 144. Flow 192E can include packets sent between CIs at two IVNs of the same VCS customer: IVN 120A and 120B (both established on behalf of customer Cl). Flow 192F can include packets transmitted between IVNs of different VCS customers: IVN 120B of customer Cl and IVN 120C of customer C2.

[0050] For each of these types of network flows, various types of IVN configuration settings included in IVN networking metadata 170 (e.g., metadata 170A, 170B, or 170C) can be efficiently and transparently implemented at SPPS 102 in the depicted embodiment, such as security rules for inbound or outbound traffic and / or network access control lists (NACLs). Moreover, depending on the particular application sending network packets to or from a CI 160, additional packet processing operations can be performed at SPPS 102, e.g., represented by a pipeline that includes one or more stages designed or developed by a third-party application vendor, where each stage includes one or more rules and / or one or more actions. In some embodiments, a respective pipeline can be implemented for a given network packet flow for (a) IVN configuration-related packet processing tasks and (b) vendor-defined packet processing tasks. IVN integration resources 124 implemented at one or more computing devices of the SPPS can obtain IVN networking metadata 170 from various IVNs, e.g., generate corresponding packet processing programs and rules that implement the security and other requirements of the IVNs, and transmit the programs and rules to fast path resources 122 (e.g., via exception path resources 123 or directly).

[0051] In some embodiments, a vendor of a packet processing application such as an intrusion detection operation or a data exfiltration detection application can submit a programming request to the SPPS to establish an instance of a vendor-defined packet processing pipeline to be employed for traffic to / from a set of CIs 160 of one or more IVNs 120. The vendor need not specify a particular SPPS resource to be used for the pipeline in the depicted embodiment, and indeed can not even be aware of the architecture or constituent tiers and nodes of the SPPS. The request can include information about the various stages of the pipeline, including for example respective rules to be applied to perform packet processing at the various stages (and / or programs to be run to perform packet processing at the various stages), as well as rules for determining whether a given packet qualifies for the program. The programs and / or rules can be expressed using a custom framework optimized for packet processing operations in various embodiments, such as eBPF or the like. A number of fast path packet processing nodes (FPPNs), implemented using a combination of hardware and software fast path resources 122, can be assigned to the CIs 160 for which the pipeline is requested, e.g., by a component of the SPPS control plane in the depicted embodiment. The programs and / or rules of the pipeline can be automatically transferred or pushed to the FPPNs in some embodiments, e.g., prior to receiving a qualifying packet that is to be processed using the pipeline; in other embodiments, the programs and / or rules can be stored at one or more slower path resources 123 and transferred to the FPPNs on demand. In some embodiments, pipeline components (rules, programs / actions, and / or related metadata such as lookup tables) can be cached at the FPPNs. In various embodiments, as another preliminary configuration operation associated with the pipeline, one or more ingress and / or egress points (e.g., using one or more virtual network interfaces at the appropriate IVN) can be established for each pipeline instance in response to the respective programming request (e.g., from the vendor for which the pipeline instance is being configured on behalf of).

[0052] After a pipeline element has been deployed, connections can be enabled in various embodiments between the assigned FPPN and the source / destination of traffic for which a pipeline has been deployed. When a packet of a flow is received at such an FPPN, the FPPN can attempt to find a cache rule and / or program that will be used to perform packet processing operations on the received packet. If such a rule or program is found in the cache, the operations indicated in the cache entry can be performed; otherwise, in at least some embodiments, a query indicating that a cache miss has occurred can be sent to the exception path resource 123 along with the received packet. The exception path resource 123 can look up its own metadata repository of IVN configuration information and / or vendor-supplied pipelines. If an appropriate set of one or more pipelines (e.g., to implement IVN configuration settings and / or to perform vendor-defined operations) is identified, the corresponding programs and / or rules can be provisioned to the requesting FPPN in the depicted embodiment, where these programs / rules can be applied and cached for potential reuse. Depending on the details of the identified pipelines, one or more packets corresponding to the received packet (e.g., versions of the packet where one or more headers have been modified) can be transmitted to one or more destinations, and / or additional actions can be initiated in various embodiments (such as simply discarding the received packet and / or raising an alert about a potential network attack).

[0053] Example subcomponents of a packet processing service

[0054] Figure 2Example subcomponents of a packet processing service are illustrated in accordance with at least some embodiments. In the depicted embodiment, serverless packet processing service 201 can include a number of fast path processing nodes (FPPNs) 265, a series of slower path packet processing resources 207, isolated virtual network (IVN) configuration collection and code generation nodes 276, provisioning manager nodes (PMNs) 290, control plane metadata repository nodes (CMRNs) 291, and configuration request handlers (CRHs) 295. Slower path packet processing resources 207 in turn can include IVN configuration based packet processing nodes 270, vendor pipeline management nodes 272, and state information repository nodes (SRNs) 274. In various embodiments, individual nodes of each of the depicted node types can include some combination of hardware and software components. FPPNs 265 and slower path packet processing resources 207 can be examples of SPPS data plane components, whereas IC3Ns 276, PMNs 290, CMRNs 291, and CRHs 295 can be examples of SPPS control plane components. SPPS clients, such as networking application vendors and end users of applications defined by such vendors, can utilize various programmatic interfaces 204 in the depicted embodiment to submit configuration related requests to the SPPS control plane, and receive corresponding replies. Programmatic interfaces 204 may, for example, include a set of APIs (application programming interfaces), a web-based console, command line tools, graphical user interfaces, and the like. When a request is received via an interface 204, it can be examined at a CRH and then passed to the appropriate internal components of SPPS 201.

[0055] In the depicted embodiment, SPPS 201 can be employed in order to implement various types of packet processing operations (corresponding to one or more stages of one or more pipelines) on traffic originating from multiple compute instances (CIs) 260 running at virtualization hosts (VHs) 250, such as CIs 260A and 260B at VH 250A and CI 260C at VH 250B. Compute instances 260 can be configured within one or more isolated virtual networks (IVNs) of a virtualized computing service. In at least some embodiments, the number of FPPNs provisioned and / or assigned by provisioning manager node 290 to handle traffic for a given packet processing application pipeline or appliance instance can be determined based on the number of traffic sources (such as compute instances 260) whose traffic is to be processed. Thus, for example, if more CIs 260 are instantiated over time within an IVN whose outbound traffic is to be processed via a pipeline, additional FPPNs (and / or additional slower path packet processing resources) can be assigned. In some embodiments, a shuffling sharding algorithm can be utilized to assign particular FPPNs 265 to a given CI 260, such that fundamentally different (but potentially overlapping) subsets of the pool of FPPNs are assigned to each CI. As shown by way of example, FPPNs 265A and 265D have been assigned to handle traffic originating from CI 260A, FPPNs 265B and 265D have been assigned to CI 260B, and FPPNs 265C and 265E have been assigned to CI 260C. In some embodiments, a given FPPN can be assigned to a virtualization host 250 (i.e., to handle traffic from all CIs at the host), rather than on a per-CI basis. In one embodiment, in addition to or instead of determining the number of FPPNs based on a count of traffic sources, the number of FPPNs can be determined based on a count of traffic destinations associated with a given packet processing pipeline or appliance. PMN 290 can also be responsible for configuring replacement FPPNs in the event that one of the FPPNs assigned to a pipeline fails, scaling up and down slower path resources 207 as needed, and / or configuring replacement slower path resources in the event that a slower path resource fails.

[0056] At least two categories of packet processing operations can be performed on packets originating from the compute instance 260 in the depicted embodiment: operations corresponding to a pipeline programmatically defined and specified by a networking application vendor, and operations corresponding to configuration settings of the IVN to which the CI 260 belongs (and / or the IVN to which the destination of traffic generated at the CI 260 belongs). In some embodiments, a vendor can submit a request indicating a pipeline via one or more APIs from a SPPS client device 295 (e.g., a desktop, laptop, or mobile computing device). Each of the one or more stages of the vendor-defined pipeline can include at least one rule (e.g., indicating an executable program to run or an action to perform). The rules, programs, and / or actions can be provided to the SPPS by the vendor in any of a variety of formats— e.g., in some embodiments, the rules, programs, or actions can be written in eBPF, whereas in other embodiments, other programming languages or scripting languages can be used. To enable the direction of traffic to and from the pipeline instance, a virtual appliance can be created at the SPPS and one or more ingress and egress endpoints such as virtual network interfaces can be configured for the virtual appliance in various embodiments. In the depicted embodiment, metadata representing the instance of the vendor-defined pipeline as well as the ingress and egress points can be stored in the SPPS CMRN 291, and / or propagated to the vendor pipeline management node (VPMN 272). In some embodiments, the programs and rules of a vendor-supplied pipeline can be pushed or propagated to the FPPN 265B by the VPMN, e.g., upon receipt of the pipeline information at the VPMN or when requested by the FPPN. In various embodiments, the FPPN can store the received programs and rules in a local cache.

[0057] In the depicted embodiment, configuration information related to the IVN to which CI 260 belongs can be obtained at IC3N 276, e.g., from a control plane of a virtualization computing service in which the IVN is configured. In various embodiments, the configuration information can include, e.g., security rules for limiting ingress and egress traffic of the IVN, network access control lists (NACLs), routing rules based on routing tables, etc. At least some of the security settings of the IVN can have been selected by a client on whose behalf the IVN was established. In some embodiments, the IVN configuration settings can be transformed into one or more stages of an internal (i.e., not vendor-defined) packet pipeline, e.g., at IC3N. In the depicted embodiment, operations of the pipeline based on the internal IVN configuration and the vendor-defined pipeline can thus be performed for various received packets; indeed, a single pipeline can be performed that includes both IVN configuration-based operations and vendor-defined operations. In at least some embodiments, the security and routing-related requirements of the IVN configuration can determine the ordering of IVN configuration-related packet processing operations relative to operations of the vendor-defined pipeline. For example, in one scenario, some IVN configuration-related packet processing operations can be performed on a given packet prior to any vendor-defined operations for security-related reasons, and other IVN configuration-related packet processing tasks can be performed after all vendor-defined operations on the given packet for routing-related reasons. In some embodiments, program code (e.g., in eBPF or some other language) for performing the IVN configuration-related operations can be generated at a code generation component or module of IC3N. The IVN configuration-based packet processing node (IPPN) 270 in the depicted embodiment can obtain an indication of the internal pipeline from IC3N and pass the program / rules of the pipeline to the FPPN for caching. If / when the configuration of the IVN changes, then IC3N can automatically determine that a change has occurred and make changes to the IVN configuration-related pipeline as needed. Note that as presented in Figure 1 the IPPN plays a somewhat similar role with respect to IVN configuration-based packet processing as the VPMN plays with respect to vendor-defined packet processing operations. In some embodiments, the IPPN and VPMN can be replaced with a common data SPPS plane component (e.g., a pipeline intermediary node) that is responsible for conveying both IVN configuration-related operation information and vendor-defined operation information to the FPPN.

[0058] In some cases, state information related to packet processing tasks being performed can be stored at one or more SRNs 274. For example, decisions made with respect to an initial packet of a given flow (such as whether the packet satisfies ingress or egress traffic limits of an IVN) that affect operations that can have to be performed for subsequent packets of the same flow (e.g., further ingress limit checks are unnecessary) can be stored in a state information entry at an SRN. If the FPPN to which the decision was propagated fails, its replacement FPPN can query one or more slower path packet processing resources to determine how to continue processing another packet of a flow that was being processed by the now-failed FPPN, and can use the state information entry to respond to such queries. Slower path resources 207 in the depicted embodiment can be responsible for handling cache misses of FPPNs, as well as for revalidating cache entries (or invalidating them). As mentioned earlier, in some embodiments a timeout can be associated with a given cache entry at an FPPN when the entry is stored in the cache. When the timeout expires, the FPPN can send a request to slower path resources 207 to determine whether the rules / actions of the cached entry are still valid. If the entry remains valid (e.g., if no changes have been made to the flow pipeline or IVN configuration), in some embodiments a response can be provided to the FPPN indicating that the timeout will be extended; otherwise, the entry can be allowed to expire or replaced with a new entry generated at the slow path resources.

[0059] Example customer view of packet processing service

[0060] Figure 3 An example customer view of operations performed at a packet processing service in accordance with at least some embodiments is illustrated. In the depicted embodiment, network packets originating from a set of source compute instances (e.g., CIs 360A) of one isolated virtual network (IVN) 320A are to be communicated to another set of destination compute instances (e.g., CIs 360B) within a second IVN 320B, as indicated by arrow 370. Operations of a pipeline defined by a third party vendor (i.e., an entity other than the SPPS operator and other than the source of the traffic) and operations related to IVN configuration, such as security and routing operations, are to be performed for packets of CIs such as 360A. The vendor-defined pipeline can include any desired application-level processing, such as per-user data bandwidth usage analysis, data exfiltration detection, network penetration detection, policy-based routing, etc.

[0061] From the perspective of clients of the SPPS (including end users and third party vendors), the SPPS 310 can appear to be a "black box" that includes a near infinite set of packet processing resources that can be employed for desired types of packet processing tasks. A virtual appliance representing a vendor-defined pipeline can be created at the SPPS in response to a programming request from a vendor, and a pair of appliance gateways 361A and 361B (an ingress point and an egress point for the pipeline) can be configured on behalf of the vendor. In at least some embodiments, the gateways can be assigned respective virtual network interfaces in the IVN in which the gateway is configured - e.g., an ingress point VNI for gateway 361A within IVN 320A and an egress point VNI for gateway 361B within IVN 320B. The gateways can be visible to the client on whose behalf the CI is configured, however other details of the SPPS (e.g., the number of FPPNs and other nodes assigned to the appliance) can remain inaccessible. In the depicted embodiment, both the vendor and the user of the pipeline can have to focus on themselves in requesting or selecting FPPS resources. Internally, within the SPPS, IVN configuration-based (e.g., security-related and routing-related) operations 350 and operations 380 of the third party vendor pipeline can be performed, e.g., using a combination of data plane nodes that are provisioned dynamically based on the number of traffic sources in source IVN 320A. As suggested by the shapes used to represent operations 350 and 380, in the depicted example scenario some IVN configuration-related operations can be performed prior to the vendor pipeline operations, however other IVN configuration-related operations can be performed after the vendor pipeline operations.

[0062] Example summary of fast path operations and slower path operations

[0063] Figure 4 An example summary of fast path and slower path operations at a packet processing service is illustrated, in accordance with at least some embodiments. In the depicted embodiment, resources of the SPPS are organized as a series of cells 427, where each cell includes a respective series of fast path resources and slow path resources. A SPPS client 410, such as an application vendor, can submit a request 443 to the SPPS control plane 414 to establish an instance of one or more pipelines that can be exposed to end users in the form of virtual appliances.

[0064] For a given instance of an appliance, the SPPS control plane 414 can select one or more SPPS data plane cells 427 in the depicted embodiment. A given cell 427 can include a series of fast path packet processing nodes (FPPNs) 468 and a series of slower path packet processing nodes (SPPNs) 425 (which can also be referred to as exception path nodes). A given FPPN 468 can include a cache 497 within which is stored a representation of the rules, actions, and / or programs of a pipeline stage to be implemented at the node. In some embodiments the pipeline stage logic can be stored in executable form in the cache, for example, as byte code expressed using instructions of a register-based virtual machine optimized for implementing network processing operations, which can be used to perform operations similar to those interfacing with the extended Berkeley Packet Filter (eBPF). In one implementation, eBPF programs can be limited in several ways - the maximum number of instructions can be limited, only a few 64-bit registers can be available, only forward conditional jumps can be supported, etc. As a result of the limitations, pipeline logic expressed as eBPF programs can be executed very quickly, but the complexity of the work that can be accomplished using a given program can be limited. For at least some stages of a vendor-defined pipeline, the vendor can supply the program code to be used and cached in the depicted embodiment. For at least some IVN configuration-related operations, the code can be generated at the SPPS control plane 414 (e.g., using a pipeline code generator 426) and also cached at the FPPNs 468.

[0065] Metadata 405 about the pipelines (including IVN configuration related operations as well as vendor defined operations) can be communicated from the SPPS control plane 414 to the SPPN 425, and commands to manage a cell 427 (e.g., to monitor its health status) can be communicated by the control plane 414 to one or more cell management agents instantiated within that cell. When a network packet is received at the FPPN 468 from a source traffic endpoint 464, a local cache 497 can be checked (e.g., using a lookup table or prefix match table) to determine whether code for processing the packet exists in the cache. If a cache hit occurs, i.e., if the appropriate code is found, the code can be executed and one or more packets (which can be rewritten versions of the received packet) can be sent along the fast path 408 to a destination traffic endpoint. If a cache miss occurs, a request 407 can be sent from the FPPN 468 to the SPPN 425. In some embodiments, the SPPN 425 of the SPPS can be able to provide the FPPN with code for processing packets (using the pipeline metadata 405), where the code can be cached for use in subsequent packet instances. In other embodiments, the vendor can have indicated that for at least some packets, at least some packet processing actions can have to be performed using one or more external exception path nodes 411. For example, such client specified external exception path resources can include one or more compute instances within a separate IVN (e.g., different from the IVN in which the source traffic endpoint 464 and destination endpoint 472 are configured, and different from the IVN in which the SPPS cells are configured), or even a collection of compute devices outside of the provider network in which the SPPS is implemented. If an external exception path node 411 is to be used, which can be determined at the internal SPPN 425 based on the pipeline metadata 405, the packet can be communicated to the external node 411 where the required operations can be performed. Depending on the nature of the application implemented using the vendor pipeline, in some cases the external node 411 can only have to be used for a subset of the packets of a given flow; in other cases (e.g., when decryption / encryption operations are required for each packet, and when such operations can only be securely performed using an external node), in some embodiments every packet of a flow can have to be communicated to the external node 411. In at least one embodiment, just as the internal SPPN 425 can be able to provide packet processing code that can be cached at the FPPN 468 for subsequent packets of a flow, the external exception path node 411 can also be able to provide packet processing code that can be cached as the FPPN for subsequent packets of a flow.Depending on the results of the processing performed at the FPPN and / or based on IVN configuration settings, some of the received packets can simply be discarded or dropped in the depicted embodiment (as indicated by arrow 466); thus, the action taken with respect to a given packet can not necessarily include sending the corresponding packet from the SPPS to the destination. For example, if a determination is made that the packet is part of a network attack or data exfiltration effort, or if the configuration settings of the IVN prohibit the transmission of packets from the traffic origin to the target destination indicated in the received packet, and so on, the packet received from the traffic origin can be dropped.

[0066] In some embodiments, a timeout-based cache entry revalidation policy 498 can be employed at the FPPN 468 to ensure that stale cache entries are not used to process packets received at the FPPN. When an entry is stored in the cache 497, a corresponding timeout can be set in such embodiments. When the timeout expires, the FPPN 468 can send a revalidation request for the entry to the SPPN 425. If the cache entry remains valid, that is, if the FPPN is able to continue using it for additional packets, the SPPN 425 can provide a response indicating that the entry can be retained. If, on the other hand, a configuration change has occurred that renders the entry invalid, the SPPN can notify the FPPN accordingly, and the entry can be removed from the cache 497. In some embodiments, when the SPPN determines that a cache entry is no longer valid, it can provide a replacement entry to the FPPN. As mentioned earlier, a multi-tiered timeout-based cache entry revalidation technique can be employed in some embodiments, in which both a soft timeout and a hard timeout can be set for a given cache entry. In at least one embodiment, the SPPN 425 can have to communicate with the external anomaly path node 411 and / or the SPPS control plane 414 to verify that a cache entry remains valid. Other cache invalidation or revalidation schemes can be used in some embodiments. The cell management agent 470 can monitor the health status of the SPPN and FPPN in the depicted embodiment (as indicated by arrow 408) and report their findings to the control plane 414, where decisions can be made about provisioning replacement FPPNs or SPPNs based on these findings.

[0067] Example Summary of Source-Based Node Provisioning

[0068] As mentioned earlier, the number of data plane nodes configured for a given packet processor or pipeline in at least some embodiments can depend on the number of traffic sources for the pipeline. Figure 5An example overview of business-source-based dynamic provisioning of nodes at a packet processing service is illustrated in accordance with at least some embodiments. Isolated virtual network 520 can comprise, at a given point in time, three compute instances 560A, 560B, and 560D from which traffic is to be processed using a pipeline. A set of five fast path packet processing nodes (FPPNs) 565A-565E can be initially configured in the depicted embodiment to process packets generated at CIs 560A-560C, with the number selected based on a count of the CIs. In various embodiments, the SPPS client (the vendor on whose behalf the pipeline is being executed, and the client on whose behalf the CIs 560 are being set up at IVN 520) can not have to request that the SPPS configure a particular number of FPPNs; instead, the FPPN control plane can use its own algorithm to select the number. A shuffling sharding algorithm 570 can be employed to assign respective subsets of the five FPPNs to individual ones of the CIs 560, e.g., to help provide a desired level of redundancy and load balancing. Thus, for example, in the depicted example each CI is assigned a pair of FPPNs: FPPNs 565A and 565D are assigned to CI 560A, FPPNs 565B and 565D are assigned to CI 560B, and FPPNs 565C and 565E are assigned to CI 560C. The particular set of FPPNs to assign to a given CI 560 can be chosen in some embodiments randomly from a pool of available FPPNs, and in other embodiments at least partially based on considerations such as physical proximity. For example, in one embodiment, a number of FPPNs can be configured in a set of racks that are well distributed across a data center, and a pair of FPPNs that happen to be (a) relatively close to the rack at which a given CI is configured and (b) not overloaded (e.g., relative to the number of CIs whose traffic they are configured to process, or relative to the number of packets they have processed per unit of time in a recent time interval) can be selected for the given CI. In some embodiments, rather than being assigned to individual compute instances, FPPNs can be assigned to virtualized hosts using a similar shuffling sharding algorithm. In at least some embodiments, the assignment of FPPNs to traffic sources / destinations such as compute instances can be changed over time, e.g., by the SPPS control plane. For example, a given FPPN FPPN-A can be initially assigned to process packets originating from compute instance CI1 when CI1 comes online, with FPPN-A selected for CI1 at least partially based on physical proximity. Later, a different FPPN, FPPN-B, can be assigned as a replacement for FPPN-A as the FPPN for traffic with respect to CI1, even though FPPN-B can be physically farther from CI1 than FPPN-A.Such dynamic reassignment can be performed in different embodiments for any of a variety of reasons - e.g., because the network path / link between CI 1 and FPPN-A has become more congested than the network path / link available between CI 1 and FPPN-B, because the resource utilization level of FPPN-A exceeds a threshold, or for other reasons.

[0069] If and when additional compute instances such as 560D and 560E that can be added to traffic to be processed using FPPNs are configured in IVN 520, the SPPS control plane can obtain an indication of the new CIs in the depicted embodiment. Based on a set of scale-out policies for FPPNs, the SPPN control plane can automatically configure additional FPPNs (565F) to handle the expected additional traffic in the depicted embodiment, without requiring a client to submit a request for additional resources. A shuffle-shard algorithm can be used to identify the FPPN to use for each new CI. In one embodiment, if the number of configured traffic sources decreases by some threshold, the SPPS can decrease the number of FPPNs assigned to the pipeline. In some embodiments, similar source-based scaling policies can be implemented to scale out (or scale in) other data plane nodes of the SPPS.

[0070] Example distribution of nodes

[0071] Figure 6 An example distribution of nodes of a packet processing service across availability containers of a provider network is illustrated in accordance with at least some embodiments. In the depicted embodiment, provider network 602 includes a virtualized compute service (VCS) 605 at which isolated virtual networks (IVNs) such as 610A or 610B can be established on behalf of various customers or clients. The processing of packets that flow between compute instances 611 (e.g., CIs 611A and 611B in IVN 610A and CIs 611C and 610D in IVN 610B) and / or between compute instances and customer- isolated networks 690A or 690B uses geographically distributed SPPS cells 612 resources. SPPS cells 612 themselves can be implemented using VCS resources as discussed below in the depicted embodiment. In some embodiments, SPPS nodes can be configured within one or more separate IVNs (i.e., IVNs that are dedicated to SPPS and not to a VCS client whose traffic is to be processed using SPPS).

[0072] In at least some embodiments, resources of the VCS 605, such as hosts on which various compute instances run, can be distributed among multiple availability containers 650, such as 650A and 650B. An availability container, in turn, can comprise some or all of one or more different locations or data centers that are engineered in such a way (e.g., with independent infrastructure components, like power-related devices, cooling devices, or physical security components) that resources in a given availability container are isolated from failures in other availability containers. A failure in one availability container can not cause a failure in any other availability container; thus, the availability profile of a given resource is intended to be independent of the availability profile of resources in different availability containers.

[0073] In the depicted embodiment, fast-path packet processing nodes (FPPNs) 625 (e.g., 625A, 625B, 625C, 625D, 625E, and 625F) and slow-path packet processing nodes (SPPNs) 627 (e.g., 627A, 627B, 627C, and 627D) can be implemented at least in part using respective compute instances of the VCS 605. The illustrated cell 612 can comprise at least two data plane sub-cells 640A and 640B, with one data plane sub-cell implemented in each of at least two availability containers 650A and 650B. Due to the use of multiple availability containers, the probability that an entire cell can be affected by any given failure event can be minimized in the depicted embodiment. In some embodiments, SPPS control plane components for managing data plane components can be distributed across multiple availability containers in a similar manner.

[0074] As shown, in the depicted embodiments, SPPS cell 612 can be used for packet processing applications involving services between at least four isolated networks—IVNs 610A and 610B and customer premises isolated networks 690A and 690B. In the depicted embodiments, isolated network 690A, which may be established, for example, at a customer premises or location outside the provider network's own data center and may include a number of hosts such as host 695A, can communicate with SPPS via a set of Virtual Private Network (VPN) paths 608. In the depicted embodiments, isolated network 690B may also be established at a set of customer premises or locations outside the provider network 602 and may include a number of hosts such as host 695K. In the depicted embodiments, services between isolated network 690B and SPPS can flow over a set of dedicated physical links 609 (e.g., rather than a set of links that can be shared with services on the public Internet), which may also be referred to as "direct-connect" links. Therefore, in the depicted embodiments, the SPPS implemented using the VCS 605 with provider network 602 can be able to handle packets generated (or directed to) various network configurations, including (but not limited to) isolated virtual networks within the VCS itself, external networks such as 690A that communicate with the VCS using VPN connections over shared links, and external networks such as 690B that communicate with the VCS using dedicated links. In some embodiments, SPPS can be used to implement VPN and / or direct connectivity functionality. Other types of networks can also be connected using SPPS in various embodiments. In at least some embodiments, the SPPS nodes may not utilize... Figure 6 The types of multi-availability container architectures shown can replace or supplement virtual machines using non-virtualized hosts, and / or may not necessarily use provider network virtualization computing services or other services.

[0075] Example pipeline components and mappings to SPPS nodes

[0076] Figure 7 An example packet processing pipeline comprising multiple levels is illustrated according to at least some embodiments. Two types of pipelines are shown: a vendor-defined pipeline 705 and an IVN-based pipeline 715. In the depicted embodiments, each of pipelines 705 and 715 may have one or more levels represented by corresponding nodes in a directed acyclic graph (DAG), one or more entry points, and one or more exit points. Exit points may include networking constructs (such as virtual network interfaces) capable of directing traffic to or from nodes implementing pipeline logic. Figure 7In the example of FIG. 7, the vendor-defined pipeline 705 can include stages 710A, 710B, 710C, 710D, and 710K, entry point 706A, and exit point 707A, while the IVN- configured pipeline 715 can include stages 720A, 720B, 720C, 720P, and 720Q, entry point 706B, and exit point 707B.

[0077] In the depicted embodiment, these two types of pipelines can differ in their source (i.e., in terms of the entity responsible for defining the pipeline), but can otherwise be quite similar. For example, a given stage’s descriptor 750 can include one or more rules 733, optional lookup data 734 (e.g., a least prefix match table, an exact match table such as a hash table, etc.), and / or one or more actions 735 to be taken based on the results of applying the rules. In at least some embodiments, executable representations of the rules and / or actions, e.g., in the form of a set of programs or functions expressed in eBPF, can be provided by a client of the SPPS as part of a pipeline setup request and / or via other programmatic interactions. Other languages / formats can also or instead be used in other embodiments. In some embodiments, if a stage has multiple rules, each rule can be applied in turn. In at least one embodiment, the rules and / or actions can be conditional. For example, a rule of stage 720A can cause either stage 720B or stage 720C to be executed for a given packet. Conditional rules can also be used in various embodiments to decide whether a particular packet processing program or action is to be executed at a given stage for a given packet: for example, a rule can cause a particular program Progl to be executed at stage SI for packet Pktl, and the same rule can cause a decision not to execute Progl at stage SI for another packet Pkt2.

[0078] Figure 8An example mapping of packet processing pipeline stages to nodes of a packet processing service is illustrated in accordance with at least some embodiments. Depending on the logic of the packet processing application being implemented, a given pipeline can be implemented entirely within a FPPN, or different groups of one or more stages can be performed at respective FPPNs. For example, FPPN 812A can be used to perform operations corresponding to each stage (810A, 810B, 810C, 810D, 810K, etc.) of one pipeline 815A. In contrast, for pipeline 815B, several different FPPNs can be used: FPPN 812B for stages 810P and 810Q, FPPN 812C for stages 810R and 810S, and FPPN 812D for stage 810T. Multiple FPPNs can be used, for example, in scenarios in which network packets of a given flow are transferred from one availability container to another availability container of a provider network, or from one geographic region to another region of a provider network. In some such scenarios, information needed to perform packet processing can not readily transfer across availability container boundaries or region boundaries, so respective FPPNs in different availability containers or regions can be employed.

[0079] Example packet context information

[0080] As network packets are processed at various stages of the pipeline of the kind introduced earlier, information about what has been done at earlier stages of the pipeline can sometimes be useful at later stages. Figure 9 Example context information that can be maintained and modified as packets are processed at various stages of a pipeline is illustrated in accordance with at least some embodiments. In Figure 1 Three types of context information that can be generated and transferred stage-by-stage for a given received packet 916 are illustrated in Figure 1 : privilege data 910 of the SPPS, pipeline instance metadata 912, and command metadata 914.

[0081] Privileged SPPS data 910 can include configuration information related to the flow to which the packet belongs, which is accessed only by SPPS nodes and not accessible from a vendor-supplied packet processing program of the pipeline. Such privileged information can include, for example, results of IVN configuration security checks, network access control lists, and the like, which can indicate that the packet has satisfied IVN security or other constraints and that further processing of the packet is therefore permitted. In the depicted embodiment, pipeline instance metadata 912 can include, for example, an identifier of the instance of the virtual pipeline appliance that is processing the packet, and / or other metadata that is (a) generally applicable to the pipeline instance and (b) accessible to programs of the pipeline as needed from vendor-supplied programs. Command metadata portion 914 of the context can be used to store specific commands to be used when the packet exits the pipeline, if any such commands are needed, e.g., to route a rewritten version of the packet toward its final destination. Context information 902 can be considered a logical equivalent of a "scratchpad memory" that temporarily retains information related to processing of the packet and from which such information is discarded when the pipeline has been traversed. Indeed, the original packet received at the pipeline entry point can be temporarily encapsulated within an envelope containing context information 902 (as well as the packet itself) as the packet travels through the pipeline stages.

[0082] Example multi-stage pipeline for IVN configuration-related tasks

[0083] Figure 10 、 Figure 11 and Figure 12 collectively illustrate example stages of a multi-stage pipeline at which isolated virtual network configuration-related packet processing operations can be performed, in accordance with at least some embodiments. Figure 10 A summary view of all stages of the multi-stage pipeline 1099 is provided, however Figure 11 and Figure 12 show additional details of individual stages and their constituent rules. In the depicted embodiment, each stage includes one or more rules that are executed sequentially; in other embodiments, such sequential execution can not be required. Rules can be applied to determine whether to perform a corresponding action on a packet being processed at a stage; in some embodiments, rules and / or actions themselves can include eBPF functions or programs. At least some operations performed at a stage can be driven by determining whether an entry for a received packet can be found in a lookup table or cache, such as a longest prefix match (LPM) table or a hash table indexed by a key generated from attributes of the received packet. In Figure 10- Figure 12Three types of rules are shown by way of example: Call Rules (CRs), Table-Driven Rules (TDRs), and Filter Rules (FRs). Call Rules can be used to launch or call one stage from another. Table-Driven Rules can utilize a lookup table to determine an action (if any) to perform on a packet; example actions can include, for example, copying information contained in a lookup table entry into context information (similar to that discussed above) of the packet to enable further processing based on the copied information. In some embodiments Table-Driven Rules can themselves include other stages and actions, where the stages are executed prior to the actions. Filter Rules can be used for some types of conditional operations - for example, to determine which stage should be executed next based on information in the context of the packet. Figure 9

[0084] In the depicted embodiment, the pipeline represents stages that can be executed to pass a packet between two virtual network interfaces (VNIs) in the same IVN, between two VNIs in different IVNs, or from an IVN to a boundary or edge networking device of a virtualized computing service (similar to that discussed above with respect to FIG. 1). In the depicted embodiment, the pipeline represents stages that can be executed to pass a packet between two VNIs in the same IVN, between two VNIs in different IVNs, or from an IVN to a boundary or edge networking device of a virtualized computing service (similar to that discussed above with respect to FIG. 1). Figure 10 Figure 11 Figure 12 The pipeline 1099, shown in high level (and in more detail in

[0085] In the depicted embodiment, the pipeline represents stages that can be executed to pass a packet between two virtual network interfaces (VNIs) in the same IVN, between two VNIs in different IVNs, or from an IVN to a boundary or edge networking device of a virtualized computing service (similar to that discussed above with respect to FIG. 1). In the depicted embodiment, the pipeline represents stages that can be executed to pass a packet between two VNIs in the same IVN, between two VNIs in different IVNs, or from an IVN to a boundary or edge networking device of a virtualized computing service (similar to that discussed above with respect to FIG. 1). Figure 1 ​​​The members of the illustrated border appliance fleet 130 perform IVN-related packet processing similar to that of the routing packets. For example, using rules, lookup tables, and actions cached at the FPPN, various types of analysis are performed with respect to packets entering the pipeline from an IVN via the ingress point represented in block 1001 until information needed for the physical network of the virtualization computing service to route the packet toward its intended destination is used for the IVN (block 1022). In the depicted embodiment, the design of the pipeline 1099, including the order of the various stages to be traversed for a given packet, can be determined, for example, by a VCS network administrator; for other pipelines, the application vendor can design the pipeline as discussed above. In at least some embodiments, a hybrid pipeline including IVN-related and vendor-designed operations can be generated (e.g., at the SPPS control plane). In the root stage 1002, for example, three rules (as indicated in Figure 10 and Figure 11 indicated in the above). First, a table-driven rule 1003 to add source VNI metadata in the packet's context information can be applied. Then, a call rule 1004 can be used to launch a stage 1005 in which ingress ACL information can be extracted, stored in the packet context, and evaluated, and finally, an ingress routing table-driven rule 1008 (which itself includes several possible stages and actions) can be applied. Depending on the results of the application of these initial rules, different paths can be taken in the graph of stages.

[0086] Some aspects of the pipeline-based approach illustrated via the pipeline 1099 toward packet processing using small, highly efficient programs that can run very quickly at each stage of the pipeline include the following. A given stage can have one rule (as in stages 1011, 1012, 1018, or 1020) or multiple rules (e.g., stages 1002, 1005, etc.). If a stage has multiple rules, they can be executed in sequence in at least some embodiments. As noted above, a rule can be a table-driven rule (TDR, e.g., rules 1003, 1006, 1007, 1008), a call rule (CR, e.g., rules 1004 or 1015) to launch other stages. Some stages can be launched according to a rule based on evaluated conditions (as in the case of the "if" clause shown in rule 1008). If a rule includes a stage as well as an action (as in the case of rule 1008), the stage can be executed before the action in at least some embodiments. Some or all rules can specify a default action: for example, if no matching entry is found in a lookup table for a packet, the packet can be dropped in some cases (indicated by "Drop packet on hit" shown in various rules including 1003 and 1008), whereas in other cases (as in rule 1016), the fact that no entry was found can simply be ignored (as indicated by "Continue on miss"). Some rules can simply serve to add context information for a packet for analysis during subsequent stages. Some of the operations performed in the IVN configuration-related operations shown in the pipeline 1099 can be security-related (e.g., in rules 1014 or 1017), whereas other operations can be related to the use of encapsulation protocols (needed in environments where the network addresses assigned to sources and destinations can be virtual addresses that must be translated to physical addresses in the underlying VCS network and routes, etc.). Rule 1017 represents an example of a conditional or filtering rule, where the contents of the packet context (which is generated at an earlier stage of the pipeline and stored in the context) are used to select the next stage to use for a given packet. Note that the pipeline 1099 is just one example of a moderately complex multi-stage pipeline that can be implemented using SPPS; in various embodiments, more complex pipelines can be implemented, or less complex pipelines can be used for some types of applications.

[0087] Example platform for a packet processing node

[0088] Figure 13 An example implementation platform for a packet processing node is illustrated in accordance with at least some embodiments. In some embodiments, the platform can be used with Figure 1The virtualization hosts 1320 (e.g., 1320A, 1320B, 1320C, or 1320D) of the VCS-like virtualized computing service (VCS) illustrated in FIG. 13 implement SPPS components such as FPPNs, SPPNs, and the like. At a given point in time, a virtualization host 1320 can include zero or more compute instances (such as guest virtual machines) 1321 (e.g., 1321A-1321D) and a set of virtualization management components 1322 (e.g., 1322A-1322D). Various virtualization management components (VMCs) can be used in different embodiments, including, for example, hypervisors running on the host processor (CPU) of the virtualization host and offloaded VMCs that do not utilize the host processor for at least some of their applications. A VMC can be referred to as "offloaded" if it takes over some virtualization management tasks (e.g., associated with encapsulation protocol processing, etc.) that would otherwise be performed on the host processor of the virtualization host, thereby offloading or reducing the workload of the host processor. In some embodiments, an offloaded VMC can include logic implemented on a system on a chip (SOC) of a card connected to the CPU via a peripheral interconnect such as a PCIe (Peripheral Component Interconnect Express) interconnect. The computing capabilities of the SOC or other elements of such a card can be used for SPPS in some embodiments, for example, in addition to or instead of the host processor.

[0089] In Figure 13 A number of alternative approaches to running SPPS nodes on virtualization hosts are illustrated by example in FIG. 13. For example, in one embodiment, SPPS node 1313A can run within compute instance 1321A of virtualization host 1320A, for example, using one or more user-mode processes, and no SPPS nodes can run using the VMC of the virtualization host. In another configuration, as illustrated by virtualization host 1320B, multiple SPPS nodes 1313B and 1313C (e.g., several different FPPNs, SPPNs, or a combination of FPPNs and SPPNs) can run in a single compute instance in some embodiments. In other embodiments, one or more SPPS nodes such as 1313D and 1313E can run at the VMC 1322B of the virtualization host. In one embodiment, SPPS nodes can run within a compute instance (such as SPPS node 1313G at CI 1321D of virtualization host 1320D) and also within a VMC (such as SPPS 1313F at VMC 1322D of virtualization host 1320D). In at least some embodiments, SPPS nodes can be implemented within an IVN of the virtualized computing service that is dedicated to packet processing operations (and thus is different from the IVN at which the source / destination of the packet is configured).

[0090] In some embodiments, SPPS nodes can run at non-virtualized hosts (e.g., SPPS nodes 1313H and 1313J at host 1360). According to one embodiment, dedicated special-purpose packet processing hardware devices 1370 can be employed for some SPPS nodes such as 1313K and 1313L. In some data centers of a provider network, or at external networks that establish SPPS data plane extensions as described below, special-purpose packet processing hardware devices 1370 can be used. Figure 13 Combinations of the SPPS node implementation approaches illustrated in the figure are possible.

[0091] Example packet processing applications

[0092] Figure 14 Example categories of packet processing applications that can be implemented at a serverless packet processing service using pipelines according to at least some embodiments are illustrated. In general, there are a wide variety of packet processing or transformation types that can be implemented at each layer, and the SPPS 102 shown can be configurable to implement any desired type of packet processing or transformation (or a combination of different types of packet processing or transformations), with nodes being dynamically assigned at each layer to support a wide range of traffic rates in a transparent and scalable manner. Figure 1

[0093] As the name suggests, the source address substitution application 1402 can involve replacing the source address and port for packets of a particular flow in a consistent manner. (As mentioned earlier, packet flows can be distinguished from one another in some embodiments based on a combination or tuple of attributes such as the network protocol in use, the source and destination network addresses, the source and destination ports, a virtual network interface identifier associated with a pipeline or virtual appliance, and / or the direction of traffic.) Source address substitution techniques can be useful, for example, when there is an overlap between private address ranges of two or more isolated networks, or when the actual addresses of one or more servers implementing a service are to be hidden from service consumers using a load balancer, and in some embodiments an SPPS can be employed as an intermediary responsible for such substitutions. In some embodiments, such address substitution operations can also be referred to as "managed network address translation (NAT)." Multicast 1404 is a networking technique that can be implemented using an SPPS in some embodiments, in which the content (e.g., body) of a single packet sent from a source is copied to multiple destinations of a specified multicast group. In contrast, anycast 1406 can involve selecting a particular destination from among a specified set of destinations for all packets of a given flow received at an SPPS.

[0094] ​As indicated by the name, scalable load balancing 1408 involves distributing packets among a set of servers in order to prevent overloading of individual servers. For scalable cross-IVN channel 1410, in some embodiments SPPS can actually be used as an intermediary between two or more distinct IVNs' dedicated address spaces. In at least some embodiments, SPPS can likewise or alternatively be used to support scalable VPN connectivity 1412 between a certain set of resources within a provider network and one or more client networks or client premises outside the provider network or multiple client premises networks linked to SPPS via VPN connections each. In some embodiments, a provider network can support connectivity with external networks via dedicated physical links referred to as "direct connect" links, and can use SPPS to manage traffic between such external networks (as well as between such external networks and IVN or VPN-connected external networks).

[0095] In one embodiment, SPPS can be employed, e.g., by a telecommunications service provider, for per-account bandwidth usage tracking 1414. In at least some embodiments, SPPS can be used for intrusion or attack detection 1416 (e.g., detection of unexpected or unwanted traffic at resources of a particular enterprise or government entity). In some embodiments, SPPS pipelines can be used to perform data exfiltration detection 1418 (e.g., detection of unwanted / unauthorized transmission of data out of an organization's network). Note that a single instance of a virtual appliance or pipeline established at SPPS can combine Figure 14 several packet processing functions (and / or other packet processing techniques) indicated in Figure 14 Other categories of packet processing applications can be supported using SPPS in different embodiments, although some embodiments can not support

[0096] Example entities managed using APIs

[0097] Data structures representing various types of objects and entities associated with packet processing can be stored in SPPS in various embodiments in a manner similar to the SPPS 102 of Figure 1 , and can support application programming interfaces for accessing and manipulating such data structures. Figure 15Figure illustrates example types of entities that can implement application programming interfaces at a packet processing service, according to at least some embodiments. In some embodiments various pipeline definitions 1501, e.g., including information about stages of a pipeline, rules to be applied at each stage, kinds of lookup tables to be used for stages, etc., can be provided by clients of the SPPS. As noted earlier, a given pipeline can include stages with operations defined by an application vendor, stages with operations defined by a network administrator associated with IVN configuration (including security and routing-related operations), and / or stages for performing a combination of vendor-defined operations and IVN configuration-related operations. Packet processing applications 1507 can represent implementations of pipeline definitions 1501; thus, for example, one application Al can implement pipeline definition PD1 for a certain set of business endpoints TE1, while another application A2 can implement the same definition for another set of business endpoints TE2. Over time, modifications can be made to pipeline definitions (e.g., eBPF code for stages can be improved), and it can be necessary to propagate such changes to end users without interrupting use of existing versions. Deployment policies 1504 can be specified in some embodiments to indicate how various versions of pipeline definitions 1501 will be rolled out or propagated, enabling business to be shifted to new versions as transparently as possible.

[0098] From the perspective of end users of packet processing applications, in the depicted embodiments at least two types of virtual entities can be used to expose applications: appliances 1510 and attachments 1513. For example, an application vendor can request that the SPPS (a) create an intrusion detection appliance 1510 that implements an application including a pipeline designed by the vendor and (b) make appropriate networking configuration changes to generate an attachment 1513 at an IVN where the vendor’s business will be analyzed for intrusion detection. In some embodiments, a particular version of a pipeline can be “pinned” to an appliance, e.g., metadata can be stored at the SPPS indicating that a new version is not to be used for the appliance. In various embodiments, an attachment can actually create one or more ingress and / or egress points reachable from sources and destinations of traffic to be processed. In some embodiments, attachments can also be used to implement connectivity between two appliances or two pipelines.

[0099] For example, the networking configuration operations required to create an attachment can include the establishment of one or more virtual network interfaces (VNIs). In various embodiments, a VNI can include a logical entity with a set of networking and security-related attributes that can be programmatically attached to a compute instance and / or other computing device (or detached from a compute instance and / or other computing device), or used for routing purposes within an IVN without necessarily being attached to a compute instance. For example, a given virtual network interface VNIi can be assigned at least one IP (Internet Protocol) address "IPaddri", and VNIi can be set with security rules that limit inbound and outbound traffic. When that VNI is programmatically attached to a given compute instance CIi launched at a host with a physical network interface card NICi, network packets indicating IPaddri as their destination address (and complying with the security rules) can be received at CIi via NICi. In addition, outbound packets generated at CIi can indicate IPaddri as their source address and can be physically transmitted toward their destinations via NICi. If VNIi is subsequently programmatically detached from CIi and attached to CI2 (which is executing at a different host with a different physical network interface card NIC2), then IPaddri traffic previously received at CIi can now be received at CI2, with the same security rules in place. In the depicted embodiment, an attachment 1513 can use one or more VNIs in any of the following three ways: (a) a standard VNI 1528 can be established for the attachment (e.g., with a dedicated IP address within the target IVN), (b) a VNI pool 1531 can be established that includes multiple VNIs, e.g., as a gateway within a geographic region of a provider network, or (c) an unnumbered VNI 1534 can be used (with a VNI that can be referred to as a destination in a routing table entry but has not yet been assigned an IP address).

[0100] An object representing an exception path 1516 can be generated for some appliances to handle packet processing operations that will not be performed at the FPPN - e.g., because the FPPN can limit the maximum size of programs that can run on the FPPN (and thus limit complexity), because an application vendor wants certain packet processing to be performed at a custom or enhanced security resource, or for other reasons. In some embodiments, as noted above, the maximum allowed size of packet processing programs that can run at the FPPN can be too small to complete some desired packet processing operations, or the maximum size of lookup tables that can be stored / inspected at the FPPN can be too small to perform some desired table-based packet processing tasks, and an exception path can be used in such scenarios in some embodiments. At least two types of exception path processing nodes can be employed in various embodiments: SPPS-managed nodes and client-controlled nodes. For example, if a pipeline's vendor or designer provides a description or specification of packet processing operations that cannot be performed using FPPNs to the SPPS, and allows the SPPS to perform those operations using a pool of computing devices selected by the SPPS (and scaled up or down as needed by it), then a SPPS-managed exception path (MEP) 1519 can be utilized. In the depicted embodiment, the client on whose behalf the MEP is implemented can not have to make resource provisioning decisions for the exception path operations; instead, the SPPS control plane can perform resource provisioning, health state monitoring, failure recovery, etc. for the exception path.

[0101] In alternative approaches, some clients can not provide the SPPS with details of their exception path operations, but instead can use a client-controlled or custom exception path (CEP) 1522. In such scenarios, the client can be responsible for resource provisioning, health state management, failure recovery, etc. of the computing devices used for exception path processing in various embodiments. In the depicted embodiment, the client can provide an indication of a set of target resources 1525 to which packets should be sent for exception path operations (e.g., IP addresses of one or more client-controlled or client-owned devices). In at least one embodiment in which traffic generated at (or directed to) one or more end user IVNs is to be processed using a pipeline of the kind described above, the resources for a client-controlled CEP can reside in a different IVN (e.g., an IVN configured on behalf of an application vendor) or even outside the provider network in which the end user IVNs reside. In some embodiments, in addition to the entities shown, other types of entities can be used to represent metadata associated with packet processing operations. Figure 15 In addition to the entities shown, other types of entities can be used to represent metadata associated with packet processing operations.

[0102] Example programming interactions

[0103] Figure 16 and Figure 17Figure illustrates programmatic interactions related to an example packet processing pipeline, in accordance with at least some embodiments. In Figure 16 and Figure 17 the depicted embodiments, SPPS 1640, which is similar in features and functionality to SPPS 102 of Figure 1 may implement a set of programmatic interfaces 1677 that can be used by clients such as application vendors and / or end users of packet processing pipelines to submit various types of configuration-related requests and receive corresponding responses. In different embodiments, programmatic interfaces 1677 can include, for example, a set of application programming interfaces, command-line tools, graphical user interfaces, web-based consoles, etc.

[0104] As shown in Figure 16 , SPPS client 1630 (such as a vendor of a packet processing application or a network administrator of a virtualized computing service) can submit a pipeline definition to SPPS 1640 in a StorePipelineDefinition request 1602. In the depicted embodiment, descriptors for stages of the pipeline, including rules to be executed at each stage, kinds of lookup tables to be used, actions to be performed, etc., can be included in the StorePipelineDefinition request. In at least some embodiments, executable or source code versions of programs (e.g., in eBPF) to be executed to implement the rules and actions can be included in the request. In one embodiment, the SPPS can provide an integrated development environment (IDE) and / or other tools that can be used by client 1630 to generate pipelines, verify that the pipelines meet resource limits (e.g., memory size limits, program length limits, etc.) of the FPPN of the SPPS, etc. In at least some embodiments, prior to accepting or deploying a pipeline, the SPPS can implement an internal management workflow to ensure that programs provided by a pipeline submitter meet desired isolation criteria— e.g., that the programs of a given pipeline do not attempt to access data structures intended to be accessed only from other pipelines. Programs / pipelines that meet such isolation criteria can be safely run in a multi-tenant manner, e.g., the programs of pipeline PI (provided by application vendor VI) can be run on the same FPPN or host as the programs of pipeline P2 (provided by application vendor V2) as needed. In response to receiving StorePipelineDefinition request 1602, a representation of the pipeline can be stored in a repository of the SPPS, and in some embodiments a PipelineDefinitionStored response 1604 can be provided.

[0105] To present an instance of a pipeline to a set of end users, a client on behalf of which the definition is stored can submit a CreateAppliance request 1606, e.g., indicating a pipeline definition to be implemented using the appliance and / or the set of compute instances that its service will use the instance to process. In response, the SPPS control plane can assign a set of FPPNs and / or other nodes, e.g., based on a count of the source of the service, store metadata identifying the appliance and the nodes selected for the appliance, and in the depicted embodiment provide an appliance identifier for the newly created appliance in an ApplianceID message 1608.

[0106] A client 1630 can submit a CreateAttachment request 1610 via the programmatic interface 1677 in some embodiments, e.g., to establish one or more VNIs (virtual network interfaces) or other constructs that can be used as targets (pipeline entry points) for service to be processed or as sources (pipeline exit points) from which results of packet processing operations performed at the pipeline can be obtained. Appropriate networking configuration operations (e.g., creation of VNIs, assignment of addresses, generation of routing table entries, etc.) can be performed at the SPPS, and in some embodiments an attachment identifier can be provided in an AttachmentID message 1612.

[0107] In some embodiments, creation of an attachment in response to a request 1610 can enable service to flow to the pipeline. In other embodiments, an ActivateAppliance request 1613 can be sent to cause packets to be directed to the pipeline via the attachment; that is, creation of an attachment alone can not be sufficient to begin processing packets at the appliance. In response to an ActivateAppliance request 1613, appropriate network configuration changes can be made at the SPPS 1640 to enable packets to be directed to the FPPN of the appliance, and an ApplianceActivated response 1615 can be sent to the client.

[0108] If the creator of a pipeline or some other authorized client wishes to change the pipeline (e.g., by modifying rules / actions / procedures corresponding to one or more stages, by adding / combining / eliminating one or more stages, etc.), a ModifyPipeline request 1616 indicating the changes can be submitted to the SPPS in some embodiments. If the changes are found to be acceptable, they can be applied, e.g., by creating a new version of the pipeline without removing the existing version, and a ModificationComplete message 1618 can be sent to the requester.

[0109] In some embodiments, a DeployModifiedPipeline request 1620 can be sent by the client to SPPS to propagate the new version of the pipeline to the appropriate data plane nodes. After the propagation of changes (e.g., according to...) Figure 15 (The deployment strategies discussed in the context of the above) can be sent by SPPS to the client as PipelineDeployed message 1622.

[0110] A DescribePipeline request 1624 can be submitted by the client at 1630 to obtain information about the created pipeline definition. The requested information can be provided by SPPS in a PipelineInfo message 1626. Similarly, to understand the attachment sets created for one or more appliances, a DescribeAttachments request 1628 can be submitted, and the requested information can be provided in an AttachmentInfo message 1630.

[0111] In at least some embodiments, SPPS can capture metrics associated with various appliances, including, for example, the total number of packets processed per unit time, the total number of bytes transmitted per unit time, etc. Such metrics can be obtained by a client 1630 submitting a GetApplianceMetrics request 1631, and in the depicted embodiments, the requested metrics can be provided using a MetricSet response message 1632.

[0112] like Figure 17 As shown, in some embodiments, the client can submit a ConfigureCustomExceptionPath request 1702 to request the establishment of the above... Figure 15 The context discusses client-managed exception paths. For example, request 1702 may include instructions for one or more targets selected by the client, to which packets may be sent for exception path handling associated with a pipeline or appliance. The information provided in request 1702 may be stored at SPPS, for example, as part of the appliance's metadata, and may be sent as a response with a CEPConfigured message 1704.

[0113] In various embodiments, the client can modify the custom exception path, for example, by adding or removing a target, or changing the pipeline level from which the exception path is accessed to obtain a given group, and such changes can be indicated via a ModifyCustomExceptionPath request 1706. After the representation of the changes has been stored at SPPS, a CEPModified response 1708 can be provided to the client.

[0114] In some embodiments in which a custom exception path is configured for a packet processing application, a client can wish to ensure that the target established for the path will respond to packets sent to them, and the SPPS 1640 can provide a mechanism to test the target. Such a mechanism can be initiated by sending a TestCustomExceptionPath request 1710 (e.g., indicating an example of a kind of packet to send to the custom exception path target and a kind of response expected), and the results of the test can be provided in a TestResult message 1712.

[0115] A ConfigureSPPSManagedExceptionPath request 1713 can be submitted by a client 1630 to request establishment of an SPPS-managed exception path of the kind discussed above in the context of Figure 15 SPPS 1640. For example, the request 1713 can include an indication of the operations to be performed at the SPPS-managed exception path. The information provided in the request 173 can be saved at the SPPS, e.g., as part of the metadata for the appliance, and a MEPConfigured message 1715 can be sent in response.

[0116] In various embodiments a client can make changes to an SPPS-managed exception path, e.g., by indicating a new sequence of operations to be performed, and such changes can be indicated via a ModifySPPSManagedExceptionPath request 1716. After the representation of the change has been stored at the SPPS, a MEPModified response 1718 can be provided to the client.

[0117] In at least some embodiments, information describing the configured exception paths of an appliance or pipeline, e.g., for SPPS-managed or client-controlled custom exception paths, can be obtained using a DescribeExceptionPaths request 1720. The SPPS can provide a response containing the available information in an ExceptionPathInfo message 1722.

[0118] In at least one embodiment, SPPS can capture metrics associated with exception paths established for various appliances, including, for example, a total number of packets processed via an exception path per unit of time, a total number of bytes communicated using an exception path per unit of time, and so forth. A GetExceptionPathMetrics request 1724 can be submitted by a client 1630 to obtain such metrics, and an EPMetricSet response message 1726 can be used to provide the requested metrics in the depicted embodiment. In one embodiment, SPPS 1640 can be able to monitor the health state of at least some custom exception path resources being used on behalf of a client (e.g., by sending ping-like messages to probe whether a resource is responsive), and can provide reference health state metrics to a client via an EPMetricSet message 1726. In some embodiments, programming interactions other than those shown can be used to perform pipeline configuration related operations. Figure 16 and Figure 17 Programming interactions other than those shown can be used to perform pipeline configuration related operations in some embodiments.

[0119] Example exception path packet processing scenario

[0120] Figure 18 An example exception path packet processing scenario is illustrated in accordance with at least some embodiments. In the depicted embodiment, three types of nodes that can perform packet processing operations are shown: a “standard” FPPN 1801 of an SPPS, exception path packet processing nodes (EPPNs) 1851A and 1851B configured in a pool managed by the SPPS, and EPPNs 1851K and 1851K used for a client-controlled custom exception path.

[0121] In the depicted embodiment, standard FPPNs 1801 can include a restricted packet processing engine 1805. Such an engine, which can for example include a register-based virtual machine that can be used for eBPF programs, while able to quickly execute packet processing programs, can be constrained with respect to the complexity and size of tasks that can be performed. For example, the total amount of memory available for a given program can be small, such that large lookup tables cannot be used, or the maximum number of instructions that can be included in a program can be low. Only certain types of executable instructions can be supported in some embodiments: for example, some types of loop constructs can not be allowed in programs executed at the engine 1805. The engine 1805 can be sufficient for a wide variety of packet processing operations, but some applications can require access to larger tables, or require more complex programs to be run for at least a subset of packets received at the pipeline. Packet processing operations to be performed can require one or more security artifacts (e.g., encryption keys) that are not accessible from the standard FPPN 1801 in some cases— i.e., the FPPN can not be granted access to, or provided with, the security artifacts. For example, some packet processing applications can involve terminating a secure networking session, such as a TLS (Transport Layer Security) or SSL (Secure Sockets Layer) session, and encryption keys for such applications can not be available from the FPPN. In some embodiments, at least some FPPNs can not have the computational resources required to perform encryption / decryption operations required for some types of packet processing (i.e., the security artifacts used for such operations are available at the FPPN as well).

[0122] To meet such demands that cannot be met by standard FPPNs 1801 of the SPPS, one or more EPPNs 1851 can be configured in the depicted embodiment. Such EPPNs can have less restricted computational resources 1855 (e.g., 1855A, 1855B, 1855K, or 1855L) with respect to the kinds of tasks that can be performed. In some cases, a pool of EPPNs 1805 can be established that is managed by the SPPS control plane, using SPPS resources 1807 to manage, including an autoscaling manager 1871, a health state manager 1872, and a provisioning manager 1872. If and when exceptional path workloads increase at such a pool (e.g., due to configuration of additional sources of traffic), or EPPNs fail, additional resources can be deployed by the management resources 1807. In effect, a client on behalf of which the pool of EPPNs managed by the SPPS is established can obtain similar benefits with respect to scalability, availability, and reliability as are obtained for standard FPPNs, without being limited to the constraints of standard FPPNs.

[0123] In some cases a client can not necessarily want to use SPPS-managed resources for exceptional path processing. This can be the case, for example, if proprietary, highly sensitive packet processing algorithms (e.g., using one or more security artifacts 1858 such as cryptographic keys) are to be used that the client does not want to expose to the SPPS, or if rapid changes to packet processing algorithms or lookup tables are expected. In some embodiments, the SPPS can enable such users to establish a pool of client-managed EPPNs 1830 such as EPPNs 1851K and 1851L. In some cases, machine learning models 1859 and / or other datasets (which can not necessarily be security-related) can be used for packet processing at EPPNs such as 1851L, e.g., in scenarios where it can not be practical to use such models or datasets at the SPPS’s standard FPPNs, or simply because the SPPS client does not want to provide the models or datasets to the SPPS. In some implementations a SPPS’s standard FPPN and / or other data plane resources can be established within an SPPS-managed IVN, and a client-managed EPPN 1830 can be configured within a different IVN (e.g., an IVN established on behalf of an application vendor); note that the IVN other than these two is different from the IVN whose traffic is to be processed. In at least one embodiment, a client-managed EPPN can be established at a premises outside of the provider network. In some embodiments, at least some security artifacts 1858 that are not accessible at the FPPN (such as cryptographic keys that can be used for TLS / SSL sessions) can be communicated to the SPPS-managed EPPN to enable some types of security-related packet processing operations that cannot be performed at the standard FPPN that is to be executed at the SPPS-managed EPPN.

[0124] Example extensions of packet processing services

[0125] Figure 19 An example system environment is illustrated in accordance with at least some embodiments in which data plane extensions of a provider network’s packet processing services can be implemented at a premises outside of the provider network. As shown, system 1900 can include resources and artifacts of several network-accessible services of a provider network 1901, including a virtualized computing service (VCS) 1910 and a serverless packet processing service (SPPS) 1902. The provider network 1901 can include one or more data centers in the depicted embodiment. Each provider network service can include a set of management or control plane resources located within a data center of the provider network (e.g., SPPS control plane resources 1926 and VCS control plane resources 1941, which can include multiple control plane servers 1912 such as 1912A or 1912B).

[0126] At a high level, the VCS 1910 and SPPS 1902 can provide functionality and features similar to those of the VCS and SPPS discussed in the context of earlier figures including Figure 1

[0127] A subset of data plane resources of both the VCS and SPPS can be configured in the depicted embodiment at customer data centers such as 1932A and 1932B and / or at other facilities outside of the provider network such as co-location facilities, etc. (e.g., in response to a programmatic request). For example, a VCS data plane expansion resource group (ERG) 1935A can include a set of compute instances 1918J and 1918K established at a virtualization host at customer data center 1932A, and another VCS data plane expansion resource group 1935B can be established at a virtualization host located at customer data center 1932B including compute instances 1918P and 1918Q. In the depicted embodiment, respective CIs in the VCS data plane ERGs can be configured within corresponding IVNs. CIs 1918J and 1918K can be configured within IVN 1915A, thereby logically extending IVN 1915A to customer data center 1932A, whereas CIs 1918P and 1918Q can be established within IVN 1915B, thereby logically extending IVN 1915B to customer data center 1932B. To securely perform configuration operations associated with VCS data plane expansions, a combination of techniques can be employed in different embodiments - for example, a set of enhanced security virtualization hosts with a verified / attested software stack can be delivered to a customer data center in a pre-configured rack, a special secure boot sequence can be used to launch respective control plane agent processes within the rack, and / or one or more outbound command communicators (OCCs) 1977 can be employed to transmit VCS control plane commands from the provider network data center to the agents via a secure (encrypted) communication channel.

[0128] ​In some embodiments, similar techniques can be used to establish SPPS data plane extension resources such as 1927A or 1927B at customer data centers 1932A and 1932B. For example, a set of enhanced security hardware appliances with a certified / proven software stack for SPPS data plane nodes (e.g., including a number of FPPNs and a number of SPPNs) can be delivered to a customer data center within a pre-configured rack, a special secure boot sequence can be used to launch the corresponding SPPS control plane agent processes within the rack, and / or one or more outbound command communicators 1925 can be employed to transmit SPPS control plane commands from the provider network data center to the agents via secure (encrypted) communication channels. In some embodiments, a given rack of hardware appliances established for VCS data plane extensions can also be used to host SPPS data plane nodes at an external premises— e.g., some SPPS nodes can be implemented using corresponding compute instances at the same facility as the compute instances whose packets will be processed using the pipelines that the SPPS implementation is handling. In one embodiment, SPPNs established at customer data center 1932 can include one or more state information repository nodes (similar to SRN 274 shown). In such embodiments, only a subset of the state information for the pipelines that are being implemented that are needed to handle the traffic for customer data center 1932 can be stored at such SRNs. Note that in at least some embodiments, when a client of a SPPS or VCS issues configuration or management related requests related to any one of the data plane extensions of the service, these requests can be transmitted to the service control plane using the same kind of paths that are used for requests related to data plane resources within the provider network; that is, in such embodiments, inbound requests can not require command communicators 1977 or 1925. Figure 1

[0129] ​In various embodiments, SPPS control plane resources 1926 can communicate packet processing metadata 1944 (e.g., 1944A or 1944B) from a provider network data center to external premises where SPPS data plane nodes are established, e.g., using one or more outbound command communicators 1925. For example, an SPPS client such as a vendor of packet processing pipelines or appliances can communicate an indication of a pipeline to be implemented for traffic of a given IVN 1915 to SPPS control plane resources. IVN integration resources 1924 of the SPPS control plane can collect configuration information for the IVN and can generate corresponding IVN configuration related packet processing pipelines in various embodiments as discussed earlier. In the depicted embodiment, a representation of at least a portion of the vendor submitted pipeline and IVN configuration related operations (e.g., in the form of eBPF programs that can be cached at FPPNs) can be communicated to SPPS data plane scaling resources 1927A or 1927B. In various embodiments, e.g., in response to a programming request received at the SPPS control plane, instances of packet processing appliances can be established at external premises and entry points and exit points for the appliances. In various embodiments, the number of external premises based nodes allocated to a given appliance or pipeline instance can be increased or dynamically increased by the SPPS control plane, e.g., based on the number of traffic sources for the pipeline. Similarly, in at least some embodiments, a particular set of SPPS nodes based on external premises that will process packets for a given traffic source of a processing pipeline can be identified using similar shuffling sharding techniques to those illustrated in FIG. 6. Figure 5 Similar shuffling sharding techniques to those illustrated in FIG. 6 can be used to identify a particular set of external premises based SPPS nodes that will process packets for a given traffic source of a processing pipeline.

[0130] When a packet is received at an SPPS data plane extension resource from a traffic source of a pipeline instance (e.g., a source at an external premises or a source within a provider network data center), an appropriate combination of IVN configuration related packet processing operations and provider defined packet processing operations can be performed at the extension resource. For example, if a rule and / or action for the packet is found in an FPPN cache at an external premises, they can be executed; otherwise slower path resources can be consulted to determine operations to be performed by the FPPN. Based on the results of packet processing performed at the SPPS data plane extension for a given received packet, in some cases one or more packets corresponding to the received packet (e.g., transformed versions of the received packet) can be transmitted to a destination (in a provider network data center, at a local premises, or at a different location); in other cases, the given received packet can simply be discarded based on the packet processing results. In at least some embodiments, custom exception path targets can also be established for pipelines instantiated at an external premises (similar to the earlier discussed targets for performing operations not performed at an FPPN of an SPPS) - for example, a target can be used that can be co-located with an SPPS data plane extension at an external premises, a target within a provider network data center can be used, or other external premises targets can be used.

[0131] In some embodiments, an external premises such as customer data center 1932 can include respective sets of customer managed servers (CMS) such as 1923A and 1923B that can be linked to a VCS data plane ERG via local premises data channel 1968A or 1968B. In various embodiments, the CMS can serve as a source and / or destination for some traffic that is processed using an SPPS data plane extension. In some embodiments, the CMS can for example include servers for applications that run from a CI of a local VCS data plane ERG, and locating the CI in proximity to such applications can help enable lower latency for message traffic associated with such applications.

[0132] According to at least some embodiments, SPPS data plane extension resources 1927 can likewise or alternatively be established at other types of premises, and can not be limited to customer data centers. In some cases, external premises at which SPPS data plane extension nodes are established can not necessarily include VCS data plane ERGs. In some embodiments, a portion of a given pipeline can be implemented using provider network-based SPPS data plane nodes, and another portion of the same pipeline can be implemented using one or more SPPS data plane extensions. In one embodiment, outbound packets generated as a result of packet processing at SPPS data plane extension nodes at one external premise can be transmitted to another SPPS data plane extension node at another external premise— that is, some pipelines can be implemented using SPPS data plane extensions at multiple premises external to a provider network data center.

[0133] Example premises for packet processing service extensions

[0134] Figure 20 An example of the types of premises in which packet processing service extensions can be implemented according to at least some embodiments is illustrated. As Figure 19 Additionally illustrated, in one example scenario, an extension resource 2027A of a data plane of a serverless packet processing service (SPPS) can be deployed at a customer data center 2032. The customer data center 2032 can in some cases include an extension resource group 2035 of a virtualized computing service (VCS), e.g., including a number of compute instances (CIs) established at an enhanced security virtualization host, such as 2018A and 2019B, although in at least some embodiments, SPPS data plane functionality to be extended to a customer data center can not require establishment of a VCS ERG 2035. In the depicted embodiment, a local premise data channel 2068 can connect the SPPS data plane resource 2027A and / or the VCS ERG 2035 to a set of customer-managed servers 2023.

[0135] According to at least one embodiment, SPPS extension resources can be deployed at premises 2034 at which a portion of a population of provider network's edge appliances is located. For example, in some embodiments, VCS edge appliance population nodes 2037 can include a number of edge routers (e.g., 2022A and 2022B) configured to mediate network packets received from the public Internet (or sent to the public Internet) or received from other networks, and such edge routers can be established within a special geographically distributed premises 2034 separate from the provider network's main data centers. For traffic leaving or entering the provider network, e.g., via virtual private network (VPN) tunnels or the like, in such embodiments, hosting SPPS data plane extension resources 2027B in the same premises as the edge appliances can help improve the performance of packet processing applications, in such embodiments.

[0136] As mentioned earlier, SPPS resources can be employed in at least some embodiments on behalf of packet processing application vendors as well as other provider network clients using applications developed by such vendors. In some embodiments, SPPS data plane extension resources 2027C can be deployed at an application vendor's data center 2036, which can also include vendor-owned computing appliances 2036, including vendor-managed servers 2023. In some embodiments using vendor-managed servers to establish custom exception path targets (functionally similar to exception path resources discussed earlier in the context of Figure 17 or Figure 18 SPPS extension resources can be especially beneficial in some embodiments in which custom exception path targets are established using vendor-managed servers (functionally similar to exception path resources discussed earlier in the context of

[0137] Some provider network clients can wish to utilize dedicated high-bandwidth physical links (i.e., links not shared with traffic of the public Internet) for traffic flowing between the provider network and clients outside the provider network. In various embodiments, connectivity achieved using such dedicated physical links can be referred to as "direct connect" connectivity. In some embodiments, co-location facilities 2038 at which provider network's networking resources 2065 (e.g., routers) as well as third-party networking appliances 2066 (or customer-owned networking appliances) are hosted can be employed in order to facilitate direct connect traffic. In some embodiments, SPPS data plane extension resources 2027D can also be deployed at such co-location facilities, e.g., to allow for implementation of complex packet processing pipelines of the kind introduced earlier for traffic via dedicated physical links 2091A (to / from provider network data centers) and 2091B (to / from customer data centers). In some embodiments, SPPS data plane extensions can be deployed at other types of premises not shown in FIG. 20. Figure 20

[0138] ​Example SPPS extension-related programmatic interactions

[0139] Figure 21 Example programmatic interactions related to packet processing service extensions are illustrated in accordance with at least some embodiments. In the depicted embodiment, SPPS 2140, which supports data plane extensions to premises outside the provider network, can implement one or more programmatic interfaces 2177, such as a set of APIs, command line tools, web-based consoles, graphical user interfaces, etc. An EstablishSPPS-ERG request 2106 can be submitted by SPPS client 2130 via programmatic interface 2177 to request establishment of an extension resource group (SPPS ERG) for an SPPS at a specified premise. After initiating or completing appropriate control plane operations for establishing the SPPS ERG, an SPPS-ERGConfigured message 2103 can be sent to client 2130 in the depicted embodiment. In some embodiments, request 2106 can include an indication of one or more resource groups, such as IVNs that have been established on behalf of the client, thereby enabling the SPPS to determine the number of packet processing-related configuration settings for the resource groups, the number of traffic sources for which fast path packet processing nodes can have to be established, etc. In some embodiments, as mentioned earlier, for example, in the context of Figure 19 In at least one such embodiment, a programmatic request submitted to the VCS control plane to establish a VCS extension resource group (VCS ERG) can also include a parameter indicating that a corresponding SPPS resource group is to be established as well— indeed, an EstablishSPPS-ERG request 2106 can be incorporated in the request for the VCS ERG.

[0140] In various embodiments, after the SPPS ERG has been created, similar to earlier (e.g., in the context of Figure 16 and Figure 17programming interactions described above in the context of the SPPS ERG can be used to establish and use pipelines at the SPPS ERG. For example, a CreateERGBasedAppliance request 2105 indicating an SPPS ERG targeted as a destination can be submitted via interface 2177 in the depicted embodiment to create an instance of a packet appliance, and an ApplianceCreated response 2107 can be communicated after the appliance has been configured using the SPPS ERG. In some embodiments, an ExtendApplianceToERG request 2109 can be submitted to extend functionality of a packet appliance created using data plane resources within a provider network data center to premises outside the provider network at which the SPPS ERG has been established. For example, a program representing rules and / or actions of a pre-existing pipeline instance established within the provider network can be communicated to the external premises in response to the ExtendApplianceToERG request 2109. After the appliance functionality has been extended to the premises, an ApplianceExtended response 2110 can be provided in some embodiments. In at least one embodiment, requests to establish an attachment at the SPPS ERG, activate an appliance at the SPPS ERG, and the like can also be submitted via interface 2177.

[0141] A Modify-SPPS-ERG request 2113 can be submitted in at least some embodiments to change one or more properties of the SPPS ERG, e.g., to increase a maximum amount of traffic to be processed at the SPPS ERG. After the requested change has been applied, e.g., by configuring additional FFPNs at the SPPS ERG, and the like, an SPPS-ERGModified response 2115 can be communicated to client 2130. Custom or SPPS-managed exception paths (similar to the exception paths discussed earlier, e.g., in the context of Figure 15- Figure 18 Custom or SPPS-managed exception paths (similar to the exception paths discussed earlier, e.g., in the context of

[0142] In at least some embodiments, SPPS can collect metrics associated with various SPPS ERGs (e.g., total number of packets processed, rate of packets processed, etc.) separately from metrics of SPPS operations performed using data plane resources in the provider network. To obtain such metrics, a client can submit a GetSPPS-ERG-Metrics request 2124, indicating the specific ERG or location for which the metric is to be provided. In such embodiments, the requested metric can be provided via an SPPS-ERG-MetricSet message 2126. In at least some embodiments, in addition to Figure 21 In addition to the programming interactions shown, it also supports other types of programming interactions related to SPPS extensions.

[0143] Methods for implementing serverless packet processing services

[0144] Figure 22 This is a flowchart illustrating various aspects of the operation of a packet processing pipeline integrated with an isolated virtual network configuration, which can be implemented according to at least some embodiments. As shown in element 2201, it can be functionally integrated with... Figure 1 The serverless packet processing service (SPPS) 120 is similar to a pool of control plane and data plane operation configuration resources for SPPS. For example, the data plane may include a pool of fast path packet processing nodes (FPPNs) that process the majority of packets from various streams, and a pool of slow path packet processing nodes (SPPNs) that can process a subset of packets from one or more streams before a representation of the operations to be performed on the remaining packets of a stream is transmitted for caching at the FPPN. In various embodiments, the FPPN may be configured to efficiently execute programs corresponding to one or more stages of a packet processing pipeline of the kind discussed earlier (each stage having one or more rules and / or actions). At least a portion of the logic of a given stage may be implemented in some embodiments using eBPF programs or functions or other similar programs capable of running in relatively constrained environments (e.g., where the program is limited in terms of maximum instruction count, available memory, etc.).

[0145] A request to establish a pipeline instance (which can also be referred to as an instance of a packet processing appliance) for traffic for a resource group such as an isolated virtual network (IVN) can be received from a networking application vendor or designer via a programmatic interface, for example at a control plane resource of the SPPS (element 2204). A series of data plane resources can be identified for the pipeline instance, including a set of FPPNs of a previously established pool (element 2207). In at least some embodiments, the number of FPPNs can be determined based on configuration information for the resource group, such as the number of compute instances, virtualization hosts, or other sources of traffic for the resource group. In one embodiment, particular FPPNs to be used for the pipeline instance can be selected based on physical proximity to sources and / or destinations of traffic for the pipeline instance.

[0146] In various embodiments, a representation of a pipeline stage (e.g., including an executable program that can be run at an FPPN using an eBPF engine and associated lookup tables) can be deployed / communicated to the selected FPPN (element 2210). Additionally, one or more ingress and / or egress points for the pipeline instance or appliance instance can be established, for example using one or more networking configuration operations such as operations to establish a virtual network interface (VNI) (element 2213). In some embodiments, the ingress and / or egress points can be established in response to one or more attachment requests of the kind discussed earlier. In one embodiment, a programmatic request to activate the instance of the pipeline can also be submitted prior to traffic being directed to the instance.

[0147] A network packet can be received at an ingress point from one of the sources of traffic for the pipeline instance and directed to a particular FPPN (element 2213), for example using a shuffling hashing algorithm in some embodiments. One or more resource group configuration-related packet processing operations can be performed with respect to the received packet in the depicted embodiment, for example operations corresponding to one or more security settings for the group (element 2216). In some embodiments, a separate pipeline can be executed for such resource group configuration-related packet processing, although in other embodiments a single integrated pipeline can be implemented that includes such operations as well as operations specified by the application vendor. One or more rules of one or more pipeline stages can be applied to the received packet (element 2222). Based at least in part on the results of the rules and / or based in part on routing-related configuration settings for the resource group, a routing action can be implemented with respect to the received packet. For example, in the depicted embodiment one or more packets corresponding to the received packet (e.g., rewritten versions of the received packet) can be sent to one or more destinations or targets, or the packet can simply be dropped without transmitting any corresponding packets (element 2225). The operations corresponding to element 2216 can also be performed for subsequent packets.

[0148] Figure 23 is a flow diagram illustrating aspects of operations that can be performed to implement a custom exception path of a packet processing pipeline, in accordance with at least some embodiments. As shown in element 2301, a pool of fast path packet processing nodes (FPPNs) can be configured for control plane and data plane operations of an SPPS similar to the serverless packet processing service (SPPS) 120. In various embodiments, the FPPNs can be configured to efficiently execute programs corresponding to one or more stages (each stage having one or more rules and / or actions) of a packet processing pipeline of the kind discussed earlier. At least a portion of the logic of a given stage can be implemented in some embodiments using eBPF programs or functions or other similar programs capable of running in a relatively restricted environment (e.g., where the programs are limited in the number of instructions, memory that can be used, etc.). Thus, while the FPPNs can be extremely efficient for many kinds of packet processing operations, they can not be suitable for more complex or more resource-intensive packet processing tasks in at least some embodiments. Figure 1

[0149] ​A request to establish a pipeline instance for traffic for a resource group such as an IVN can be received at the SPPS via one or more programmatic interfaces (element 2304). In at least some embodiments, while the request can provide rules and / or actions for the stages of the pipeline (e.g., in the form of eBPF program code), the request can not specify the resources to be used for fast path packet processing at one or more stages of the pipeline; that is, the SPPS can be responsible for assigning FPPNs for the stages. In addition, an indication of a custom exception path (CEP) processing target (e.g., one or more computing devices) to which packets that do not meet the fast path processing criteria for the pipeline are to be directed can also be received via the programmatic interfaces in the depicted embodiment. The CEP target can be responsible for performing one or more operations on received packets in various embodiments, including, for example, operations that cannot be performed at the FPPNs or that the pipeline designer has decided should not be performed at the FPPNs. The CEP target can have fewer restrictions on the kinds of operations that can be performed than the FPPNs— for example, longer and more complex programs can be run using more powerful computing resources than are available at the FPPNs, more programming languages can be supported, more memory can be available, and so on. Indeed, in some embodiments the CEP target can be configured to perform packet processing operations whose usage of the target relative to the FPPNs exceeds a resource usage limit for the FPPNs. In at least some embodiments the CEP target can have access to one or more security artifacts such as cryptographic keys that can be used for some types of packet processing tasks including, for example, terminating secure communication sessions and the like, and such security artifacts can not be accessible from the FPPNs themselves. In at least one embodiment, operations performed on received packets at the CEP target can include inspecting the contents of the body portion of the received packets (in addition to or instead of inspecting the header portion of the packets), whereas operations performed at the FPPNs can only access the header of received messages and can not even read the body portion. In one embodiment, the CEP target can perform encryption / decryption of packets, whereas the FPPNs can not be able to perform such cryptographic operations due to the complexity of the computations involved, and / or because the necessary cryptographic artifacts required for the encryption or decryption operations are not provided to the FPPNs.

[0150] In at least one embodiment, a number of different IVNs can participate in the packet processing application: for example, one or more IVNs can include compute instances that are sources / destinations of traffic to be processed, another IVN can be used for SPPS data plane nodes including FPPNs, and yet another IVN can be used for CEP targets. In some embodiments, CEP targets can be located at premises other than the data center used for SPPS data plane or control plane— for example, an application vendor's data center can be used to host CEP targets. In at least one embodiment, a pool of CEP targets can be indicated to SPPS, and a particular CEP target can be selected at a FPPN from the pool, e.g., based on client-specified criteria or randomly. In various embodiments, a network address (e.g., IP address) can be provided for a CEP target in a request to configure the CEP target. Configuration settings for a resource group, including security rules for inbound and outbound traffic, routing rules, etc., can also be obtained at SPPS using a programmatic interface (e.g., from a control plane of a virtualized compute service).

[0151] In response to the request for pipeline instance, a series of FPPNs can be identified and assigned to perform packet processing resources (element 2307) for traffic associated with the resource group in the depicted embodiment. The number of FPPNs can be determined using techniques similar to those discussed above, and in various embodiments particular FPPNs are selected from a pre-configured pool— for example, the number can be based on a count of traffic sources and / or destinations, and particular SPPNs can be selected based on physical proximity to the sources and / or destinations. Other data plane resources (e.g., slower-path packet processing nodes) can also be assigned to the pipeline instance, and connectivity between CEP targets and SPPS data plane can be established / verified in the depicted embodiment.

[0152] When a packet from one of the sources of traffic is received at a FPPN of the pipeline instance (element 2310), in at least some embodiments one or more resource group configuration-related packet processing operations can be performed first, e.g., to ensure that the packet complies with security settings of the resource group, etc. (element 2313). If the packet does not satisfy the configuration-related requirements of the resource group, in at least some embodiments the packet can be discarded. The configuration-related operations corresponding to element 2310 can be part of the same packet processing pipeline as the operations corresponding to element 2304 in some embodiments.

[0153] A determination can be made in the depicted embodiment as to whether the packet satisfies one or more criteria for fast path packet processing (element 2316). For example, if the cache of the FPPN contains an entry indicating that one or more operations can be performed at the FPPN itself to process the packet, this can indicate that the fast path processing criteria are satisfied in some embodiments. In some embodiments, it can be that the cache does not include an entry, but that an entry can be obtained from another SPPS data plane node, such as a slower path packet processing node of the kind discussed earlier, without requiring the participation of a CEP target. If the fast path criteria are satisfied, whether as a result of a cache hit at the FPPN or as a result of a cache miss satisfied from within the SPPS data plane, packet processing operations can be performed in the depicted embodiment at the FPPN layer (element 2319). Depending at least in part on the results of the packet processing operations, a final routing-related action to end the processing of the received packet can be identified and performed. For example, in some cases one or more packets can be sent toward a destination, while in other cases the received packet can simply be discarded without sending any corresponding packets toward the destination.

[0154] However, if the criteria for fast path are not satisfied (as determined also in the operation corresponding to element 2316), the packet can be transmitted from the SPPS data plane node (e.g., from the FPPN or from a slower path packet processing node) to a CEP target, as indicated in element 2322. In various embodiments, one or more packet processing operations can be performed at the CEP target with respect to the received packet. Optionally, in some embodiments, after the CEP target has processed the packet sent from the SPPS data plane, a representation of packet processing operations that can be performed for subsequent packets of the flow to which the packet belongs can be sent from the CEP target to the SPPS data plane (element 2325). If such a representation is obtained on the SPPS data plane, it can be stored in one or more FPPN caches and used later for other packets of the flow. In various embodiments, the operations corresponding to element 2310 forward can be repeated for other packets received at the FPPN assigned to the pipeline instance.

[0155] In some embodiments, as discussed earlier, for example, in connection with Figure 15 , Figure 17 and Figure 18As indicated above, in at least some embodiments, at least some of the exceptional path resources can be managed by the SPPS— e.g., the SPPS control plane can automatically instantiate additional resources for exceptional paths based on analysis of one or more metrics (e.g., indicating the number of sources of traffic for a pipeline, or the number of packets handled at an existing exceptional path resource during a certain time interval). In contrast, in at least some embodiments, the CEP resources can be managed by the client on behalf of which the corresponding pipeline instance was established— e.g., the client can be responsible for analyzing metrics, scaling (or de-scaling) CEP resources as needed, etc.

[0156] Figure 24 is a flow diagram illustrating aspects of operations that can be performed to implement an extension of a packet processing service outside of a data center of a provider network, in accordance with at least some embodiments. As shown in element 2410, a resource group can be configured on behalf of a provider network client, such as an isolated virtual network (IVN) similar to the IVN discussed earlier. Metadata for the resource group, including at least some network security-related settings, can be stored in a control plane repository within a data center of the provider network. In at least some embodiments, the resource group can be established using resources of a virtualized computing service (VCS), with the metadata stored at a VCS control plane server within the provider network. In at least one embodiment, the data plane of the VCS can be extended to one or more premises outside of the data center of the provider network (e.g., using VCS data plane extension resources similar to the ERG 1935 shown in FIG. 19A). Using such remote resources, the IVN established on behalf of the client can also be extended to these premises, in at least some embodiments, with many compute instances configured at the remote premises on behalf of the client. In other embodiments, such VCS data plane extension resource groups can not necessarily be established. Figure 19

[0157] As shown in element 2404, a set of data plane nodes of a serverless packet processing service (SPPS) similar in functionality to the SPPS 102 can be configured on behalf of the client at a premise outside of the provider network, such as a data center or co-location facility of the client. The SPPS data plane nodes established at the outside premise can include, for example, VCS compute instances similar to the VCS compute instances 1902 shown in FIG. 19A. Figure 2 ​one or more fast path packet processing nodes (FPPNs), slow path packet processing nodes, state information repository nodes, etc. For example, FPPNs can be configured to efficiently apply rules and perform actions corresponding to various stages of a packet processing pipeline similar to the packet processing pipeline introduced earlier. In some embodiments in which a VCS data plane expansion resource group has been established at a premises outside of the provider network, the same premises can be used to host SPPS data plane nodes, effectively establishing an expansion resource group for SPPS that coexists with the VCS expansion resource group. Note that control plane or management operations associated with data plane nodes can continue to be initiated from and orchestrated at the provider network data center in various embodiments— e.g., configuration related commands can be communicated from SPPS control plane resources within the provider network to the external premises via secure communication channels in such embodiments.

[0158] An indication of a particular packet processing pipeline to be implemented with respect to traffic associated with a resource group can be obtained, e.g., at the SPPS control plane via a programmatic interface (element 2407). In at least some embodiments, rules and / or actions for one or more stages of the pipeline can be defined via the programmatic interface (e.g., expressed using eBPF or a similar framework for efficient packet processing).

[0159] A representation of at least a portion of the pipeline can be communicated from the provider network data center to SPPS data plane nodes at the external premises (element 2410). In at least some embodiments, an instance of an appliance corresponding to the pipeline can be established, and a set of FPPNs can be configured for the instance (e.g., with a number of FPPNs selected based on a number of traffic sources and / or destinations). For example, one or more ingress and / or egress points for the pipeline can also be established at the external premises in response to an attachment related request similar to the attachment related requests discussed earlier. In at least one embodiment, a given pipeline can include some data plane nodes within the provider network data center and some data plane nodes at the external premises. The instance of the appliance or pipeline can be activated, e.g., a networking configuration change can be made in various embodiments at the initiation of the SPPS control plane and / or the VCS control plane to cause traffic to be directed to ingress points of the pipeline.

[0160] A network packet can be obtained, received, or detected at a particular data plane node at an external premises (element 2413). Depending on the nature of the application being implemented using the pipeline, the packet can originate from a source of traffic at a provider network or a local source of traffic at an external premises. At least two types of operations can be performed with respect to the obtained packet in the depicted embodiment (element 2416): (a) security checks and / or other operations required by the configuration of the resource group, and (b) operations corresponding to pipeline stages that are independent of the configuration of the resource group.

[0161] Based at least in part on the results of such packet processing, one or more routing-related actions can be performed to complete processing of the obtained packet. For example, in the depicted embodiment, one or more packets corresponding to the processed packet (e.g., a rewritten version of the processed packet) can be transmitted on a path toward a destination, or the packet can be discarded (element 2419). Depending on the particular application being implemented, the destination can be, for example, within a provider network data center, at an external premises where the SPPS data plane node is located, or at some other external premises. If and when the resource group configuration changes or the pipeline is modified, corresponding updates can be transmitted from the provider network data center to the data plane node at the external premises (element 2422). Similarly, as and when changes in metrics such as counts of sources / destinations of traffic for the pipeline are detected, the data plane node can be scaled up or down at the external premises in at least some embodiments.

[0162] It should be noted that, in various embodiments, at least some operations other than those illustrated in the flowcharts of Figure 22 , Figure 23 and Figure 24 may be used to implement the packet processing techniques described above. Some of the operations shown can not be implemented in some embodiments, or can be implemented in a different order, or in parallel rather than sequentially.

[0163] Use Cases

[0164] The above-described techniques for implementing a scalable and customizable serverless packet processing service (SPPS) can be extremely beneficial in a variety of scenarios. As more and more distributed applications migrate to provider network environments, especially environments in which isolated virtual networks (IVNs) are established for different clients, the need for efficient and fault-tolerant management of packet processing operations that also take IVN configuration requirements into account is increasing. By eliminating the need for both the provider and the user of packet processing operations to make requests for packet processing nodes to be allocated, the SPPS can substantially simplify client management workloads. The clean separation of functionality between tiers of fast-path processing nodes and other slower-path processing nodes can enable improved functionality or additional nodes to be pushed out at each tier without impacting work done at another tier. Support for custom exception path processing can allow application providers to design and deploy sophisticated applications that are not limited to making use of provider network resources alone. Extension of the SPPS data plane to premises outside of a provider network data center, while retaining most of the SPPS control plane functionality within the data center, can provide a desirable combination of security and low latency for packet processing. For example, some portions of a given application can be run using the client's proprietary resources at the client's data center's SPPS client, and packets generated at the proprietary resources can be processed with very low latency using SPPS nodes configured at the client's data center before being forwarded to their intended destinations. A wide variety of client-selected packet processing pipelines can be implemented using the described techniques, including, for example, virtual routing applications, intrusion detection operations, etc. For example, by quickly adding packet processing resources as the number of sources of traffic increases, the overall responsiveness of applications that make use of packet processing operations can be improved.

[0165] Illustrative computer system

[0166] In at least some embodiments, servers implementing part or all of one or more techniques described herein, including various components of packet processing services, virtualized computing services, etc., can include general-purpose computer systems that include or are configured to access one or more computer- accessible media. Figure 25 Such a general-purpose computing device 9000 is illustrated in FIG. 9. In the illustrated embodiment, computing device 9000 includes one or more processors 9010 coupled to a system memory 9020 (which can include both a non-volatile memory module and a volatile memory module) via an input / output (I / O) interface 9030. Computing device 9000 also includes a network interface 9040 coupled to I / O interface 9030. In at least some embodiments, computing device 9000 can include a motherboard to which some or all of such components can be attached.

[0167] In various embodiments, computing device 9000 can be a uniprocessor system including one processor 9010, or a multiprocessor system including several processors 9010 (e.g., two, four, eight, or another suitable number). Processors 9010 can be any suitable processors capable of executing instructions. For example, in various embodiments, processors 9010 can be general- purpose or embedded processors implementing any of a variety of instruction set architectures (ISAs), such as the x86, PowerPC, SPARC, or MIPS ISAs, or any other suitable ISA. In multiprocessor systems, each of the processors 9010 can commonly, but not necessarily, implement the same ISA. Graphics processing units (GPUs) can be used in place of or in addition to regular processors in some implementations.

[0168] System memory 9020 can be configured to store instructions and data accessible by processor(s) 9010. In at least some embodiments, system memory 9020 can include both a volatile portion and a non-volatile portion; in other embodiments, only a volatile portion can be used. In various embodiments, the volatile portion of system memory 9020 can be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM, or any other type of memory. For the non-volatile portion of system memory (which can include one or more NVDIMMs, for example), in some embodiments a flash-based memory device can be used, including a not-and (NAND) flash device. In at least some embodiments, the non-volatile portion of system memory can include a power source, such as a supercapacitor or other power storage device (e.g., a battery). In various embodiments, a memristor-based resistive random access memory (ReRAM), three-dimensional NAND technology, ferroelectric RAM, magnetoresistive RAM (MRAM), or any of various types of phase change memory (PCM) can be used at least for the non-volatile portion of system memory. In the illustrated embodiment, program instructions and data implementing one or more desirable functions, such as those methods, techniques, and data described above, are shown stored in system memory 9020 as code 9025 and data 9026.

[0169] In one embodiment, I / O interface 9030 can be configured to coordinate I / O traffic between processor 9010, system memory 9020, and any peripheral devices in the device, including network interface 9040 or other peripheral interfaces such as various types of persistent and / or volatile storage devices. In some embodiments, I / O interface 9030 can perform any necessary protocol, timing or other data transformations to convert data signals from one component (e.g., system memory 9020) into a format suitable for use by another component (e.g., processor 9010). In some embodiments, I / O interface 9030 can include support for devices attached through various types of peripheral buses, such as a variant of the Peripheral Component Interconnect (PCI) bus standard or the Universal Serial Bus (USB) standard, for example. In some embodiments, functionality of I / O interface 9030 can be split into two or more separate components, such as a north bridge and a south bridge, for example. Also, in some embodiments some or all of the functionality of I / O interface 9030 can be

[0170] Network interface 9040 can be configured to allow data to be exchanged between computing device 9000 and other devices 9060 attached to one or more networks 9050, such as Figure 1 to Figure 24 other computer systems or devices illustrated in FIG. 1 1 1. For example, in various embodiments, network interface 9040 can support communication via any suitable wired or wireless general data networks, such as types of Ethernet

[0171] In some embodiments, system memory 9020 can be an embodiment of a computer- accessible medium configured to store data for use by or in connection with an embodiment of Figure 1 to Figure 24The program instructions and data described can be implemented in a same manner as described above for the program instructions and data described above. In some embodiments, the program instructions and / or data can be received, transmitted or stored on a different type of computer- accessible medium. Generally, a computer- accessible medium can include non-transitory storage media or memory media such as volatile or non-volatile storage, removable or non-removable storage, erasable or non-erasable media, or memory that can be implemented in a computer-readable storage medium, such as RAM (e.g., SDRAM, DDR SDRAM, RDRAM, SRAM, etc.), ROM, EEPROM, EPROM, FLASH, etc., which can be located in a computing device 9000, such as system memory 9020, or elsewhere within computing device 9000. A computer- accessible medium can also include transmission media or signals, such as wired or wireless communication media, which can be implemented in a computer-readable storage medium, which is embodied in a computer-program product, which can be executed by one or more processors. Figure 25 Some or all of the computing devices in the plurality of computing devices illustrated can be used to implement the described functionality in various embodiments; for example, software components running on various different devices and servers can cooperate to provide functionality. In some embodiments, in addition to or instead of being implemented using general purpose computer systems, some of the described functionality can be implemented using storage devices, network devices, or special purpose computer systems that are specifically programmed or configured for that purpose. The term "computing device" as used herein refers to at least all of these types of devices, and is not limited to these types of devices.

[0172] Embodiments of the present disclosure can be described in view of the following clauses:

[0173] 1. A system comprising:

[0174] one or more computing devices;

[0175] wherein the one or more computing devices include instructions that, when executed on or across one or more processors, cause the one or more processors to:

[0176] obtain, via one or more programmatic interfaces, a request to establish a packet processing pipeline for network traffic of an isolated virtual network to be virtualized computing service, wherein the packet processing pipeline comprises a plurality of stages, wherein the request does not specify resources to be used to implement at least a portion of the pipeline, and wherein the request indicates, corresponding to a stage of the plurality of stages, at least a rule;

[0177] identify, based at least in part on a number of traffic sources within the isolated virtual network, a plurality of fast-path packet processing nodes for the network traffic;

[0178] deploy, to each of the plurality of fast-path packet processing nodes, a representation of one or more stages of the packet processing pipeline;

[0179] in response to a programmatic request, establish a traffic entry point for the packet processing pipeline; and

[0180] in response to receiving a particular packet at the traffic entry point,

[0181] initiate one or more packet processing operations in accordance with security-related configuration settings of the isolated virtual network, wherein the one or more packet processing operations are not part of the packet processing pipeline;

[0182] apply the rule at the selected fast-path packet processing node;

[0183] implement a routing action with respect to the particular packet, wherein the routing action is based at least in part on (a) a result of the application of the rule and (b) routing configuration settings of the isolated virtual network.

[0184] 2. The system of clause 1, wherein the routing action comprises one of: (a) transmitting at least one packet corresponding to the particular packet to a destination or (b) discarding the particular packet.

[0185] 3. The system of any of clauses 1-2, wherein the one or more computing devices comprise further instructions that, when executed on or across the one or more processors, further cause the one or more processors to:

[0186] in response to configuration of one or more of: (a) an additional traffic source within the isolated virtual network or (b) an additional traffic destination within the isolated virtual network, automatically configure one or more additional fast-path packet processing nodes.

[0187] 4. The system of any of clauses 1-3, wherein the one or more computing devices comprise further instructions that, when executed on or across the one or more processors, further cause the one or more processors to:

[0188] prior to obtaining the request to establish the first packet processing pipeline, pre-configure respective groups of one or more fast-path packet processing nodes at a plurality of racks of a data center; and

[0189] assigning, to a resource of the isolated virtual network, a particular fast path packet processing node from a rack of the plurality of racks, the particular fast path packet processing node satisfying a proximity criterion with respect to the resource.

[0190] 5. The system of any of clauses 1-4, wherein the one or more computing devices include further instructions that, when executed on or across the one or more processors, further cause the one or more processors to:

[0191] store, at a cache of the selected fast path packet processing node, a cache entry comprising the rule; and

[0192] initiate, from the selected fast path packet processing node, a revalidation operation with respect to the cache entry based at least in part on expiration of a timeout associated with the cache entry.

[0193] 6. A method comprising:

[0194] executing, at one or more computing devices:

[0195] obtaining, via one or more programmatic interfaces, a request to establish a packet processing pipeline for network traffic of a resource set of a resource group, wherein the pipeline comprises one or more stages, wherein the request indicates, for at least one stage of the one or more stages, a program for at least performing one or more packet processing operations;

[0196] assigning, based at least in part on a configuration of the resource set, one or more fast path packet processing nodes for the resource set; and

[0197] in response to detecting a particular packet at an entry point of the pipeline,

[0198] initiating, in accordance with the configuration of the resource group, one or more security operations with respect to the particular packet;

[0199] executing the program at a fast path packet processing node of the one or more fast path packet processing nodes; and

[0200] implementing a routing action associated with the particular packet, wherein the routing action is based at least in part on a result of the execution of the program.

[0201] 7. The method of clause 6, wherein the one or more fast path packet processing nodes comprise a plurality of fast path packet processing nodes, wherein the resource group comprises a plurality of compute instances of a virtualized computing service, including a first compute instance and a second compute instance, the method further comprising executing, at the one or more computing devices:

[0202] assigning a first subset of the plurality of fast path packet processing nodes to perform one or more operations of one or more packet processing pipelines with respect to packets originating from the first computing instance; and

[0203] assigning a second subset of the plurality of fast path packet processing nodes to perform one or more operations of one or more packet processing pipelines with respect to packets originating from the second computing instance, wherein the second subset includes at least one node that is not in the first subset.

[0204] 8. The method of any of clauses 6-7, further comprising performing, at the one or more computing devices:

[0205] in response to a request received via a programmatic interface, providing an indication of one or more entry points of the packet processing pipelines.

[0206] 9. The method of clause 8, wherein the one or more entry points include one or more virtual network interfaces.

[0207] 10. The method of any of clauses 6-8, wherein the one or more fast path packet processing nodes include a plurality of fast path packet processing nodes, the method further comprising performing, at the one or more computing devices:

[0208] assigning a first subset of the plurality of fast path packet processing nodes to perform one or more operations of one or more packet processing pipelines with respect to packets originating from one or more resources of the resource group; and

[0209] assigning a second subset of the plurality of fast path packet processing nodes to perform one or more operations of one or more packet processing pipelines with respect to packets directed to one or more resources of the resource group, wherein the second subset includes at least one fast path packet processing node that is not in the first subset.

[0210] 11. The method of any of clauses 6-8 or 10, wherein a first fast path packet processing node comprises one of: (a) a server comprising one or more hardware cards attached via a peripheral interconnect, wherein at least a portion of the program is executed at the one or more hardware cards, or (b) a compute instance of a virtualized computing service.

[0211] 12. The method of any of clauses 6-8 or 10-11, further comprising performing, at the one or more computing devices:

[0212] configuring a repository for state information associated with one or more network packet flows for which packet processing is to be performed; and

[0213] obtaining state information from the repository to implement at least one stage of a packet processing pipeline.

[0214] 13. The method of any of clauses 6-8 or 10-12, wherein the program comprises one or more eBPF (extended Berkeley Packet Filter) operations.

[0215] 14. The method of any of clauses 6-8 or 10-13, further comprising performing, at the one or more computing devices:

[0216] receiving an indication of a rule associated with a particular stage of the packet processing pipeline; and

[0217] applying the rule to determine (a) that at least a portion of a program associated with the particular stage is to be executed with respect to the particular packet and (b) that at least a portion of the program is not to be executed with respect to another packet.

[0218] 15. The method of any of clauses 6-8 or 10-14, wherein the program implements at least a portion of one or more of: (a) a custom routing algorithm, (b) a network address translation algorithm, (c) a multicast algorithm, (d) a source substitution algorithm, (e) an algorithm for transferring packets from one isolated virtual network to another isolated virtual network of a virtualized computing service, (f) an algorithm for securely transferring data to or from an isolated private network, (g) an intrusion detection algorithm, (h) a bandwidth usage tracking algorithm, (i) a load balancing algorithm, (j) an algorithm for securely transferring data between a provider network data center and a premises outside of the provider network, (k) a data exfiltration detection algorithm, or (1) a proprietary algorithm that implements business logic of an application.

[0219] 16. One or more non-transitory computer-accessible storage media storing program instructions that, when executed on or across one or more processors, cause the one or more processors to:

[0220] determine that one or more packet processing operations are to be performed for a set of network traffic to a set of resources of a resource group, wherein the resource group has one or more security settings, wherein at least one of the one or more packet processing operations comprises executing a program indicated via a programmatic interface;

[0221] transmitting the program to individual ones of one or more fast path packet processing nodes, wherein the one or more fast path packet processing nodes are assigned to the resource set based at least in part on metadata of the resource group; and

[0222] responsive to detecting the particular packet at the one or more fast path packet processing nodes, causing,

[0223] initiating one or more security operations in accordance with the security settings;

[0224] executing the program at a fast path packet processing node of the one or more fast path packet processing nodes; and

[0225] implementing a routing operation with respect to the particular packet based at least in part on a result of the execution of the program.

[0226] 17. The one or more non-transitory computer-accessible storage media of clause 16, the media storing further program instructions that, when executed on or across the one or more processors, further cause the one or more processors to:

[0227] determine that a number of traffic sources in the resource set has changed; and

[0228] modify a number of fast path packet processing resources assigned to the resource set based at least in part on the change.

[0229] 18. The one or more non-transitory computer-accessible storage media of any of clauses 16-17, wherein the one or more fast path packet processing nodes comprise a plurality of fast path packet processing nodes, wherein the resource group comprises a plurality of compute instances of a virtualized computing service, including a first compute instance and a second compute instance, and wherein the one or more non-transitory computer-accessible storage media store further program instructions that, when executed on or across the one or more processors, further cause the one or more processors to:

[0230] assign a first subset of the plurality of fast path packet processing nodes to perform the one or more packet processing operations with respect to packets associated with the first compute instance; and

[0231] assign a second subset of the plurality of fast path packet processing nodes to perform the one or more packet processing operations with respect to packets associated with the second compute instance.

[0232] 19. The one or more non-transitory computer-accessible storage media of any of clauses 16-18, the media storing further program instructions that, when executed on or across the one or more processors, further cause the one or more processors to:

[0233] establish a repository at which to store state information related to one or more packet processing pipelines employed for one or more packet flows; and

[0234] subsequent to assigning at least a portion of a workload of a particular fast path packet processing node to another fast path packet processing node, cause state information to be obtained at the particular fast path packet processing node from the repository.

[0235] 20. The one or more non-transitory computer-accessible storage media of any of clauses 16-19, the media storing further program instructions that, when executed on or across the one or more processors, further cause the one or more processors to:

[0236] establish a plurality of independently updated cells of fast path packet processing nodes, including a first cell and a second cell, the first cell comprising a first fast path packet processing node of the one or more fast path packet processing nodes, the second cell comprising a second fast path packet processing node of the one or more fast path packet processing nodes;

[0237] propagate a software update to fast path packet processing nodes of the first cell; and

[0238] verify that the first cell meets stability criteria after the software update has been applied at the first cell prior to propagating the software update to fast path packet processing nodes of the second cell.

[0239] 21. A system comprising:

[0240] one or more computing devices;

[0241] wherein the one or more computing devices include instructions that, when executed on or across one or more processors, cause the one or more processors to:

[0242] store, at one or more data centers of a provider network, metadata associated with an isolated virtual network established on behalf of a client of the provider network, wherein the metadata comprises one or more security settings for network traffic;

[0243] at a premises outside the provider network, on behalf of the client, configure a set of packet processing nodes;

[0244] obtain an indication of a packet processing pipeline to implement for a subset of packets associated with at least the isolated virtual network;

[0245] transmit, from a data center of the one or more data centers to the premises outside the provider network, a representation of at least a portion of the packet processing pipeline;

[0246] in response to detecting a particular packet at the premises outside the provider network,

[0247] perform, with respect to the particular packet, a security operation in accordance with the one or more security settings; and

[0248] perform, with respect to the particular packet, one or more packet processing operations of the packet processing pipeline at the set of packet processing nodes.

[0249] 22. The system of clause 21, wherein the one or more computing devices include further instructions that, when executed on or across the one or more processors, further cause the one or more processors to:

[0250] detect establishment of one or more of: (a) an additional source of traffic at the premises or (b) an additional destination of traffic at the premises; and

[0251] in response to detecting the establishment, automatically configure one or more additional packet processing nodes at the premises.

[0252] 23. The system of any of clauses 21-22, wherein the one or more computing devices include further instructions that, when executed on or across the one or more processors, further cause the one or more processors to:

[0253] assign a first subset of the set of packet processing nodes to perform one or more operations of one or more packet processing pipelines with respect to packets originating from a first computing instance of a plurality of computing instances launched at the premises; and

[0254] assign a second subset of the set of packet processing nodes to perform one or more operations of the one or more packet processing pipelines with respect to packets originating from a second computing instance of the plurality of computing instances, wherein the second subset includes at least one node that is not in the first subset.

[0255] 24. The system of any of clauses 21-23, wherein the premises external to the provider network comprises one of: (a) a data center of the client, or (b) a co-location facility comprising resources owned by one or more entities other than an operator of the provider network.

[0256] 25. The system of any of clauses 21-24, wherein the set of packet processing nodes comprises one of: (a) a server comprising one or more hardware cards attached via a peripheral interconnect, wherein at least a portion of the packet processing pipeline is executed at the one or more hardware cards, or (b) a compute instance of a virtualized computing service.

[0257] 26. A method comprising:

[0258] performing, at one or more computing devices:

[0259] storing, at a provider network, a representation of a packet processing pipeline obtained from a client of the provider network, wherein the packet processing pipeline comprises one or more stages at which respective sets of packet processing operations are performed;

[0260] configuring, on behalf of the client, one or more packet processing nodes at a premises external to the provider network;

[0261] transmitting, from the provider network to the premises, the representation of the packet processing pipeline; and

[0262] causing, at the one or more packet processing nodes, one or more packet processing operations of the packet processing pipeline to be performed in response to detection of a particular packet at the premises external to the provider network.

[0263] 27. The method of clause 26, wherein the premises external to the provider network comprises one of: (a) a data center of the client, or (b) a co-location facility comprising resources owned by one or more entities other than an operator of the provider network.

[0264] 28. The method of any of clauses 26-27, wherein the one or more packet processing nodes comprise a plurality of packet processing nodes, the method further comprising:

[0265] causing, by a control plane component of a virtualized computing service of the provider network, a plurality of compute instances to be launched at the premises external to the provider network, wherein the plurality of compute instances comprises a first compute instance and a second compute instance;

[0266] assigning a first subset of the plurality of packet processing nodes to perform one or more operations of one or more packet processing pipelines with respect to packets originating from the first compute instance; and

[0267] assigning a second subset of the plurality of packet processing nodes to perform one or more operations of one or more packet processing pipelines with respect to packets originating from the second compute instance, wherein the second subset includes at least one node that is not in the first subset.

[0268] 29. The method of any of clauses 26-28, further comprising performing, at the one or more computing devices:

[0269] configuring an entry point of the packet processing pipeline in response to a request received via a programmatic interface, wherein the particular packet is directed to the entry point.

[0270] 30. The method of clause 29, wherein the entry point comprises one or more virtual network interfaces.

[0271] 31. The method of any of clauses 26-29, wherein a first packet processing node of the one or more packet processing nodes comprises one of: (a) a server comprising one or more hardware cards attached via a peripheral interconnect, wherein at least a portion of the packet processing pipeline is performed at the one or more hardware cards, or (b) a compute instance of a virtualized computing service.

[0272] 32. The method of any of clauses 26-29 or 31, further comprising performing, at the one or more computing devices:

[0273] configuring a repository for state information associated with one or more network packet flows for which packet processing is to be performed; and

[0274] obtaining, at a packet processing node at the premises outside the provider network, state information from the repository to implement at least one stage of a packet processing pipeline.

[0275] 33. The method of any of clauses 26-29 or 31-32, further comprising performing, at the one or more computing devices:

[0276] receiving an indication of an exception path target associated with the packet processing pipeline; and

[0277] in response to determining, at the one or more packet processing nodes at the premises outside the provider network, that another packet does not satisfy fast path criteria of the packet processing pipeline, causing the another packet to be processed at the exception path target.

[0278] 34. The method of any of clauses 26-29 or 31-33, further comprising performing, at the one or more computing devices:

[0279] detecting establishment of one or more of: (a) an additional traffic source at the premises or (b) an additional traffic destination at the premises; and

[0280] automatically configuring one or more additional packet processing nodes at the premises in response to detecting the establishment.

[0281] 35. The method of any of clauses 26-29 or 31-34, wherein the one or more packet processing operations performed at a packet processing node at the premises outside the provider network comprise at least a portion of one or more of: (a) a custom routing algorithm, (b) a network address translation algorithm, (c) a multicast algorithm, (d) a source substitution algorithm, (e) an algorithm for transferring packets from one isolated virtual network to another isolated virtual network of a virtualized computing service, (f) an algorithm for securely transferring data to or from an isolated private network, (g) an intrusion detection algorithm, (h) a bandwidth usage tracking algorithm, (i) a load balancing algorithm, (j) an algorithm for securely transferring data between a provider network data center and a premises outside the provider network, (k) a data exfiltration detection algorithm, (1) a secure session termination protocol, or (m) a proprietary algorithm implementing business logic of an application.

[0282] 36. One or more non-transitory computer-accessible storage media storing program instructions that, when executed on or across one or more processors, cause the one or more processors to:

[0283] store, at a provider network, a representation of one or more packet processing operations obtained from a client of the provider network;

[0284] configure, at a premises outside the provider network, a set of packet processing nodes on behalf of the client;

[0285] transfer, from the provider network to the premises, the representation of the one or more packet processing operations; and

[0286] at the set of packet processing nodes, cause the one or more packet processing operations to be performed in response to receiving a particular packet at the premises outside the provider network.

[0287] 37. The one or more non-transitory computer-accessible storage media of clause 36, wherein the premises outside the provider network comprises one of: (a) a data center of the client, or (b) a co-location facility comprising resources owned by one or more entities other than an operator of the provider network.

[0288] 38. The one or more non-transitory computer-accessible storage media of any of clauses 36-37, wherein the one or more packet processing nodes comprise a plurality of packet processing nodes, wherein the one or more non-transitory computer-accessible storage media store further program instructions that, when executed on or across the one or more processors, further cause the one or more processors to:

[0289] assign a first subset of the plurality of packet processing nodes to perform one or more operations of one or more packet processing pipelines with respect to packets originating from a first computing instance of a plurality of computing instances running at the premises outside the provider network; and

[0290] assign a second subset of the plurality of packet processing nodes to perform one or more operations of one or more packet processing pipelines with respect to packets originating from a second computing instance of the plurality of computing instances, wherein the second subset comprises at least one node that is not in the first subset.

[0291] 39. The one or more non-transitory computer-accessible storage media of clause 36, wherein a first packet processing node of the set of packet processing nodes comprises one of: (a) a server comprising one or more hardware cards attached via a peripheral interconnect, wherein at least one of the one or more packet processing operations is performed at the one or more hardware cards, or (b) a computing instance of a virtualized computing service.

[0292] 40. The one or more non-transitory computer-accessible storage media of clause 36, wherein the one or more non-transitory computer-accessible storage media store further program instructions that, when executed on or across the one or more processors, further cause the one or more processors to:

[0293] obtain an indication of an exception path target associated with the one or more packet processing tasks; and

[0294] provide the indication of the exception path target to the set of packet processing nodes at the premises outside the provider network, wherein the set of packet processing nodes comprises at least one node configured to transmit packets that do not satisfy fast path criteria to the exception path target.

[0295] 41. A system comprising:

[0296] one or more computing devices;

[0297] wherein the one or more computing devices include instructions that, when executed on or across one or more processors, cause the one or more processors to:

[0298] obtain, via one or more programmatic interfaces, (a) a request for a packet processing pipeline to be implemented for a subset of packets associated with a first isolated virtual network configured for at least a first client that represents a network-accessible service, wherein the request does not indicate fast-path packet processing nodes to be used for the pipeline, (b) an indication of an exception path target configured to perform at least one operation for packets that do not satisfy criteria for fast-path processing at the packet processing pipeline, and (c) one or more security requirements for the first isolated virtual network;

[0299] identify a set of fast-path packet processing nodes for implementing at least a portion of the packet processing pipeline;

[0300] in response to (a) receiving a first packet at the set of fast-path packet processing nodes and (b) determining that the first packet satisfies the one or more security requirements and the criteria for fast-path processing, performing one or more packet processing operations of the packet processing pipeline at one or more nodes of the set; and

[0301] in response to (a) receiving a second packet at the set and (b) determining that the second packet satisfies the one or more security requirements and does not satisfy the criteria for fast-path processing,

[0302] communicating the second packet from a particular fast-path packet processing node of the set to the exception path target.

[0303] 42. The system of clause 41, wherein the first isolated virtual network is configured at one or more data centers, wherein the set of fast-path processing nodes are configured within a second isolated virtual network, and wherein the exception path target comprises one or more of: (a) a computing device in a third isolated virtual network, or (b) a computing device at a premises that is external to the one or more data centers.

[0304] 43. The system of any of clauses 41-42, wherein the second packet belongs to a particular packet flow, and wherein the one or more computing devices include further instructions that, when executed on or across the one or more processors, further cause the one or more processors to:

[0305] obtain, at a particular fast path packet processing node subsequent to transmission of the second packet, an indication of one or more additional packet processing operations to perform for subsequent packets of the particular packet flow;

[0306] cache, at the particular fast path packet processing node, executable versions of the one or more additional packet processing operations; and

[0307] perform, at the particular fast path packet processing node, the one or more additional packet processing operations on one or more subsequent packets of the particular packet flow using the executable versions.

[0308] 44. The system of any of clauses 41-43, wherein the set of fast path packet processing nodes have a resource usage limit, and wherein the exception path target is configured to perform packet processing operations whose resource usage exceeds the resource usage limit.

[0309] 45. The system of any of clauses 41-44, wherein the exception path target is configured to perform one or more of: (a) a decryption operation on a packet or (b) an encryption operation on the packet.

[0310] 46. A method comprising:

[0311] performing, at one or more computing devices:

[0312] obtaining (a) an indication of an exception path target of a packet processing pipeline, wherein the packet processing pipeline is to be implemented for a subset of packets associated with at least a resource group, and (b) one or more configuration settings of the resource group;

[0313] in response to (a) receiving a first packet at a set of fast path packet processing nodes configured for the packet processing pipeline and (b) determining that the first packet satisfies one or more requirements of the configuration settings and criteria for fast path processing, performing one or more packet processing operations of the packet processing pipeline at one or more nodes of the set; and

[0314] in response to (a) receiving a second packet at the set of fast path packet processing nodes and (b) determining that the second packet does not satisfy the criteria for fast path processing, transmitting the second packet to the exception path target.

[0315] 47. The method of clause 46, wherein the second packet belongs to a particular packet flow, the method further comprising performing, at the one or more computing devices:

[0316] subsequent to the transmitting the second packet, obtaining, at the particular fast path packet processing node from the exception path target, an indication of one or more additional packet processing operations to perform for subsequent packets of the particular packet flow;

[0317] caching, at the particular fast path packet processing node, executable versions of the one or more additional packet processing operations; and

[0318] using the executable versions to perform, at the particular fast path packet processing node, the one or more additional packet processing operations on one or more subsequent packets of the particular packet flow.

[0319] 48. The method of any of clauses 46-47, wherein the set of packet processing nodes has a resource usage limit, and wherein the exception path target is configured to perform packet processing operations that exceed the resource usage limit.

[0320] 49. The method of clause 48, wherein the resource usage limit comprises one or more of: (a) a memory limit, (b) a computing resource limit, or (c) a program instruction count limit.

[0321] 50. The method of any of clauses 46-48, further comprising performing, by the one or more computing devices:

[0322] accessing, by the exception path target, a security artifact to perform one or more operations associated with the second packet, wherein the security artifact is not accessible at the one or more fast path packet processing nodes.

[0323] 51. The method of any of clauses 46-48 or 50, further comprising performing, at the one or more computing devices:

[0324] inspecting, at the exception path target, content of a body of the second packet, wherein packet processing operations performed at the one or more fast path packet processing nodes (a) include reading content of one or more headers of a network packet and (b) do not include reading content of a body of a network packet.

[0325] 52. The method of any one of clauses 46-48 or 50-51, further comprising performing, at the one or more computing devices:

[0326] instantiating, by a control plane component of the packet processing service, an additional exception path target resource automatically based on analysis of one or more metrics.

[0327] 53. The method of any one of clauses 46-48 or 50-52, wherein determining that the second packet does not satisfy the criteria for fast path processing comprises determining that a cache stored at the one or more fast path packet processing nodes does not satisfy an entry comprising executable program instructions indicating that packet processing operations are to be implemented to perform packet processing operations.

[0328] 54. The method of any one of clauses 46-48 or 50-53, further comprising performing, at the one or more computing devices:

[0329] obtaining, via a programmatic interface, program code representing a packet processing operation of the one or more packet processing operations performed at a fast path processing node with respect to the first packet.

[0330] 55. The method of any one of clauses 46-48 or 50-54, further comprising performing, at the one or more computing devices:

[0331] causing, at the exception path target, at least a portion of one or more of: (a) a custom routing algorithm, (b) a network address translation algorithm, (c) a multicast algorithm, (d) a source substitution algorithm, (e) an algorithm for transferring packets from one isolated virtual network to another isolated virtual network of a virtualized computing service, (f) an algorithm for securely transferring data to or from an isolated private network, (g) an intrusion detection algorithm, (h) a bandwidth usage tracking algorithm, (i) a load balancing algorithm, (j) an algorithm for securely transferring data between a provider network data center and a premises outside of the provider network, (k) a data exfiltration detection algorithm, (1) a secure session termination protocol, or (1) a proprietary algorithm implementing business logic of an application.

[0332] 56. One or more non-transitory computer-accessible storage media storing program instructions that, when executed on or across one or more processors, cause the one or more processors to:

[0333] obtain (a) an indication of one or more packet processing operations to be implemented for a subset of packets associated with at least a resource group and (b) one or more configuration settings of the resource group;

[0334] performing the one or more packet processing operations at one or more nodes of the set in response to (a) receiving a first packet at a set of fast path packet processing nodes configured for the resource group and (b) determining that the first packet satisfies one or more requirements indicated in the one or more configuration settings and (c) determining that the first packet satisfies criteria for fast path processing; and

[0335] in response to (a) receiving a second packet at the set of fast path packet processing nodes and (b) determining that the second packet does not satisfy the criteria for fast path processing, transmitting the second packet to an exception path target.

[0336] 57. The one or more non-transitory computer-accessible storage media of clause 56, the media storing further program instructions that, when executed on or across the one or more processors, further cause the one or more processors to:

[0337] subsequent to transmitting the second packet, obtaining at the particular fast path packet processing node from the exception path target an indication of one or more additional packet processing operations to perform for subsequent packets of a particular packet flow to which the second packet belongs;

[0338] caching at the particular fast path packet processing node executable versions of the one or more additional packet processing operations; and

[0339] using the executable versions to perform the one or more additional packet processing operations at the particular fast path packet processing node for one or more subsequent packets of the particular packet flow.

[0340] 58. The one or more non-transitory computer-accessible storage media of any of clauses 56-57, wherein the set of fast path packet processing nodes has a resource usage limit, and wherein the exception path target is configured to perform packet processing operations that exceed the resource usage limit.

[0341] 59. The one or more non-transitory computer-accessible storage media of any of clauses 56-58, wherein the exception path target is configured to access a security artifact for processing associated with the second packet, wherein the security artifact is not accessible at the one or more fast path packet processing nodes.

[0342] 60. The one or more non-transitory computer-accessible storage media of any of clauses 56-59, the media storing further program instructions that, when executed on or across the one or more processors, further cause the one or more processors to:

[0343] inspecting, at the exception path target, contents of a body of the second packet, wherein the packet processing operations performed at the one or more fast path packet processing nodes (a) include reading contents of one or more headers of a network packet and (b) do not include reading contents of a body of a network packet.

[0344] CONCLUSION

[0345] Various embodiments can further include receiving, sending or storing instructions and / or data implemented in accordance with the foregoing description upon a computer- accessible medium. Generally, a computer-accessible medium can include storage media or memory media such as magnetic or optical media, e.g., disk or DVD / CD-ROM, volatile or non-volatile media such as RAM (e.g. SDRAM, DDR, RDRAM, SRAM, etc.), ROM, etc. as well as transmission media or signals such as electrical, electromagnetic, or digital signals, conveyed via a communication medium such as network and / or a wireless link.

[0346] The various methods as illustrated in the figures and described herein represent exemplary embodiments of methods. The methods can be implemented in software, hardware, or a combination thereof. The order of method can be changed, and various elements can be added, reordered, combined, omitted, modified, etc.

[0347] It will be apparent to those skilled in the art that various modifications and variations can be made. It is intended that all such modifications and variations be included within the scope of the disclosure and thus are intended to be covered by the above description and claims.

Claims

1. A system for providing virtualized packet processing services, comprising: one or more computing devices; wherein the one or more computing devices include instructions that, when executed on or across one or more processors, cause the one or more computing devices to: obtain, via one or more programmatic interfaces, (a) a request for a packet processing pipeline to be performed on at least a subset of packets associated with a first isolated virtual network configured for a first client representative of a network-accessible service, wherein the request does not indicate a fast-path packet processing node to be used for the pipeline, (b) an indication of an exception path target configured to perform at least one operation on a packet that does not satisfy criteria for fast-path processing at the packet processing pipeline, and (c) one or more security requirements for the first isolated virtual network; identify a set of fast-path packet processing nodes for implementing at least a portion of the packet processing pipeline; in response to (a) receiving a first packet at the set of fast-path packet processing nodes and (b) determining that the first packet satisfies the one or more security requirements and the criteria for fast-path processing, perform one or more packet processing operations of the packet processing pipeline at one or more nodes of the set; and in response to (a) receiving a second packet at the set and (b) determining that the second packet satisfies the one or more security requirements and does not satisfy the criteria for fast-path processing, transmit the second packet from a particular fast-path packet processing node of the set to the exception path target.

2. The system of claim 1, wherein the first isolated virtual network is configured at one or more data centers, wherein the set of fast-path processing nodes are configured within a second isolated virtual network, and wherein the target comprises one or more of: (a) a computing device in a third isolated virtual network, or (b) a computing device at a premises located outside of the one or more data centers.

3. The system of claim 1, wherein the second packet belongs to a particular packet flow, and wherein the one or more computing devices include further instructions that, when executed on or across one or more processors, further cause the one or more computing devices to: obtain, at a particular fast-path packet processing node following transmission of the second packet, an indication of one or more additional packet processing operations to be performed on subsequent packets of the particular packet flow; cache, at the particular fast-path packet processing node, executable versions of the one or more additional packet processing operations; and perform, at the particular fast-path packet processing node, the one or more additional packet processing operations on one or more subsequent packets of the particular packet flow using the executable versions.

4. The system of claim 1, wherein the set of fast-path packet processing nodes have a resource usage limit, and wherein the exception path target is configured to perform packet processing operations that exceed the resource usage limit.

5. The system of claim 1, wherein the exception path target is configured to perform one or more of: (a) a decryption operation on a packet or (b) an encryption operation on the packet.

6. A method for providing virtualized packet processing services, comprising: performing, at one or more computing devices: obtaining (a) an indication of an exception path target of a packet processing pipeline, wherein the packet processing pipeline is to be performed for at least a subset of packets associated with a resource group, and (b) one or more configuration settings of the resource group; in response to (a) receiving a first packet at a set of fast-path packet processing nodes configured for the packet processing pipeline and (b) determining that the first packet satisfies one or more requirements of the configuration settings and criteria for fast-path processing, performing one or more packet processing operations of the packet processing pipeline at one or more nodes of the set; and in response to (a) receiving a second packet at the set of fast-path packet processing nodes and (b) determining that the second packet does not satisfy the criteria for fast-path processing, transmitting the second packet to the exception path target.

7. The method of claim 6, wherein the second packet belongs to a particular packet flow, the method further comprising performing, at the one or more computing devices: subsequent to the transmitting the second packet, obtaining, at a particular fast-path packet processing node from the exception path target, an indication of one or more additional packet processing operations to be performed for subsequent packets of the particular packet flow; caching, at the particular fast-path packet processing node, executable versions of the one or more additional packet processing operations; and performing, at the particular fast-path packet processing node, the one or more additional packet processing operations on one or more subsequent packets of the particular packet flow using the executable versions.

8. The method of claim 6, wherein the set of packet processing nodes have a resource usage limit, and wherein the exception path target is configured to perform packet processing operations that exceed the resource usage limit.

9. The method of claim 8, wherein the resource usage limit comprises one or more of: (a) a memory limit, (b) a computational resource limit, or (c) a program instruction count limit.

10. The method of claim 6, further comprising performing, by the one or more computing devices: accessing, by the exception path target, a security artifact to perform one or more operations associated with the second packet, wherein the security artifact is not accessible at the one or more fast-path packet processing nodes.

11. The method of claim 6, further comprising performing, at the one or more computing devices: inspecting, at the exception path target, content of a body of the second packet, wherein the packet processing operations performed at the one or more fast path packet processing nodes (a) include reading content of one or more headers of a network packet and (b) do not include reading content of a body of a network packet.

12. The method of claim 6, further comprising performing, at the one or more computing devices: automatically instantiating, by a control plane component of the packet processing service, additional exception path target resources based on analysis of one or more metrics.

13. The method of claim 6, wherein determining that the second packet does not satisfy criteria for fast path processing comprises determining that a cache stored at the one or more fast path packet processing nodes does not include an entry indicating an executable program to be implemented to perform packet processing operations.

14. The method of claim 6, further comprising performing, at the one or more computing devices: obtaining, via a programmatic interface, program code representing a packet processing operation of the one or more packet processing operations performed at a fast path processing node for the first packet.

15. The method of claim 6, further comprising performing, at the one or more computing devices: at the exception path target, causing performance of at least a portion of one or more of: (a) a custom routing algorithm, (b) a network address translation algorithm, (c) a multicast algorithm, (d) a source substitution algorithm, (e) an algorithm for transferring packets from one isolated virtual network to another isolated virtual network of a virtualized computing service, (f) an algorithm for securely transferring data to or from a virtual private network, (g) an intrusion detection algorithm, (h) a bandwidth usage tracking algorithm, (i) a load balancing algorithm, (j) an algorithm for securely transferring data between a provider network data center and a premises outside of the provider network, (k) a data exfiltration detection algorithm, (1) a secure session termination protocol, or (1) a proprietary algorithm implementing business logic of an application.

16. One or more non-transitory computer-accessible storage media storing program instructions that when executed on or across one or more processors cause one or more computer systems to: obtain (a) an indication of one or more packet processing operations to be performed for at least a subset of packets associated with a resource group and (b) one or more configuration settings of the resource group; in response to (a) receiving a first packet at a set of fast path packet processing nodes configured for the resource group, (b) determining that the first packet satisfies one or more requirements indicated in the one or more configuration settings and (c) determining that the first packet satisfies criteria for fast path processing, perform the one or more packet processing operations at one or more nodes of the set; and in response to (a) receiving a second packet at the set of fast-path packet processing nodes and (b) determining that the second packet does not satisfy the criteria for fast-path processing, transmitting the second packet to an exception path target.

17. The one or more non-transitory computer-accessible storage media of claim 16, the media storing further program instructions that, when executed on or across the one or more processors, further cause one or more computer systems to: subsequent to transmitting the second packet, obtaining, at a particular fast-path packet processing node, from the exception path target, an indication of one or more additional packet processing operations to perform for subsequent packets of a particular packet flow to which the second packet belongs; cache, at the particular fast-path packet processing node, executable versions of the one or more additional packet processing operations; and perform, at the particular fast-path packet processing node, the one or more additional packet processing operations for one or more subsequent packets of the particular packet flow using the executable versions.

18. The one or more non-transitory computer-accessible storage media of claim 16, wherein the set of fast-path packet processing nodes has a resource usage limit, and wherein the exception path target is configured to perform packet processing operations that exceed the resource usage limit.

19. The one or more non-transitory computer-accessible storage media of claim 16, wherein the exception path target is configured to access a security artifact for processing associated with the second packet, wherein the security artifact is not accessible at the one or more fast-path packet processing nodes.

20. The one or more non-transitory computer-accessible storage media of claim 16, the media storing further program instructions that, when executed on or across the one or more processors, further cause one or more computer systems to: inspect, at the exception path target, content of a body of the second packet, wherein packet processing operations performed at the one or more fast-path packet processing nodes (a) include reading content of one or more headers of a network packet and (b) do not include reading content of a body of a network packet.

Images