Efficient state maintenance of execution environment in on-demand code execution system

Abstract

Systems and methods are described for providing maintenance of state information during processing of a data set by executing code on an on-demand code execution system. Rather than requiring the execution environment of such a system to maintain state, an intermediary device is disclosed that retrieves calls to the system from a call queue and iteratively submits the calls to the system. Each call within the queue corresponds to a data item of the data set to be analyzed. Upon submitting a call to the system, the intermediary device submits state information within the call that reflects the state of processing the data set. The response to the call includes state information that is updated based on the processing of the data item within the call. Thus, state information is maintained for use in processing the data set without the need to retain state information within a separate execution environment.

Description

Background Art

[0001] Computing devices may utilize communication networks to exchange data. Companies and organizations operate computer networks that interconnect many computing devices to support operations or provide services to third parties. Computing systems may be located in a single geographic location or in multiple different geographic locations (e.g., interconnected by private or public communication networks). Specifically, a data center or data processing center (generally referred to herein as a "data center") may include many interconnected computing systems to provide computing resources to users of the data center. A data center may be a private data center operated on behalf of an organization, or a public data center operated on behalf of the public or for the public interest.

[0002] To facilitate improved utilization of data center resources, virtualization technology allows a single physical computing device to host one or more virtual machine instances that appear and operate as independent computing devices to users of the data center. Through virtualization, a single physical computing device can dynamically create, maintain, delete, or otherwise manage virtual machines. In turn, users can request computer resources from the data center, including configurations of a single computing device or networked computing devices, and be provided with different quantities of virtual machine resources.

[0003] In some cases, virtual machine instances may be configured according to a variety of virtual machine instance types to provide specific functionality. For example, various computing devices may be associated with different combinations of operating systems or operating system configurations, virtualized hardware resources, and software applications to enable the computing devices to provide different desired functionality, or to provide similar functionality more efficiently. These virtual machine instance type configurations are typically contained within a device image, which includes static data that includes the software that the virtual machine will run once it is started (e.g., the OS and applications and their configuration and data files, etc.). The device image is typically stored on a disk used to create or initialize the instance. Therefore, the computing device can process the device image in order to achieve the desired software configuration.

[0004] An exemplary use of a data center is to process or analyze large data sets that may be impractical to analyze using a single computing device. Various technologies have been developed to allow multiple computing devices (or in some cases, multiple processors within a single computing device) to process data simultaneously. This concurrent data processing is sometimes referred to as "parallelization". A technology for allowing parallelization in processing data sets is the "MapReduce" programming model. This programming model usually requires a centralized "infrastructure" or "framework" that controls the execution of two functions by separate computing devices within a group of devices. The execution of the first function (i.e., the "mapping" function) causes multiple devices to process parts (or "chunks") of a full set of raw data to generate a set of intermediate results, such as the counts of individual words in a text corpus. The execution of the second function (i.e., the "reduction" function) causes one or more devices to combine multiple sets of intermediate results (from multiple mapping functions) to produce a set of aggregated results. The reduction function can be executed multiple times, with each execution further reducing the number of aggregated results until a single aggregated result record is created. In traditional implementations of the map-reduce programming model, an infrastructure or framework must typically execute serially to coordinate the execution of the map and reduce functions until a result is provided. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1 is a block diagram depicting an illustrative environment in which an on-demand code execution system is operable to process a data set by maintaining state information of the processed data set using a polling device such that separate execution environments within the on-demand code execution system do not need to maintain state information;

[0006] Figure 2 Describing Provide Figure 1 The overall architecture of the computing device of the worker manager;

[0007] Figure 3 is a diagram for maintaining the use of a polling device by Figure 1 A flowchart of illustrative interactions for executing code within an execution environment of an on-demand code execution system to process state information of a data set;

[0008] Figure 4 is a graphical depiction of the process of processing a data set and in particular of a device enabling a set of data items to be split into a plurality of subsets each of which can be processed independently during redistribution. Figure 1 A diagram of transmitting status information between polling devices;

[0009] Figure 5 is a graphical representation of a process for processing a data set and in particular of a means for enabling the merging of multiple sets of data items into a single superset during redistribution of the Figure 1A diagram of communicating status information between polling devices; and

[0010] Figure 6 is used to maintain the Figure 1 A declarative routine that executes code within an execution environment of an on-demand code execution system to process state information of a data set. DETAILED DESCRIPTION

[0011] In general, aspects of the present disclosure relate to an on-demand code execution environment that enables a client to request execution of user-defined code in order to process a set of data. More specifically, aspects of the present disclosure enable stateful processing of a set of data without maintaining state information in the execution environment in which the code is executed. In the context of data set processing, stateful processing generally refers to processing in which the result of processing a later data item in a set depends at least in part on the retained state information associated with the previously processed data item, which is distinct from stateless processing in which the operation does not depend on the retained state information. Stateful processing can be utilized to implement a map-reduce programming model, such as by counting the number of occurrences of a particular data item in a set. Although stateful processing is often desired by end users, the successful and resilient implementation of stateful processing may present challenges to distributed systems. Typically, such processing is implemented by code that stores state information locally, so that processing of later data items can be based on locally stored state information. However, since each data item in a data set must be routed to an environment containing the correct state information, local storage of state information prevents portability of code execution. Local storage of state information also places additional demands on the resilience of the execution environment, as loss of state information can leave analysis of a data set in an uncertain state. For these and other reasons, many systems that were originally intended for stateful processing, such as on-demand code execution systems (sometimes referred to as "serverless" computing systems), do not support stateful code execution in general. The embodiments disclosed herein address these issues by providing stateful processing within execution environments without the need to store state information locally within those environments.

[0012] For ease of explanation, embodiments of the present disclosure are described with reference to an implementation of a map-reduce programming model, which can be used to analyze large data sets and generally relies at least in part on stateful code to "reduce" the output of the initial "map" phase of the model. As known in the art, the map-reduce programming model is a logical data processing model for enabling a distributed computing system (e.g., including multiple different devices that communicate with each other) to process data sets and, in particular, large data sets (e.g., on the order of gigabytes, terabytes, petabytes, etc.). In general, the map-reduce programming model includes at least two data processing phases: a mapping phase and a reduction phase. During the mapping phase, a separate device within the distributed computing system collects a portion of the data set (e.g., a block of n gigabytes) and "maps" the contents of the portion into a set of outputs. Each set of outputs is then processed by a "reduce" function to aggregate, combine, or otherwise determine the result of processing the contents within the output set. A common example of a map-reduce paradigm is to count the number of occurrences of a word within a corpus. For example, suppose a user obtains a text data set of several terabytes and wants to determine which words exist in the data set and the frequency of each word. It takes a lot of time to process the data set using a single device. However, under the map-reduce model, each device in the distributed system can process a limited part (or "blocking") of the data set according to a mapping function. In this example, the mapping function can correspond to counting the number of occurrences of each word in the part of the data set. By increasing the number of devices in the system, the processing speed can be improved. The limitation of this processing is that each device will only process a limited part of the data set, and therefore only understand the count of the words in the part. In order to aggregate counts across devices, a reduction function is used. Although a single device can be used to aggregate the word counts generated at each mapping function, this configuration once again causes a processing bottleneck. In order to solve this problem, the map-reduce model enables multiple reduction functions to be implemented across distributed systems, and each function processes a subset of the output of the mapping function according to the attributes of the output. For example, in the example of word counting, 26 reduction functions can be implemented, each of which produces a count of words that begin with a specific character in the English alphabet. Illustratively, a first reduce function may obtain from a mapping function the total count of words beginning with the letter 'a', a second reduce function may obtain the total count of words beginning with 'b', and so on. Thus, each reduce function may obtain the total word count of words in a portion of a corpus. The outputs of the reduce functions may be combined to obtain the total word count. Typically, combining the outputs of reduce functions is a relatively low cost operation because the outputs are distinct and non-overlapping. Therefore, the mapreduce programming model may be utilized to facilitate fast distributed processing of data sets.

[0013] In view of the above, it will be understood that the simplification stage of the map-reduce model is usually implemented in a stateful manner, because the result of the nth data item in the processing data set generally depends on the result of processing the (n-1)th data item. In order to provide a stateful simplification stage, the map-reduce model has been traditionally implemented on a dedicated distributed system, and the dedicated distributed system is configured with the framework software of the generation and configuration of the worker computer node in the processing system. Among other potential advantages, the use of dedicated worker nodes provides a close relationship between worker nodes and the parts of the data set to which they implement the simplification stage. Each worker node can be provided with the appropriate part of the output of the mapping stage, so as to ensure that the simplification stage is correctly performed relative to the part. However, traditional map-reduce implementations also have disadvantages. For example, worker nodes are usually only dedicated to the realization of the map-reduce model, thereby limiting the ability of their execution of other computing tasks. Therefore, setting up this distributed system may be time-consuming and inefficient, particularly when the user does not need a continuous dedicated system to realize the map-reduce model.

[0014] Embodiments of the present disclosure solve the above-noted problems by typically enabling stateful processing (such as the reduce phase of a map-reduce programming model) within a stateless execution environment (such as those provided by an on-demand code execution system). As will be described in more detail below, embodiments of the present disclosure enable an intermediary device to maintain state information associated with iterative data processing (such as the reduce phase of a map-reduce model) and to submit state information in each request to execute code on the on-demand code execution system. The intermediary device may obtain updated state information in response to each call and include the updated state information in the next call. In this way, the execution environment on the on-demand code execution system is exempted from the responsibility of maintaining state information and may continue to operate statelessly. However, despite this, state information may still be maintained when processing a set of data, enabling successful stateful analysis of the data set. As will be described below, the intermediary device may be configured to ensure operational resilience so that failures within the system that processes the data set may be identified and corrected. In addition, the intermediary device may be configured to ensure effective resilience so that providing resilience does not have a substantial negative impact on the system's ability to process data sets.

[0015] In one embodiment, the intermediate device is a poller device that operates to retrieve a set of calls (the set of calls invoke code on the on-demand code execution system (e.g., to process data items included or referred to in the calls)) and pass each call to the on-demand code execution system. The set of calls can be illustratively maintained as a queue, such as a message queue on a stream data processing system. Illustratively, a poller device according to an embodiment of the present disclosure can determine initial state information for processing data items on the queue (e.g., as an empty state), retrieve a batch of initial calls from the queue, and iteratively submit each call and current state information representing the state of the processing queue to the on-demand code execution system. The poller device can be configured to receive updated state information in response to each call, which information can be included in subsequent calls. Because the state information of the queue is passed in each call, the environment on the on-demand code execution system itself does not need to maintain state information. For this reason, there is generally no need for a close relationship between the poller device and the environment in which the call is processed. Instead, the on-demand code execution system can route calls from the poller device to any suitable environment, thereby increasing the flexibility of the on-demand code execution system in executing code corresponding to the call. The poller device may be configured to periodically save state information to a resilient storage system (e.g., a network storage location with built-in redundancy) and resume processing based on the saved state information in the event of a failure. Thus, maintaining state information at the poller device provides an efficient mechanism for implementing stateful data processing at an on-demand code execution system.

[0016] Although other mechanisms for implementing stateful data processing at the on-demand code execution system are considered in this article, these other mechanisms are generally not as satisfactory as maintaining state information at the intermediate (e.g., poller) device. For example, it is conceivable that the on-demand code execution system is configured to provide close relationships for multiple calls for executing a given set of codes, so that each call is routed to the same execution environment. It is further conceivable that the on-demand code execution system enables each such environment to maintain local state information, thereby enabling stateful execution of code within the environment. However, this method significantly reduces the operational flexibility of the on-demand code execution system, which requires the system to maintain the execution environment for a long time. In addition, this method may not be suitable for solving problems that frequently occur in distributed processing systems, such as the need to provide operational flexibility or the need to expand or reduce the number of environments in response to changing operational loads. For example, in order to solve these problems, the on-demand code execution system may need to frequently save the state information of each environment, thereby significantly increasing the resource utilization of the system. The system may also need to provide the transmission of state information between environments during the zoom-in or zoom-out event, thereby increasing the complexity of managing such environments. Another possible mechanism for retaining state information between the processing of calls is to configure each execution environment to write its state information to a persistent external location, such as a network data storage device, during the processing of the call. Therefore, subsequent executions can retrieve the state information from the persistent location to facilitate the processing of subsequent calls. However, in distributed systems, writing to an external storage location is generally considered a "heavy" operation because it significantly increases the computing resources used to process the call. For example, writing to a network location may require initiating a transmission control protocol (TCP) session with the network location, a process that takes up a lot of time and resources (in terms of resources originally required to process a single call). In the case of a large number of calls (for example, when processing a data set of several terabytes), the additional overhead required for such a heavy operation can be very large.

[0017] Embodiments of the present disclosure enable state information to be maintained between the processing of calls without these disadvantages. For example, an intermediary device may pass state information of a call while submitting the call to an on-demand code execution system, and may receive updated state information in response to the call. Thus, the on-demand code execution system does not require additional network communications. In addition, the intermediary device may provide resilience by periodically saving state information with a periodicity that is adjustable based on the resources available to the device and the overhead required to restore operation in the event of a failure. Specifically, because the intermediary device is able to take a "long-term" view of the processing of a call queue, the device does not need to ensure that state information is saved after each call, as an on-demand code execution system or an external data storage device can do.

[0018] As described in detail herein, an on-demand code execution system may provide a network-accessible service that enables a user to submit or mark a computer executable code to be executed by a virtual machine instance on an on-demand code execution system. Each set of code on an on-demand code execution system may define a "task" and implement a specific functionality corresponding to the task when executed on a virtual machine instance of the on-demand code execution system. The separate implementation of a task on an on-demand code execution system may be referred to as the "execution" (or "task execution") of a task. The on-demand code execution system may further enable a user to trigger the execution of a task based on a variety of potential events (such as, detection of new data at a network-based storage system, transmission of an application programming interface ("API") call to an on-demand code execution system, or transmission of a specially formatted hypertext transfer protocol ("HTTP") packet to an on-demand code execution system). Therefore, a user may utilize an on-demand code execution system to execute any specified executable code "on demand" without configuring or maintaining the underlying hardware or infrastructure on which the code is executed. Additionally, the on-demand code execution system may be configured to execute tasks in a rapid manner (eg, within 100 milliseconds [ms]), thereby enabling execution of tasks in "real time" (eg, with little to no end-user-perceptible delay).

[0019] Because the on-demand code execution system can provide the ability to execute tasks on demand without configuring the underlying device on which the code is to be executed, the on-demand code execution system can provide an excellent platform on which the map-reduce programming model is to be implemented. For example, as described in U.S. Patent Application No. 16 / 144,997 ("the '997 application"), entitled "MAPREDUCE IMPLEMENTATION IN AN ON-DEMAND NETWORK CODE EXECUTION SYSTEM AND STREAM DATA PROCESSING SYSTEM" filed on September 27, 2018 (the "'997 application") (this application is hereby incorporated by reference in its entirety), the on-demand code execution system can be used in conjunction with a stream data processing system to implement a map-reduce programming model without the need for a centralized controller. The present disclosure can further improve the operation of the on-demand code execution system to implement the map-reduce programming model or other data analysis by providing an efficient way to maintain state information of such data analysis without maintaining such state information within the environment of the on-demand code execution system, or persisting such state information to another external location by such an environment.

[0020] As will be appreciated by those skilled in the art in view of the present disclosure, the embodiments disclosed herein improve the ability of a computing system (such as an on-demand code execution system) to process and analyze data sets in a parallelizable manner. More specifically, the embodiments disclosed herein enable state information to be effectively maintained between code executions on an on-demand code execution system without the need to maintain such state information in the execution environment of the system. In addition, the embodiments disclosed herein address technical problems inherent in computing systems; specifically, the need to maintain state information when processing data sets in a stateful manner, and the difficulty of maintaining such information when such processing occurs without increasing computing resources or reducing flexibility for processing data sets. These technical problems are solved by various technical solutions described herein, which include using an intermediate device to retrieve calls to an on-demand code execution system from a queue, and iteratively submitting calls to the on-demand code execution system while maintaining state information passed with the call. Therefore, the present disclosure generally represents an improvement to existing data processing systems and computing systems.

[0021] The general execution of tasks on the on-demand code execution environment will now be discussed. Specifically, in order to perform tasks, the on-demand code execution environment described herein can maintain a pre-initialized virtual machine instance pool ready to be used as soon as a user request is received. Due to the pre-initialization nature of these virtual machines, the delay (sometimes referred to as latency) associated with executing user code (such as instance and language runtime startup time) can be significantly reduced, usually reduced to a level below 100 milliseconds. Illustratively, the on-demand code execution environment can maintain a pool of virtual machine instances on one or more physical computing devices, wherein each virtual machine instance has one or more software components (e.g., operating system, language runtime, library, etc.) loaded thereon. When the on-demand code execution environment receives a request to execute the user's program code ("task") (the request specifies one or more computing constraints for executing the user's program code), the on-demand code execution environment can select a virtual machine instance for executing the user's program code based on one or more computing constraints specified by the request, and cause the user's program code to be executed on the selected virtual machine instance. The program code can be executed in an isolated container created on a virtual machine instance. Because the virtual machine instances in the pool have already been started and loaded with a specific operating system and language runtime before a request is received, the latency associated with finding computing power that can process a request (e.g., by executing user code in one or more containers created on the virtual machine instances) is significantly reduced.

[0022] The on-demand code execution environment may include a virtual machine instance manager configured to receive user code (threads, programs, etc. written in any of a variety of programming languages) and execute the code in a highly scalable, low-latency manner without requiring the user to configure the virtual machine instance. Specifically, the virtual machine instance manager may create and configure the virtual machine instance according to a set of predetermined configurations before receiving the user code and before receiving any information about any particular virtual machine instance configuration from the user, each configuration corresponding to any one or more of a variety of runtime environments. Thereafter, the virtual machine instance manager receives a request to execute code initiated by the user, and identifies a preconfigured virtual machine instance to execute the code based on the configuration information associated with the request. The virtual machine instance manager may further dispatch the identified virtual machine instance to execute the user's code at least in part by creating and configuring a container inside the dispatched virtual machine instance. Various embodiments for implementing a virtual machine instance manager and executing user code on a virtual machine instance are described in more detail in U.S. Patent No. 9,323,556, entitled “PROGRAMMATIC EVENT DETECTION AND MESSAGEGENERATION FOR REQUESTS TO EXECUTE PROGRAM CODE,” filed on September 30, 2014 (the “'556 patent”), the entire contents of which are hereby incorporated by reference.

[0023] As used herein, the term "virtual machine instance" is intended to refer to the execution of software or other executable code that mimics the hardware that provides an environment or platform ("execution environment") on which software can be executed. Virtual machine instances are generally executed by hardware devices, which may be different from the physical hardware that the virtual machine instance mimics. For example, when executed on a processor and memory of a first type, a virtual machine may mimic a processor and memory of a second type. Therefore, a virtual machine can be used to execute software intended for a second execution environment (e.g., a second operating system) on a physical device that is executing a first execution environment (e.g., a first operating system). In some cases, the hardware mimicked by a virtual machine instance may be the same or similar to the hardware of the underlying device. For example, a device with a first type of processor may implement multiple virtual machine instances, each virtual machine instance mimicking an instance of the first type of processor. Therefore, a virtual machine instance can be used to divide a device into multiple logical sub-devices (each sub-device is called a "virtual machine instance"). Although virtual machine instances can generally provide an abstract level of hardware away from the underlying layer of the device, such abstraction is not necessary. For example, assuming that a device implements multiple virtual machine instances, each virtual machine instance mimics hardware that is exactly the same as the hardware provided by the device. In this case, each virtual machine instance can allow software applications to execute code on the underlying hardware without translation, while maintaining logical separation between software applications running on other virtual machine instances. This process (generally referred to as "native execution") can be used to increase the speed or performance of the virtual machine instance. Other technologies that allow direct use of the underlying hardware, such as hardware pass-through technology, can also be used.

[0024] Although a virtual machine executing an operating system is described herein as one example of an execution environment, other execution environments are possible. For example, a task or other process may be executed within a software "container" that provides a runtime environment without itself providing hardware virtualization. A container may be implemented within a virtual machine to provide additional security, or may be run external to a virtual machine instance.

[0025] The foregoing aspects of the present disclosure and many of the attendant advantages will be more readily appreciated as the following detailed description is better understood when considered in conjunction with the accompanying drawings.

[0026] Figure 11 is a block diagram of an illustrative operating environment 100 in which an on-demand code execution system 110 may operate based on communications with a user computing device 102, an auxiliary service 106, and a network-based data storage service 108. As an illustration, various exemplary user computing devices 102, including a desktop computer, a laptop computer, and a mobile phone, are shown in communication with the on-demand code execution system 110. In general, the user computing device 102 may be any computing device, such as a desktop, laptop or tablet computer, a personal computer, a wearable computer, a server, a personal digital assistant (PDA), a hybrid PDA / mobile phone, a mobile phone, an electronic book reader, a set-top box, a voice command device, a camera, a digital media player, and the like. The on-demand code execution system 110 may provide one or more user interfaces, command line interfaces (CLIs), application programming interfaces (APIs), and / or other programming interfaces to the user computing device 102 for generating and uploading user executable code, calling user-provided code (e.g., submitting a request to execute user code on the on-demand code execution system 110), scheduling event-based jobs or timed jobs, tracking user-provided code, and / or viewing other log records or monitoring information related to its requests and / or user code. Although one or more embodiments may be described herein as using a user interface, it should be understood that such embodiments may additionally or alternatively use any CLI, API, or other programming interface.

[0027] The illustrative environment 100 also includes one or more auxiliary services 106, which can interact with the on-demand code execution system 110 to implement the desired functionality on behalf of the user. The auxiliary service 106 may correspond to a network-connected computing device such as a server, which generates data accessible to the on-demand code execution system 110 or otherwise communicates with the on-demand code execution system 110. For example, the auxiliary service 106 may include a web service (e.g., associated with the user computing device 102, the on-demand code execution system 110, or a third party), a database, a real simple integration ("RSS") reader, a social networking site, or any other network-accessible service source or data source. In some cases, the auxiliary service 106 may be associated with the on-demand code execution system 110, for example, to provide billing or logging services to the on-demand code execution system 110. In some cases, the auxiliary service 106 actively transmits information such as API calls or other task triggering information to the on-demand code execution system 110. In other cases, the auxiliary service 106 can be passive, so that the data is accessible to the on-demand code execution system 110. As described below, components of the on-demand code execution system 110 may periodically poll such passive data sources and trigger the execution of tasks within the on-demand code execution system 110 based on the data provided. Figure 1106 are depicted as being distinct from the user computing device 102 and the on-demand code execution system 110 , but in some embodiments, the various auxiliary services 106 may be implemented by the user computing device 102 or the on-demand code execution system 110 .

[0028] The illustrative environment 100 also includes one or more network-based data storage services 108, which are configured to enable the on-demand code execution system 110 to store data and retrieve data from one or more persistent or substantially persistent data sources. Illustratively, the network-based data storage service 108 can enable the on-demand code execution system 110 to retrieve a set of data to be analyzed, and store information (e.g., results) about the analysis. The network-based data storage service 108 can represent, for example, a relational or non-relational database. In another example, the network-based data storage service 108 can represent a network attached storage (NAS), which is configured to provide access to data arranged as a file system. Various other functionalities may be included in the network-based data storage service 108 that can be used within the embodiments of the present disclosure. The network-based data storage service 108 can further enable the on-demand code execution system 110 to query and retrieve information about the data stored in the on-demand code execution system 110, such as by querying multiple related files or records, the size of these files or records, the file or record name, the file or record creation time, etc. In some cases, the network-based data storage service 108 may provide additional functionality, such as the ability to separate data into logical groups (e.g., groups associated with personal accounts, etc.). Although illustrated as distinct from auxiliary services 106, in some cases, the network-based data storage service 108 may also represent a type of auxiliary service 106.

[0029] The illustrative environment 100 also includes a stream data processing system 170. As discussed above, the stream data processing system may enable upstream devices to place data onto the message stream 172, such as by publishing a "message" onto the stream 172, which may be labeled based on a particular "topic." Figure 1A single stream 172 is shown in , but the system 170 may provide multiple streams on behalf of multiple parties. The system 170 may make messages within the stream 172 available to downstream devices, typically in a "first in, first out" ("FIFO") or near-FIFO order. In some cases, the stream data processing system 170 "pushes" messages to downstream devices. In other cases, downstream devices "pull" messages from the message stream 172 upon request. In general, the stream data processing system 170 is configured to provide resiliency so that data successfully published to the stream is less likely to be lost due to device failure of the stream data processing system 170. For example, the system 170 may replicate messages placed on the stream 172 to multiple computing devices used to implement the stream (e.g., physical computing devices or virtual devices implemented on a physical host). In addition, the stream data processing system 170 may be configured to provide parallelization of devices that maintain the message stream 172. For example, a user configuring a message stream may indicate a partition key for the stream, which is used to divide the stream into substreams, each of which is processed by one or more parallelized devices. Substreams are processed in parallel. Figure 1 174A-174N. Each message shard 174 may generally represent one or more computing devices configured to obtain and make available a subset of messages on a message stream, the subset selected by the system 170 based on a partition key and the volume of messages on the stream 170 (e.g., such that additional shards are created or excess shards are destroyed based on the ability of the shard 174 to service messages on the stream 172). Examples of stream data processing systems known in the art include AMAZON TM KINESIS TM Network Services and Apache TM KAFKA TM system.

[0030] The user computing device 102, auxiliary service 106, network-based data storage service 108, and stream data processing system 170 can communicate with the on-demand code execution system 110 through network 104, which may include any wired network, wireless network, or combination thereof. For example, network 104 may be a personal area network, a local area network, a wide area network, an air broadcast network (e.g., for radio or television), a cable network, a satellite network, a cellular telephone network, or a combination thereof. As another example, network 104 may be a publicly accessible network of linked networks that may be operated by various different parties, such as the Internet. In some embodiments, network 104 may be a private or semi-private network, such as a company or university intranet. Network 104 may include one or more wireless networks, such as a global system for mobile communications (GSM) network, a code division multiple access (CDMA) network, a long-term evolution (LTE) network, or any other type of wireless network. Network 104 may use protocols and components for communicating through any of the Internet or other aforementioned types of networks. For example, the protocols used by the network 104 may include Hypertext Transfer Protocol (HTTP), HTTP Secure (HTTPS), Message Queuing Telemetry Transport (MQTT), Constrained Application Protocol (CoAP), etc. Protocols and components used to communicate over the Internet or any of the foregoing types of communication networks are well known to those skilled in the art and are therefore not described in further detail herein.

[0031] The on-demand code execution system 110 and the stream data processing system 170 are Figure 1 is depicted as operating in a distributed computing environment that includes several computer systems interconnected using one or more computer networks ( Figure 1 ). One or both of the on-demand code execution system 110 and the stream data processing system 170 may also be configured to communicate with Figure 1 The invention is operated within a computing environment having a fewer or greater number of devices than illustrated in the example. Figure 1 The depiction of the on-demand code execution system 110 and the stream data processing system 170 in the 10000 embodiment should be considered illustrative rather than limiting of the present disclosure. For example, the on-demand code execution system 110 and the stream data processing system 170 or various components thereof may implement various web service components, hosted or "cloud" computing environments, and / or peer-to-peer network configurations to implement at least a portion of the processes described herein.

[0032] In addition, the on-demand code execution system 110 and the stream data processing system 170 may be implemented directly in hardware or software executed by a hardware device, and may, for example, include one or more physical or virtual servers implemented on physical computer hardware configured to execute computer executable instructions for performing the various features described herein. One or more servers may be geographically dispersed or geographically co-located (e.g., in one or more data centers). In some cases, one or more servers may operate as part of a system that rapidly supplies and releases computing resources (commonly referred to as a "cloud computing environment").

[0033] exist Figure 1 In the example of FIG. 1 , the on-demand code execution system 110 and the stream data processing system 170 are illustrated as being connected to the network 104. In some embodiments, any component within the on-demand code execution system 110 and the stream data processing system 170 can communicate with other components of the on-demand code execution system 110 and the stream data processing system 170 via the network 104. In other embodiments, another network (such as Figure 1 A private network (not shown) may enable communication between components within each of the on-demand code execution system 110 and the stream data processing system 170 or between these systems.

[0034] exist Figure 1 In the example of the embodiment of the present invention, a user may interact with an on-demand code execution system 110 through a user computing device 102 to provide executable code, and establish rules or logic that define when and how such code should be executed on the on-demand code execution system 110 to thereby establish a "task". For example, a user may wish to run a piece of code related to a web or mobile application that the user has developed. One method of running the code would be to obtain a virtual machine instance from a service provider that provides infrastructure as a service, configure the virtual machine instance to meet the needs of the user, and use the configured virtual machine instance to run the code. To avoid the complexity of this process, the user may alternatively provide the code to the on-demand code execution system 110 and request the on-demand code execution system 110 to execute the code using one or more pre-established virtual machine instances. The on-demand code execution system 110 may handle the acquisition and configuration of computing power (e.g., containers, instances, etc., described in more detail below) based on the code execution request, and use the computing power to execute the code. The on-demand code execution system 110 can automatically scale up and down based on capacity, thereby alleviating the burden on users of having to worry about over-utilization (e.g., acquiring too few computing resources and having performance issues) or under-utilization (e.g., acquiring more computing resources than necessary to run the code and thus overpaying). According to embodiments of the present disclosure, a task established by a user may correspond to code that can be executed to implement a "reduce" function with respect to a data set, or other functions that aggregate, combine, or otherwise determine a result based on the entire data set.

[0035] In order to enable interaction with the on-demand code execution system 110, the system 110 includes one or more front ends 120 that enable interaction with the on-demand code execution system 110. In an illustrative embodiment, the front end 120 acts as a "front door" to other services provided by the on-demand code execution system 110, enabling users to provide computer executable code (through the user computing device 102), request its execution, and view its results. The front end 120 includes a variety of components that implement the interaction between the on-demand code execution system 110 and other computing devices. For example, each front end 120 may include a request interface, which provides the user computing device 102 with the ability to upload or otherwise transmit the user-specified code to the on-demand code execution system 110 and request the execution of the code thereafter. In one embodiment, the request interface communicates with an external computing device (e.g., user computing device 102, auxiliary service 106, etc.) via a graphical user interface (GUI), CLI, or API. The front end 120 processes the request and ensures that the request is properly authorized. For example, the front end 120 can determine whether the user associated with the request is authorized to access the user code specified in the request.

[0036] As used herein, reference to user code may refer to any program code (e.g., program, routine, subroutine, thread, etc.) written in a specific programming language. In the present disclosure, the terms "code," "user code," and "program code" are used interchangeably. Such user code may be executed to, for example, implement a specific function in conjunction with a specific web application or mobile application developed by a user. As noted above, a separate set of user code (e.g., implementing a specific function) is referred to herein as a "task," and the specific execution of the code is referred to as "task execution" or simply "execution." Tasks may be written in JavaScript (e.g., node.js), Java, Python, and / or Ruby (and / or another programming language) by way of non-limiting examples. Tasks may be "triggered" to be performed on the on-demand code execution system 110 in a variety of ways. In one embodiment, a user or other computing device may transmit a request to perform a task, which may be generally referred to as a "call" to perform a task. Such a call may include the user code to be executed (or its location) and one or more variables for executing the user code. For example, a call may provide the user code of a task and a request to perform the task. In another example, the call may identify a previously uploaded task by name or identifier. In yet another example, the code corresponding to the task may be included in the call for the task and may be uploaded in a separate location (e.g., a storage device of the auxiliary service 106 or a storage system inside the on-demand code execution system 110) before the on-demand code execution system 110 receives the request. The on-demand code execution system 110 may change its execution strategy for the task based on the location of the code at which the task is available when processing the call for the task. The request interface of the front end 120 may receive a call to perform the task from the user as a hypertext transfer protocol secure (HTTPS) request. Moreover, when performing the task, any information (e.g., headers and parameters) included in the HTTPS request may also be processed and utilized. As discussed above, any other protocol (including, for example, HTTP, MQTT, and CoAP) may be used to pass the message containing the task call to the request interface.

[0037] The call to perform the task can specify one or more third-party libraries (including native libraries) to be used together with the user code corresponding to the task. In one embodiment, the call can provide a ZIP file to the on-demand code execution system 110, and the ZIP file contains the user code and any library corresponding to the task requested to be executed (and / or the identification of its storage location). In some embodiments, the call includes the program code indicating the task to be performed, the language in which the program code is written, the user associated with the call, and / or the metadata of the computing resources (e.g., memory, etc.) to be reserved for executing the program code. For example, the program code of the task can be provided with the call, uploaded by the user in advance, provided by the on-demand code execution system 110 (e.g., standard routines), and / or provided by a third party. In some embodiments, such resource-level constraints (e.g., how much memory to be allocated to execute a specific user code) are specified for a specific task and can remain unchanged during each task execution. In such cases, the on-demand code execution system 110 can access such resource-level constraints before receiving each individual call, and the individual call may not specify such resource-level constraints. In some embodiments, the call may specify other constraints, such as permission data indicating what kind of permission or authorization the call invokes to perform a task. Such permission data may be used by the on-demand code execution system 110 to access private resources (e.g., on a private network).

[0038] In some embodiments, the call may specify the behavior that should be used to process the call. In such embodiments, the call may include an indicator for enabling one or more execution modes for performing the tasks mentioned in the call. For example, the call may include a flag or header for indicating whether the task should be performed in debug mode, in which debugging and / or logging output (e.g., through a console user interface) generated in conjunction with the execution of the task may be provided back to the user. In this example, the on-demand code execution system 110 may check the call and look for the flag or header, and if present, the on-demand code execution system 110 may modify the behavior of the container (e.g., logging facility) in which the task is performed, and cause output data to be provided back to the user. In some embodiments, the user interface provided to the user by the on-demand code execution system 110 adds the behavior / mode indicator to the call. Other features, such as source code analysis, remote debugging, etc., may also be enabled or disabled based on the instructions provided in the call.

[0039] According to an embodiment of the present disclosure, the code submitted by the user may correspond to an analysis function of a set of data. The function may be embodied in computer executable code, represented by one or more analysis tasks 164 on the on-demand code execution system 110. In one embodiment, the analysis task 164 implements the simplification phase of the map-reduce programming model. In such a model, the specific functionality of the simplification phase may vary depending on the data to be processed. However, in general, the simplification function corresponds to the code that "simplifies" or aggregates multiple intermediate results into an aggregated result. For example, the initial mapping function may correspond to a code that processes a corpus of words (e.g., a book or a series of books) and generates counts of individual words (or in some cases, words of a specific category, such as surnames) within the corpus. The corresponding simplification function may correspond to a code that aggregates the counts of individual words such as those generated by a separate execution of the mapping function. Illustratively, the simplification function may obtain one thousand intermediate count data files such as those generated by one thousand instances of the mapping function, and generate a single data file that aggregates the counts of each word within the one thousand intermediate count data files. In the case where only a single simplification function is executed, the single data file may represent the analysis result. In the case where multiple reduce functions are executed, resulting in multiple outputs (each output aggregates individual counts within a set of inputs), additional reduce functions can process the multiple outputs to further aggregate word counts. This process can continue until a single reduce function is called to produce a single output file as the result of the analysis. Alternatively, each of the multiple reduce functions can write output to a common location (e.g., a database) as the final output of the analysis.

[0040] Because analysis functions are typically specific to the data to be processed, code corresponding to these functions can be provided by an end user requesting analysis of the corresponding data set. In some cases, the on-demand code execution system 110 can also provide one or more sets of corresponding analysis functions (e.g., corresponding to commonly desired analysis types and configured to process data within an expected format).

[0041] To enable storage of tasks, the on-demand code execution system 110 may include a task data storage device 160, which may correspond to a persistent or substantially persistent data storage device, such as a hard disk drive (HDD), a solid-state drive (SDD), a network attached storage device (NAS), a tape drive, or any combination thereof. According to an embodiment of the present disclosure, the task data storage device may include one or more analysis tasks 164.

[0042] To manage requests for code execution, front end 120 may include an execution queue ( Figure 1). Illustratively, the number of simultaneous task executions of the on-demand code execution system 110 is limited, and as such, new task executions initiated at the on-demand code execution system 110 (e.g., by an API call, by a call from a task that has already been executed or is being executed, etc.) can be placed on the execution queue 124 and processed, for example, in a first-in, first-out order. In some embodiments, the on-demand code execution system 110 may include multiple execution queues, such as a separate execution queue for each user account. For example, a user of the on-demand code execution system 110 (e.g., for cost reasons) may desire to limit the rate of task execution on the on-demand code execution system 110. Therefore, the on-demand code execution system 110 may utilize an account-specific execution queue to limit the rate of simultaneous task executions of a particular user account. In some cases, the on-demand code execution system 110 may prioritize task executions so that task executions of a particular account or a specified priority bypass the execution queue or are prioritized in the execution queue. In other cases, the on-demand code execution system 110 may execute the task immediately or substantially immediately after receiving a call for the task, and therefore, the execution queue may be omitted.

[0043] In addition to tasks that are performed based on explicit user calls and data from the auxiliary service 106, the on-demand code execution system 110 may in some cases operate independently to trigger the execution of tasks. For example, the on-demand code execution system 110 may operate (based on instructions from the user) to trigger the execution of a task at each of a plurality of specified time intervals (e.g., every 10 minutes).

[0044] The front end 120 may also include an output interface ( Figure 1 The output interface may be configured to output information about the execution of a task on the on-demand code execution system 110. The output interface may illustratively transmit data about the execution of the task (e.g., the result of the task, errors associated with the execution of the task, or details of the execution of the task (such as the total time required to complete the execution, the total data processed by the execution, etc.) to the user computing device 102 or to an auxiliary service 106, which may include, for example, a billing or logging service. The output interface may further enable the transmission of data such as service calls to the auxiliary service 106. For example, the output interface may be used during the execution of a task to transmit API requests to the auxiliary service 106 (e.g., to store data generated during the execution of the task).

[0045] In some embodiments, the on-demand code execution system 110 may include multiple front ends 120. In such embodiments, a load balancer ( Figure 1140A) to distribute incoming calls to multiple front ends 120, for example, in a round-robin manner. In some embodiments, the way in which the load balancer distributes incoming calls to multiple front ends 120 may be based on the location or status of other components of the on-demand code execution system 110. For example, the load balancer may distribute calls to front ends 120 that are geographically nearby, or to front ends that have the ability to serve the calls. In the case where each front end 120 corresponds to a separate instance of another component of the on-demand code execution environment (such as, the active pool 140A described below), the load balancer may distribute calls according to the capacity or load on these other components. As will be described in more detail below, in some cases, calls may be deterministically distributed between the front ends 120 so that a given call to perform a task will always (or almost always) be routed to the same front end 120. This may, for example, assist in maintaining an accurate execution record of a task to ensure that the task is only executed the desired number of times. Although the call distribution performed by the load balancer is illustratively described, other distribution techniques (such as, anycast routing) will be apparent to those skilled in the art.

[0046] The on-demand code execution system 110 also includes one or more worker managers 140 that manage execution environments, such as virtual machine instances 150 (illustrated as VM instances 150A and 150B, collectively referred to as "VMs"), for servicing incoming calls for execution tasks, and manage memory states of the execution environments. Although the following will be described with reference to virtual machine instances 150 as an example of such an environment, embodiments of the present disclosure may utilize other environments, such as software containers. Figure 1 In the illustrated example, each worker manager 140 manages an active pool 140A, which is a set (sometimes referred to as a pool) of virtual machine instances 150 executing on one or more physical host computing devices that are initialized to perform a given task (e.g., by loading the task code and any dependent data objects into the instance). The active pool 140A is illustratively implemented using the main memory (e.g., RAM) of the host device that implements the worker manager 140 or is under the control of the worker manager 140.

[0047] Although virtual machine instances 150 are described herein as being assigned to specific tasks, in some embodiments, instances may be assigned to a group of tasks such that the instance is associated with the group of tasks and any task in the group may be executed in the instance. For example, tasks in the same group may belong to the same security group (e.g., based on their security credentials) such that executing one task in one container on a particular instance 150 followed by executing another task in another container on the same instance does not pose a security risk. As another example, tasks in the group may share common dependencies such that the environment used to execute one task in the group may be quickly modified to support the execution of another task in the group.

[0048] Once the front end 120 has successfully processed the triggering event for executing the task, the front end 120 passes the request to the worker manager 140 to execute the task. In one embodiment, each front end 120 may be associated with a corresponding worker manager 140 (e.g., a worker manager 140 co-located with or geographically nearby the front end 120), and therefore, the front end 120 may pass most or all requests to the worker manager 140. In another embodiment, the front end 120 may include a location selector configured to determine the worker manager 140 to which the execution request is to be passed. In one embodiment, the location selector may determine the worker manager 140 to receive the call based on hashing the call, and distribute the call to the worker manager 140 selected based on the hash value (e.g., by a hash ring). Various other mechanisms for distributing calls between worker managers 140 will be apparent to those skilled in the art.

[0049] Thereafter, the worker manager 140 may modify the virtual machine instance 150 (if necessary) and execute the task code within the instance 150. Figure 1As shown, the corresponding instance 150 may have an operating system (OS) 152 (shown as OS 152A and OS152B), a language runtime 154 (shown as runtime 154A and runtime 154B), and user code 156 (shown as user code 156A and user code 156B). OS 152, runtime 154, and user code 156 may collectively enable execution of user code to implement tasks. In some cases, each VM 150 may be associated with additional information (such as state information) maintained across separate executions of tasks. For example, when initially created, VM 150 may initialize OS 152, and each time user code 156 is executed in VM 150, the state of VM 150 may change. The state of VM 150 may be maintained, for example, in registers of a virtual CPU of VM 150, in RAM of VM 150, in a virtual disk drive of VM 150, and the like. According to embodiments of the present disclosure, analysis tasks 164 can be performed across data sets without maintaining state in VM 150, even if VM 150 is configured to enable maintenance of state information. Beneficially, this can enable calls to perform analysis tasks 164 to be routed to any available VM 150, rather than to a VM 150 at which appropriate state information is stored.

[0050] As noted above, execution of a task at the on-demand code execution system 110 may be triggered based on an explicit call from the user computing device 102 (e.g., as received at the request interface 122). Alternatively or additionally, execution of a task at the on-demand code execution system 110 may be triggered based on data retrieved from one or more auxiliary services 106, network-based data storage services 108, or a streaming data processing system 170.

[0051] To facilitate interaction with auxiliary service 106, system 110 includes a polling fleet 130 that operates to poll auxiliary service 106, data storage service 108, or stream data processing system 170 for data. Illustratively, polling fleet 130 may include one or more computing devices (in Figure 1132N), the one or more computing devices are configured to periodically transmit requests to the auxiliary service 106, the data storage service 108, or the stream data processing system 170 to retrieve any new available data (e.g., social network "posts", news articles, files, records, etc.) and determine whether the data corresponds to user-established criteria that triggers the execution of tasks on the on-demand code execution system 110. Illustratively, the criteria for executing tasks may include, but are not limited to: whether new data is available at the auxiliary service 106, the data storage service 108, or the stream data processing system 170, the type or content of the data, or timing information corresponding to the data. In some cases, the auxiliary service 106, the data storage service 108, or the stream data processing system 170 may function to notify the front end 120 of the availability of new data, and thus with respect to such services, the polling fleet 130 may be unnecessary.

[0052] According to an embodiment of the present disclosure, a message stream 172 may be configured to store calls for processing data items within a data set. Illustratively, each message on the message stream may correspond to a call to perform an analysis task 164 relative to a separate data item within the data set. Messages may be distributed between shards 174A-174N based on a partition key for the message stream 172, which may be represented by or included in the data item of the message. In one embodiment, messages are placed on the message stream 172 by operations of the mapping phase of a map-reduce model. Exemplary embodiments for operations of such a mapping phase are discussed in the '997 application, incorporated by reference above. In another embodiment, messages are placed on the stream 172 by other mechanisms, such as during operation of a device generating data to be analyzed (e.g., "real time").

[0053] like Figure 1 As shown, the poller fleet 130 may be configured to include a dynamic number of poller devices 132A-132N (eg, implemented as virtual machine instances on an underlying computing system) based on the number of message fragments 174 within the message stream 172. For example, Figure 1As shown by the dotted lines in FIG. 1 , message slice 174A may correspond to poller device 132A, message slice 174B may correspond to poller device 132B, and so on. Therefore, as the number of message slices 174 changes (e.g., due to the amount of message streams), the number of poller devices 132 may also change. Therefore, the poller fleet 130 may communicate with the stream data processing system 170, and the system 170 may notify the poller fleet 130 of changes in the message slices 174. In this configuration, each poller device 132A may be configured to poll the message slice 174 to retrieve messages in the substream corresponding to the message slice phase. Messages may be retrieved individually or in batches (e.g., 10 messages per batch, 50 messages per batch, 100 messages per batch, 500 messages per batch, etc.). Thereafter, the poller device 132 may invoke a call for the analysis task 164 to process each message, the call including state information for processing the current state of the message within the shard 174 corresponding to the poller device 132. Upon receiving the results of executing the analysis task 164 to process the message, the device 132 may update its state information, which the device 132 may then submit with subsequent calls. In one embodiment, a call to the analysis task 164 may be made for each individual message. In another embodiment, a call to the analysis task 164 may be made for each batch of messages, where the batch size matches or differs from the batch size used to retrieve messages from the message shards. In some cases, the calls executed from each poller device 132 to the analysis task 164 may be made synchronously, so that the poller device 132 waits for confirmation of successful execution before making the next call. The use of synchronous calls may advantageously rate limit calls to the reduce task, so that an overall one-to-one correspondence is maintained between the number of poller devices 132 and the number of analysis tasks 164 executed.

[0054] Although some functionality is generally described herein with reference to individual components of the on-demand code execution system 110 or the stream data processing system 170, other components or combinations of components may additionally or alternatively implement such functionality. For example, although the poller device 132A is operable to poll the message slice 174 for messages, the message slice 174 may additionally or alternatively be configured to notify the on-demand code execution system 110 (e.g., the front end) of new messages on the slice 174.

[0055] Figure 2 Depicted is the overall architecture of a computing system (referenced as worker manager 140 ) that manages virtual machine instances in on-demand code execution system 110 . Figure 2 The overall architecture of the worker manager 140 depicted in FIG. 1 includes an arrangement of computer hardware and software modules that can be used to implement various aspects of the present disclosure. The hardware modules can be implemented using physical electronic devices, as discussed in more detail below. The worker manager 140 can include: Figure 2More (or fewer) elements are shown in the drawings. However, it is not necessary to show all of these generally conventional elements in order to provide an enabling disclosure. Additionally, Figure 2 The overall architecture shown in can be used to implement Figure 1 104. As illustrated, the worker manager 140 includes a processing unit 190, a network interface 192, a computer-readable medium drive 194, and an input / output device interface 196, all of which can communicate with each other via a communication bus. The network interface 192 can provide a connection to one or more networks or computing systems. Therefore, the processing unit 190 can receive information and instructions from other computing systems or services via the network 104. The processing unit 190 can also communicate to or from the memory 180 via the input / output device interface 196, and further provide output information for an optional display (not shown). The input / output device interface 196 can also receive input from an optional input device (not shown).

[0056] The memory 180 may contain computer program instructions (grouped into modules in some embodiments) that the processing unit 190 executes to implement one or more aspects of the present disclosure. The memory 180 generally includes random access memory (RAM), read-only memory (ROM), and / or other persistent, auxiliary or non-transitory computer-readable media. The memory 180 may store an operating system 184 that provides computer program instructions for use by the processing unit 190 in the general management and operation of the worker manager 140. The memory 180 may also include computer program instructions and other information for implementing various aspects of the present disclosure. For example, in one embodiment, the memory 180 includes a user interface unit 182 that generates a user interface (and / or instructions thereof) to be displayed on the computing device, for example, through a navigation and / or browsing interface (such as a browser or application installed on the computing device). Additionally, the memory 180 may include and / or communicate with one or more data repositories (not shown) to, for example, access user program code and / or libraries.

[0057] In addition to and / or in conjunction with the user interface unit 182, the memory 180 may include an instance dispatch unit 186 and a user code execution unit 188 that may be executed by the processing unit 190. In one embodiment, the user interface unit 182, the instance dispatch unit 186, and the user code execution unit 188 individually or collectively implement various aspects of the present disclosure, such as finding a computing capability (e.g., a container) to be used to execute user code, causing the user code to be loaded and executed on the container, etc., as further described below.

[0058] Instance dispatch unit 186 looks for computing capacity to be used to service the request to execute user code. For example, instance dispatch unit 186 identifies virtual machine instances and / or containers that satisfy any constraints specified by the request and assigns the identified virtual machine instances and / or containers to the user or the request itself. Instance dispatch unit 186 may perform such identification based on the programming language in which the user code is written. For example, if the user code is written in Python, instance dispatch unit 186 may look for a virtual machine instance and / or container preloaded with the Python runtime (e.g., Figure 1 In another example, if the program code specified in the user's request has been loaded on an existing container or assigned to the user (e.g., Figure 1 If the request is not processed on another virtual machine instance in the active pool 140A of the request, the instance dispatch unit 186 may cause the request to be processed in the container or in a new container on the virtual machine instance. In some embodiments, if multiple language runtimes are loaded on the virtual machine instance, the instance dispatch unit 186 may create a new container on the virtual machine instance based on the computing constraints specified in the request and load the appropriate language runtime on the container.

[0059] Once a particular virtual machine instance has been assigned to a user associated with a request, and a container on the particular virtual machine instance has been assigned to the request, user code execution unit 188 manages the execution of program code specified by the user's request. If the code is preloaded in a container on the virtual machine instance assigned to the user, the code is executed only in the container. If the code is available through a network storage device (e.g., Figure 1 The user code execution unit 188 downloads the code to a container on the virtual machine instance and causes the code to be executed once it has been downloaded (e.g., by communicating with Figure 1 front-end 120 communication).

[0060] Although the instance dispatch unit 186 and the user code execution unit 188 Figure 2 140, but in other embodiments, all or part of the instance dispatch unit 186 and the user code execution unit 188 may be implemented by other components of the on-demand code execution system 110 and / or another computing device. For example, in certain embodiments of the present disclosure, another computing device in communication with the on-demand code execution system 110 may include several modules or components that operate similarly to the modules and components illustrated as part of the worker manager 140.

[0061] In some embodiments, the worker manager 140 may also include, in addition to Figure 2For example, memory 180 may also include a container manager for managing the creation, preparation, and configuration of containers within a virtual machine instance.

[0062] refer to Figure 3 , depicts an illustrative interaction for analyzing a set of data items by executing an analysis task 164 on an on-demand code execution system 110 while maintaining state information of such analysis at a poller device 132. In one embodiment, Figure 3 The interaction description may be performed by the system 110 to implement the interaction of the reduce phase of the map-reduce programming model.

[0063] Figure 3 The interaction begins at (1), where the submission device 301 enqueues a call to the analysis task 164 on the message flow 172. The submission device 301 may generally correspond to any device, which may include the user device 102, the device of the auxiliary service 106, the device of the on-demand code execution system 110, etc. For example, the submission device 301 may be a device of the on-demand code execution system 110 that has executed the mapping function. As another example, the submission device 301 may be an end-user device that submits data to be analyzed by the analysis function. The enqueued call generally includes a data item to be analyzed as part of a data set represented by the call on the message flow 172. In some cases, the enqueued call may be an explicit call to invoke the analysis task 164, such as by identifying the analysis task 164. In other cases, the enqueued call may not be an explicit call, but may simply pass the data item, but this results in a call to let the analysis task 164 process the data item due to the configuration of the flow 172, the on-demand code execution system 110, or both.

[0064] At (2), the message stream 172 (e.g., the stream data processing system 170 hosting the message stream 172) enqueues the call onto the appropriate shard 174 of the stream 172. As discussed above, the particular shard onto which the call is placed may depend, for example, on the partition key of the stream 172 as applied to the data item of the call. For example, the stream data processing system 170 may implement a consistent hashing mechanism (examples of which are known in the art) that selects the shard 174 onto which the call is to be placed based on attributes of the data item that has been identified as the partition key of the stream 172. At (3), the stream 172 confirms to the submission device 301 that the call has been enqueued.

[0065] exist Figure 3, interactions (1) through (3) represent an asynchronous call loop 302 that may be repeated any number of times to enqueue additional calls onto stream 172. Loop 302 is labeled as being associated with asynchronous calls because confirmation of each enqueue is delivered asynchronously relative to the actual implementation of the call. Thus, the interactions of asynchronous call loop 302 may occur independently of other interactions. For example, while the interactions of loop 302 are Figure 3 301. In the example of FIG. 10 , the first and second interactions are shown in the flow 172 before other interactions, but these interactions may be repeated at least partially simultaneously with other interactions. In addition, the interactions may be repeated by multiple different submission devices 301. Thus, by repeating interactions (1)-)(3), multiple calls (e.g., each call related to a data item of a data set to be analyzed) may be queued onto the message flow 172.

[0066] Interactions (4) through (10) illustrate interactions that may occur to process calls from individual shards 174 of a message stream 172 by submitting those calls to the on-demand code execution system 110. Although interactions (4) through (10) are described with respect to particular shards 174 and poller devices 132, these interactions may be repeated (e.g., by different poller devices 132 operating at least partially in parallel) to process calls for each shard 174.

[0067] Specifically, at (4), the poller device 132 obtains the next one or more calls from the slice 174 of the stream 172. In some cases, the poller device 132 may obtain a single call from the slice 174. In other cases, the poller device 132 may obtain multiple calls from the slice 174, such as the next 10, 100, or 500 calls. For ease of reference, the next one or more calls are referred to herein as a "batch" of calls. Thus, the interactions (4) to (10) are described in detail in FIG. Figure 3 denoted as batch loop 304. These interactions may be repeated while calls exist within a given shard 174, for example.

[0068] As noted above, shards 174 illustratively operate in a generally (and possibly strictly) first-in, first-out ("FIFO") manner. Thus, while interactions (1) through (3) conceptually enqueue calls at the "back" of shards 174, interaction (4) obtains calls from the "front" of shards 174.

[0069] Thereafter, the poller device 132 enters a synchronous call loop 306 during which individual calls from the received batches are passed to the on-demand code execution system 100 for processing and state information reflecting the processing is maintained at the poller device 132. Specifically, at (5), the poller device 132 submits the next unprocessed call within the current batch to the on-demand code execution system (e.g., to the front end 120). Illustratively, in a first instance of loop 306, the submitted call may be the first call enqueued into the shard 174. The call may be submitted via any submission process accepted by the system 110, such as via an API call. Thereafter, at (6), the system 110 distributes the call to the on-demand code execution system 100. Figure 3 150, in which the code of the analysis task 164 is executed to process the call. Then at (7), the analysis task 164 is executed within the environment, and at (8), the result of the execution is returned to the polling device 132.

[0070] To facilitate the execution of the analysis task 164, at interaction (5), in addition to the call, the poller device 132 also submits state information reflecting the current state of the data items within the processing slice 174 to the on-demand code execution system 110. The poller device 132 may initialize the state information to a null value. Thus, the first call to the on-demand code execution system 110 may include a null value, or may not include state information. The result of the task execution, as returned at (8), may include new state information, such as updated to reflect the task execution. At (9), the poller device 132 updates its current state information with the new state information (e.g., by replacing its own state information with the new state information), and if interaction (5) is repeated, the updated state information may then be passed in the next call.

[0071] As a specific example of interactions (5) through (10), consider an implementation of a mapreduce model that intends to count words within a corpus. In this case, each call within a particular shard 174 may include the words of the corpus as data items. The poller device 132 may initially set the state information to a null value, indicating that no words have yet been counted for the shard 174. As each call is submitted to the on-demand code execution system 110, the execution environment may return state information reflecting a current count value (e.g., initially null) that is incremented based on the words within the current call. Thus, a first call with the data item "apple" may result in state information "{apple:1}", indicating that the current count of the word "apple" is "1". A second call may include the state information and request processing of the word "ball". Execution of the analysis task 164 relative to this call may result in state information "{apple:1; ball:1}". This process may continue so that the state information returned by each task execution represents each call within a shard 174 of the processing flow. Although a simplistic word count example is provided herein, those skilled in the art will appreciate that data analysis applications can actually encompass a variety of very complex analyses.

[0072] exist Figure 3 , loop 306 is labeled as a synchronous call loop, indicating that in these illustrative interactions, the response to each call to the system 110 is received only after the execution of the analysis task 164 is completed, and includes the results of the execution. For example, interactions (5) and (8) may represent HTTP calls and responses, respectively, and are therefore referred to as "synchronous HTTP requests". The use of synchronous requests can beneficially reduce the computing resources required to communicate state information between the poller device 132 and the execution environment, because such state information can be communicated within the requests and responses that the on-demand code execution system 110 will provide in any case. Thus, for example, no additional network connection is required at the execution environment in order to communicate the state information of the task execution to the poller device 132. However, in some embodiments, it is contemplated that the poller device 132 may also utilize asynchronous calls. For example, the poller device 132 may submit an asynchronous request to the system 110 to process a call from a shard 174, and the task execution may cause the execution environment to initiate communication with the poller device 132 to return the state information of the task execution.

[0073] In some embodiments, such as where the data analysis depends on the ordering of calls to the message stream 172, the call loop 360 may occur strictly iteratively, such that each call must be submitted to the system 110 and the result received before the poller device 132 submits the next call to the system 110. In other cases, the poller device 132 may implement the call loop 306 at least partially in parallel, such as by submitting multiple calls to the system 110 simultaneously.

[0074] In view of the above, it will be appreciated that interactions (5) through (9) are generally not dependent on any particular assignment of calls to particular execution environments by the on-demand code execution system 110. For example, each instance of the call loop 306 may result in the assignment of calls to a different execution environment. Beneficially, this configuration enables the environments to operate statelessly, such that they do not need to maintain any information about the state of processing message streams 172. Stateless operation may enable the on-demand code execution system 110 to operate more efficiently by assigning calls to environments based on, for example, the computing resources available to such environments rather than on whether such environments have appropriate state information. In this way, overall efficiency is improved relative to alternative state maintenance solutions.

[0075] As discussed above, it is generally expected that distributed systems operate resiliently so that the failure of one or more devices in the system does not significantly affect the operation of the entire system. In this model, it is assumed that the message stream 172 operates resiliently in most failure situations, such as by providing redundant storage of messages queued on the message stream. It is similarly assumed that the entire system 110 operates resiliently in most failure situations, such as by providing multiple execution environments on different physical devices. It is not generally necessary for each execution environment to ensure resilience, because the poller device 132 can be configured to retry any failed calls to the execution analysis task 164, and the system 110 can illustratively distribute failed calls to different (non-failed) environments. Indeed, it may be expected to provide the resilience of separate execution environments on the system 110 with little effort, because providing resilience generally increases the computing resources required to provide such environments.

[0076] Given the assumed resilience of the message flow 172 and the on-demand code execution system, it may be desirable for the poller device 132 to also operate resiliently so that data analysis tasks can proceed even under a wide variety of failure scenarios. To provide such resilience, the poller device 132 may periodically or occasionally write current state information reflecting the state of the shards 174 of the processing flow 172 to a resilient data store, such as the network data storage service 108. It is generally desirable for such writes to occur as infrequently as possible, as these writes are typically "heavy" operations. In the Figure 3In the illustrative interaction of , the write is shown as occurring at interaction (10) at the end of batch cycle 304. Thus, as poller device 132 processes each batch of calls, state information reflecting the processing of the batch (and any previous batches) may be written to data storage service 108. The state information also illustratively reflects the last call processed by poller device 132. The last call may be identified, for example, by reference to a position in shard 174 of stream 172.

[0077] If the poller fleet 130 detects that a poller device 132 has failed, a new poller device 132 may be created and initialized with the latest state information written to the data storage service 108. The new poller device 132 may then retrieve the next batch of calls (e.g., the next few messages of the shard 174 that are not reflected in the state information as having been processed), thereby effectively resuming the operation of the failed poller device 132 and providing resilient operation of the poller device 132. Because the frequency of writes to the data storage service 108 may be controlled based on the number of calls in each instance of the batch cycle 304 (the "batch size"), the operation of the poller device 132 may be tuned (e.g., by an administrator or end user) based on desired operation. For example, a large batch size may result in less frequent external writes, thereby reducing the computational resource usage of the poller device 132, but increasing the amount of calls that will need to be repeated in the event of a failure. Conversely, a small batch size may result in more frequent external writes, but may reduce the calls that are repeated in the event of a failure. In either case, embodiments of the present disclosure enable this tradeoff to be selected independently of the operation of other components, such as the on-demand code execution system 110 .

[0078] A beneficial aspect of the stream data processing system 170 described herein is the ability to scale its capacity based on the volume of messages in the message stream 172, thereby enabling the amount of parallelization of data analysis tasks to vary even during the operation of the data analysis. For example, an increase in the volume of calls enqueued onto the message stream 172 may cause the number of shards 174 of the stream to increase, thereby increasing the capacity of the stream 172. Conversely, a decrease in the volume of calls may cause the number of shards 174 to decrease. Because the poller device 132 may be configured to have a one-to-one correspondence with the shards 174, the poller device 132 may be configured to appropriately communicate status information under these conditions to prevent data analysis from being interrupted. Figure 4 and Figure 5 An exemplary illustration of how status information may be communicated between poller devices 132 is shown in FIG.

[0079] Specifically, Figure 4Depicts how state information can be communicated from the first poller device 132A corresponding to the first shard 174A in the event that the shard 174A "splits" into two new shards (shards 174B and 174C, respectively). For ease of illustration, it will be assumed that Figure 4 Shards 174A-174C in reflect calls to a data analysis task that computes the frequencies of data items within a data set. It will be further assumed (for simplicity) that the data set consists of the characters "a" and "b" in a repeating pattern: "a, b, a, b, a, b", etc. Assume that initial shard 174A is initially the only shard of stream 172 until stream 172 is split into two shards 174B and 174C. In this illustrative example, the characters of the data set are selected as the partition key for stream 172. Thus, shard 174B contains a series of characters "a", while shard 174C contains a series of characters "b". It is illustratively desired that the analysis of the data set maintain an accurate count of these characters as stream 172 is processed. Thus, for example, the first message of stream 172 (message number 1 of shard 174A) should result in a count of 1 for the character "a". The second message of stream 172 (message number 2 of fragment 174A) should result in a count of 1 for the character "a" and a count of 1 for the character "b", and so on.

[0080] exist Figure 4 , state information 402 of the corresponding poller device 132 is shown above each poller device 132 after processing (e.g., by analysis task 164) the message in the corresponding shard 174 to which the state information points. Thus, state information 402A shows state information maintained at poller device 132A after processing message number 2 on shard 174A. Figure 4 The state information of is depicted as a three-value tuple that includes an identifier (“I”) of the data item being analyzed, the value of the data item (e.g., a count), and the position of the value of the last modified data item within the message stream 172. Thus, in Figure 4 , the state 402A of the poller device 132 after processing message 2 of fragment 174 is that data item 'a' has been counted '1' times, where the last count was performed at "A1" of the first message representing fragment 174A, and data item 'b' has been counted '1' times, where the last count was performed at "A2" of the first message representing fragment 174A.

[0081] exist Figure 4, assume that after processing message number 2 of fragment 174A, the fragment is split into fragments 174B and 174C. Therefore, message 1 of fragment 174B represents a message that was originally message 3 of fragment 174A, message 1 of fragment 174C represents a message that was originally message 4 of fragment 174A, and so on. It is assumed that each poller device 132 has a one-to-one correspondence with a fragment 174. Therefore, if poller devices 132B and 132C are initialized with empty state information, their current state information will not accurately reflect the state of stream 172 relative to those data values ​​included in their corresponding fragments 174. That is, poller device 132B may maintain state information indicating that the 'a' on message 1 of fragment 174B is the first 'a' in stream 172 when in fact it is the second.

[0082] To address this situation, each poller device 132 may be configured to make their current state information available to the poller devices 132 corresponding to the "child" shard 174 when the shard 174 corresponding to the device is stopped. Illustratively, each poller device 132 may be configured to write their state information to the data storage service 108. Upon initialization, each poller device 132 may be configured to determine whether the corresponding shard 174 is a child shard of the previous shard 174. Illustratively, the stream data processing system 170 may make this information available to the poller fleet 130. If the poller device 132 corresponds to a child shard 174, the poller device 132 may be configured to initialize the saved state information of the poller device 132 corresponding to the parent shard 174 as its state information. Thus, at Figure 4 , each of poller devices 132B and 132C has been initialized using state information 402A of poller device 132A (as it existed when operation of poller device 132A last processed a message of stream 172) prior to processing messages of their respective shards 174B and 174C. For this reason, state information 402B of poller device 132B reflects that, starting with message 1 of shard 174B, a total count of 2 is assigned to character 'a'. Similarly, state information 402C of poller device 132C reflects that, starting with message 1 of shard 174C, a total count of 2 is assigned to character 'b'. Thus, the state information of respective devices 132B and 132C reflects accurate states with respect to those data items partitioned into their respective shards 174.

[0083] As in Figure 4174, in addition to the accurate values ​​of the data items partitioned to the respective shards 174, the state information of the poller devices 132B and 132C also includes outdated state information with respect to those data items that are not partitioned to the respective shards 174. For example, the state information 402B reflects a value of '1' for the data item 'b', even though the shard 174B has been partitioned from the stream 172 such that the data item 'b' is not included in the shard 174B. Similarly, the state information 402C reflects a value of '1' for the data item 'a', even though the shard 174C has been partitioned from the stream 172 such that the data item 'a' is not included in the shard 174C.

[0084] This redundancy is addressed by configuring the poller devices 132, or any other device desiring to obtain the overall result of processing calls in the stream 172, to deduplicate the values ​​reflected in the state information of the respective poller devices 132. Specifically, because each data item tuple includes the last position of the value of the updated data item in the stream 172, deduplication can be performed by marking the latest position tuple as authoritative for the data item.

[0085] Figure 5 An illustrative example of deduplication of state information is depicted in FIG. Figure 4 , Figure 5 Illustrative state information for poller devices 132B and 132C is depicted, reflecting the processing of calls on respective shards 174B and 174C, each corresponding to a data item in stream 172. Figure 5 , assuming that each of the poller devices 132B and 132C is in a position to process additional messages on their respective shards 174 , then the state information 402B and 402C thus reflect processing the third message on those respective shards 174 .

[0086] Figure 5 The diagram of FIG. 1 depicts how state information from multiple devices 132 may be combined when two shards 174 are merged into a single shard 174. Specifically, Figure 5 , shards 174A and 174B are depicted as being merged into shard 174D, which can be considered a sub-shard of shards 174A and 174B. Figure 4 As shown, each of the poller devices 132B and 132C may be configured to pass its state information to the poller device 132D corresponding to the sub-shard 174D upon stopping their respective shards 174. However, due to the interactions described above, the state information 402B and 402C contain redundant information, with each data item 'a' and 'b' having two potential values.

[0087] To eliminate redundant information, the poller device 132D is configured to deduplicate the state information received for each mother shard by treating the state information with the latest position value in the message stream 172 as authoritative, such as in Figure 5 , as reflected by the third value of each data item tuple. Thus, when comparing the two tuples of data item 'a' (i.e., a tuple with position value 'B3' (or message 3 of shard B) from state information 402B and a tuple with position value 'A1' from state information 402C), poller device 132D determines that state information 402B is authoritative for data item 'a' because shard 174B is a child shard of shard 174A and is therefore a later position in stream 172. If desired, poller device 132D may illustratively receive predecessor data from stream data processing system 170 to determine the relationship between shards 174 of stream 172. Through a similar process, poller device 132D may determine that state information 402C represents the value of data item 'b'. Thus, state information 402D reflects a deduplicated, accurate record of the counts of the two data items starting from the initialization of shard 174D.

[0088] Although simplified examples are described above, in view of the present disclosure, those skilled in the art will appreciate that sharing of state information between polling devices 132 and deduplication of such information can facilitate any number of modifications to the number of shards 174 of the stream 172 while enabling the polling device 132 to maintain accurate values ​​at least with respect to the data items partitioned to its corresponding shard 174. In addition, although a technique for deduplicating state information is discussed above, other techniques are contemplated in the present disclosure. For example, in the case where the value of a known data item changes only in a single direction (e.g., a count that can only increase), duplicate values ​​of the data item can be deduplicated by assuming the farthest value in the direction (e.g., the largest count) as authoritative. In this way, deduplication can be performed without the need to maintain the position of each value. As another example, in addition to or in lieu of position information within a shard, other ordering information of the individual data items being processed may be included. For example, the state information may include an absolute position identifier across a message stream (e.g., across multiple shards), or may include timestamp information corresponding to the ordered message. The timestamp information may include, for example, the time when the message was enqueued into the message stream. Thus, a higher timestamp value for state information may identify that the state information represents the state of processing a later message within a message stream and may be considered authoritative compared to state information representing the state of processing an earlier message in the message stream.

[0089] refer to Figure 6, a block diagram depicting an illustrative routine 600 for maintaining state information reflecting the state of processing messages (e.g., calls identifying data items) on a message stream 172 representing a data set at a poller device 132 will be described. The routine 600 illustratively may be implemented at a poller device 132 corresponding to a slice 174 of a message stream 172.

[0090] The routine 600 begins at block 602, where the poller device 132 obtains relevant previous state information related to the shard 174. Illustratively, if a previous poller device 132 was processing the shard 174 and failed, the current poller device 132 may obtain the state information of the previous poller device 132 and resume processing of the shard 174 from the last position reflected in the state information of the previous poller device 132. As a further illustration, if the shard 174 is a child shard of one or more previous shards 174, the poller device 132 may obtain and adopt the state information of the poller device 132 corresponding to the parent shard of the shard 174. Illustratively, upon initialization, the poller device 132 may query the stream data processing system 170 to identify whether the shard 174 has any parent shards 174. If so, the poller device 132 may obtain the state information of those parent shards 174, such as from the data storage service 108. If the mother shard 174 is not present, the poller device 132 may initialize the state information to a null value or a predefined value established when the device 132 is configured.

[0091] At block 604, the poller device 132 deduplicates the state information of the parent shard if any duplicate information exists. As noted above, deduplication may include considering the value associated with the furthest position in the message stream 172 as authoritative for a given data item.

[0092] At block 606, the poller device 132 enters the batch loop 304, as described above with reference to Figure 3 304, and at block 606, a batch of calls is obtained from the shard 174. Each call may correspond to a message enqueued in the shard 174 and identify or include a data item from the data set to be analyzed. The number of calls retrieved at block 606 may vary depending on the desired frequency of the batch cycle 304. Illustratively, a batch of calls may include 500 calls.

[0093] At block 608, the poller device 132 enters the synchronous call loop 306 and submits the next call from the current batch to the on-demand code execution system 110 along with the current state information maintained at the poller device 132. As noted above, the passing of state information enables the call to be routed to any appropriate execution environment by the system 110. The call may be submitted, for example, as a synchronous HTTP request.

[0094] At block 610, poller device 132 obtains a response to the call, which includes new state information. The new state information illustratively represents the state information passed within the call as updated by execution of analysis task 164 on on-demand code execution system 110. Poller device 132 may therefore consider the new state information to be authoritative because it reflects the state of the data item identified in the shard.

[0095] At block 612, if there are additional calls within the batch that have not yet been submitted to the on-demand code execution system, the routine 600 returns to block 608. Alternatively, the routine 600 proceeds to block 614. At block 614, the poller device 132 checks its status, such as by writing current status information to the elastic network data storage service. As noted above, if the poller device 132 fails during implementation of the routine 600, a subsequent poller device 132 can utilize the status information to resume processing of the shard 174.

[0096] At block 616, if there are additional calls within the shard 174, the routine 600 returns to block 606. Alternatively, if all calls within the shard have been processed, then processing of the shard may be considered complete, and the last written state information of the poller device 132 will represent processing of all data items on the shard 174. Accordingly, the routine 600 may end at block 618.

[0097] All methods and processes described above may be embodied in and fully automated by software code modules executed by one or more computers or processors. The code modules may be stored in any type of non-transitory computer-readable medium or other computer storage device. Some or all of the methods may alternatively be embodied in dedicated computer hardware.

[0098] Unless otherwise specifically stated, conditional language such as "can," "may," "might," or "might" should generally be understood in context to indicate that certain embodiments include and other embodiments do not include certain features, elements, and / or steps. Thus, such conditional language is generally not intended to imply that the features, elements, and / or steps are required in any way for one or more embodiments, or that one or more embodiments must include logic for deciding whether to include such features, elements, and / or steps, or whether to perform such features, elements, and / or steps in any particular embodiment, with or without user input or prompting.

[0099] Unless otherwise specifically stated, disjunctive language such as the phrase "at least one of X, Y, or Z" should be understood in context as generally used to indicate that an item, term, etc. can be X, Y, or Z, or any combination thereof (e.g., X, Y, and / or Z). Thus, such disjunctive language is generally not intended to, and should not, imply that certain embodiments require that at least one of X, at least one of Y, or at least one of Z each be present.

[0100] Unless expressly stated otherwise, articles such as "a" or "an" are generally interpreted as including one or more of the items described. Thus, phrases such as "a device configured to..." are intended to include one or more of the listed devices. Such one or more listed devices may also be collectively configured to perform the stated statements. For example, "a processor configured to perform statements A, B, and C" may include a first processor configured to perform statement A working together with a second processor configured to perform statements B and C.

[0101] Any routine description, element or box described herein and / or depicted in the accompanying drawings should be understood to potentially represent a code module, segment or portion including one or more executable instructions for implementing a specific logical function or element in the routine. Alternative embodiments are included within the scope of the embodiments described herein, in which, as will be understood by those skilled in the art, elements or functions may be deleted, or executed out of the order shown or discussed (including substantially synchronously or in reverse order), depending on the functionality involved.

[0102] It should be emphasized that many changes and modifications may be made to the above-described embodiments, and the elements in the embodiments should be understood to be among other acceptable examples. In this document, all such modifications and changes are intended to be included within the scope of this disclosure and protected by the following claims.

[0103] Examples of embodiments of the present disclosure may be described in terms of the following:

[0104] Clause 1. A system for maintaining state information representing a state of processing a message enqueued in a message stream, the system comprising:

[0105] a stream data processing system that maintains the message stream, wherein individual messages within the message stream correspond to requests to invoke user-specified code to process individual data items within a data set;

[0106] An on-demand code execution system configured to obtain the request to invoke the user-specified code and, for a single request, select an execution environment of the on-demand code execution system in which to execute the user-specified code; and

[0107] A poller device configured with computer executable instructions to:

[0108] Initializing the state information representing the state of processing messages enqueued in the message stream;

[0109] retrieving a set of messages from the message stream; and

[0110] For each message in the set of messages:

[0111] transmitting the state information representing the state of processing a message enqueued within the message flow and the request corresponding to the message to the on-demand code execution system, wherein the request requests an invocation of the user-specified code to process an individual data item identified by the message;

[0112] receiving a response to the call, the response comprising a result of processing the individual data item, wherein the result of processing the individual data item is based at least in part on processing the status of messages enqueued within the message flow; and

[0113] The state information is updated based at least in part on the response.

[0114] Clause 2. A system as described in clause 1, wherein the on-demand code execution system is further configured to implement multiple executions of the map task that enqueues the individual messages within the message flow as a map phase of a map-reduce programming model, and wherein the user-specified code implements the reduce phase of the map-reduce programming model.

[0115] Clause 3. A system as described in Clause 1, wherein the message stream is divided into multiple message fragments, wherein individual messages within the message stream are distributed among the multiple message fragments according to the data items, wherein the poller device corresponds to a first fragment of the multiple fragments from which the group of messages is retrieved, and wherein the status information represents a status of processing messages within the first fragment.

[0116] Clause 4. The system of clause 3, further comprising a second poller device configured with computer executable instructions to:

[0117] Initializing second state information representing a state of processing a message enqueued in a second message fragment among the plurality of fragments;

[0118] retrieving a second set of messages from the second message fragment; and

[0119] For each message in the second set of messages:

[0120] transmitting the state information representing the state of processing the message enqueued within the second message fragment and the request corresponding to the message to the on-demand code execution system, wherein the request requests an invocation of the user-specified code to process a separate data item identified by the message;

[0121] receiving a response to the request, the response comprising a result of processing the individual data item, wherein the result of processing the individual data item is based at least in part on processing the state of the message enqueued within the second message fragment; and

[0122] The state information representing the state of processing the message enqueued within the second message fragment is updated based at least in part on the response.

[0123] Clause 5. A computer-implemented method comprising:

[0124] Initializing state information representing a state of processing messages enqueued within a message stream, wherein individual messages within the message stream correspond to requests to execute user-specified code on an on-demand code execution system to process individual data items within a data set;

[0125] retrieving a set of messages from the message stream; and

[0126] For each message in the set of messages:

[0127] transmitting the state information representing the state of processing a message enqueued within the message flow and the request corresponding to the message to the on-demand code execution system, wherein the request requests an invocation of the user-specified code to process an individual data item identified by the message;

[0128] receiving a response to the request, the response comprising a result of processing the individual data item, wherein the result of processing the individual data item is based at least in part on processing the state of messages enqueued within the message flow; and

[0129] The state information is updated based at least in part on the response.

[0130] Clause 6. A computer-implemented method as described in Clause 5, wherein initializing the state information comprises: retrieving one or more past instances of the state information from a data storage device; and initializing the state information based at least in part on the one or more past instances of the state information.

[0131] Clause 7. The computer-implemented method of Clause 6, wherein the message stream is divided into a plurality of message slices, and wherein the one or more past instances of state information include state information from a failed device.

[0132] Clause 8. A computer-implemented method as described in Clause 6, wherein the message stream is divided into multiple message shards, wherein the group of messages is retrieved from a first shard of the multiple message shards, and wherein the one or more past instances of state information include state information from a device that processes messages enqueued in one or more parent shards of the first shard.

[0133] Clause 9. A computer-implemented method as described in Clause 8, wherein the one or more mother shards include at least two mother shards, and wherein the computer-implemented method further comprises: deduplicating state information of the at least two mother shards.

[0134] Clause 10. A computer-implemented method as described in Clause 9, wherein the state information of the at least two mother shards indicates, for an individual data item in the state information of the at least two mother shards, the last position in the message flow at which the value of the individual data item was modified.

[0135] Clause 11. The computer-implemented method of Clause 5, further comprising: writing the state information to a data storage device after receiving a response to each call corresponding to each message in the set of messages.

[0136] Clause 12. The computer-implemented method of Clause 11, further comprising: detecting a failure in a device implementing at least a portion of the computer-implemented method, and retrieving the state information from the data storage device to recover the implementation of the at least a portion of the computer-implemented method.

[0137] Clause 13. A non-transitory computer-readable storage medium comprising computer-executable instructions that, when executed by a computing system, cause the computing system to:

[0138] Initializing state information representing a state of processing messages enqueued within a message stream, wherein individual messages within the message stream correspond to requests to execute user-specified code on an on-demand code execution system to process individual data items within a data set;

[0139] retrieving a set of messages from the message stream; and

[0140] For each message in the set of messages:

[0141] transmitting the state information representing the state of processing a message enqueued within the message flow and the request corresponding to the message to the on-demand code execution system, wherein the request requests an invocation of the user-specified code to process an individual data item identified by the message;

[0142] receiving a response to the request, the response comprising a result of processing the individual data item, wherein the result of processing the individual data item is based at least in part on processing the state of messages enqueued within the message flow; and

[0143] The state information is updated based at least in part on the response.

[0144] Clause 14. The non-transitory computer-readable medium of Clause 13, wherein the state information and the call are transmitted to the on-demand code execution system using synchronous Hypertext Transfer Protocol (HTTP) requests.

[0145] Clause 15. The non-transitory computer-readable medium of Clause 14, wherein the response is an HTTP response to the HTTP request.

[0146] Clause 16. The non-transitory computer-readable medium of Clause 14, wherein the user-specified code executes statelessly on the on-demand code execution system without reference to state information maintained on the on-demand code execution system independent of a call to execute the user-specified code.

[0147] Clause 17. A non-transitory computer-readable medium as described in Clause 13, wherein, to initialize the state information, the computer-executable instructions cause the system to retrieve one or more past instances of the state information from a data storage device and initialize the state information based at least in part on the one or more past instances.

[0148] Clause 18. The non-transitory computer-readable medium of Clause 13, wherein the message stream is divided into a plurality of message slices, and wherein the one or more past instances of state information include state information from a failed device.

[0149] Clause 19. A non-transitory computer-readable medium as described in Clause 13, wherein the message stream is divided into multiple message shards, wherein the group of messages is retrieved from a first shard of the multiple message shards, and wherein the one or more past instances of state information include state information from a device that processes messages enqueued in one or more parent shards of the first shard.

[0150] Clause 20. A non-transitory computer-readable medium as described in Clause 19, wherein the one or more mother shards include at least two mother shards, and wherein the non-transitory computer-readable medium also includes the following computer-executable instructions, which can be executed by the computing system to deduplicate the status information of the at least two mother shards.

[0151] Clause 21. A non-transitory computer-readable medium as described in Clause 19, wherein the computer-executable instructions cause the system to simultaneously transmit a first request corresponding to a first message in the set of messages and a second request corresponding to a second message in the second set of messages to the on-demand code execution system.

Claims

1. A system for maintaining state information representing a state of processing a message enqueued in a message stream, the system comprising: a stream data processing system that maintains the message stream, wherein individual messages within the message stream correspond to calls to invoke user-specified code to process individual data items within a data set; an on-demand code execution system configured to obtain the call invoking the user-specified code to process an individual data item within a data set and, for the individual call, select an execution environment of the on-demand code execution system in which to execute the user-specified code; and A poller device configured with computer executable instructions to: Initializing the state information representing the state of processing messages enqueued in the message stream; retrieving a set of messages from the message stream; and For each message in the set of messages: transmitting to the on-demand code execution system the state information representing the state of processing a message enqueued within the message flow and the call corresponding to the message for invoking the user-specified code to process an individual data item within a data set, wherein the call requests to invoke the user-specified code to process an individual data item identified by the message; receiving a response to the call, the response comprising a result of processing the individual data item identified by the message, wherein the result of processing the individual data item identified by the message is based at least in part on processing the state of the message enqueued within the message flow; and updating the state information based at least in part on the response, wherein The poller means is configured to include the updated status information in the next call.

2. The system of claim 1 , wherein the on-demand code execution system is further configured to implement multiple executions of a map task that enqueues the individual messages within the message flow as a map phase of a map-reduce programming model, and wherein the user-specified code implements the reduce phase of the map-reduce programming model.

3. A system as claimed in claim 1, wherein the message stream is divided into multiple message slices, wherein individual messages within the message stream are distributed among the multiple message slices according to individual data items within the data set, wherein the poller device corresponds to a first slice of the multiple slices from which the group of messages is retrieved, and wherein the status information represents a status of processing messages within the first slice.

4. The system of claim 3, further comprising a second poller device configured with computer executable instructions to: Initializing second state information representing a state of processing a message enqueued in a second message fragment among the plurality of fragments; retrieving a second set of messages from the second message fragment; and For each message in the second set of messages: transmitting to the on-demand code execution system the state information representing the state of processing the message enqueued within the second message fragment and the call corresponding to the message for invoking the user-specified code to process the individual data item within the data set, wherein the call requests to invoke the user-specified code to process the individual data item identified by the message; receiving a response to the call, the response comprising a result of processing the individual data item identified by the message, wherein the result of processing the individual data item identified by the message is based at least in part on processing the state of the message enqueued within the second message fragment; and The state information representing the state of processing the message enqueued within the second message fragment is updated based at least in part on the response.

5. A computer-implemented method comprising: Initializing state information representing a state of processing messages enqueued within a message stream, wherein individual messages within the message stream correspond to calls to execute user-specified code on an on-demand code execution system to process individual data items within a data set; retrieving a set of messages from the message stream; as well as For each message in the set of messages: transmitting to the on-demand code execution system the state information representing the state of processing a message enqueued within the message flow and a call corresponding to the message for executing user-specified code on the on-demand code execution system to process an individual data item within a data set, wherein the call requests to invoke the user-specified code to process the individual data item identified by the message; receiving a response to the call, the response comprising a result of processing the individual data item identified by the message, wherein the result of processing the individual data item identified by the message is based at least in part on processing the state of the message enqueued within the message flow; and The state information is updated based at least in part on the response, wherein the updated state information is included in a next call.

6. The computer-implemented method of claim 5, wherein initializing the state information comprises: retrieving one or more past instances of state information from a data storage device; and initializing the state information based at least in part on past instances of the one or more state information.

7. The computer-implemented method of claim 6, wherein the message stream is divided into a plurality of message slices, and wherein the one or more past instances of state information include state information from a failed device.

8. A computer-implemented method as described in claim 6, wherein the message stream is divided into multiple message shards, wherein the group of messages is retrieved from a first shard of the multiple message shards, and wherein the one or more past instances of state information include state information from a device that processes messages enqueued in one or more parent shards of the first shard.

9. The computer-implemented method of claim 8, wherein the one or more mother shards include at least two mother shards, and wherein the computer-implemented method further comprises: Deduplication is performed on the status information of the at least two parent shards.

10. A computer-implemented method as described in claim 9, wherein the state information of the at least two mother shards indicates, for an individual data item in the state information of the at least two mother shards, a last position in the message flow at which a value of the individual data item was modified.

11. The computer-implemented method of claim 5, further comprising: The status information is written to the data storage device after receiving a response to each call corresponding to each message in the set of messages.

12. The computer-implemented method of claim 11, further comprising: A failure in a device implementing at least a portion of the computer-implemented method is detected, and the state information is retrieved from the data storage device to resume implementation of the at least a portion of the computer-implemented method.

13. The computer-implemented method of claim 5, wherein transmitting the state information and the call to the on-demand code execution system comprises: A synchronous Hypertext Transfer Protocol (HTTP) request is transmitted to the on-demand code execution system.

14. The computer-implemented method of claim 13, wherein the response is an HTTP response to the HTTP request.

15. The computer-implemented method of claim 5, wherein the user-specified code executes statelessly on the on-demand code execution system without reference to state information maintained on the on-demand code execution system independent of a call to execute the user-specified code.

Images