Work allocation service

Abstract

The present disclosure provides a work allocation service that is a multi-zone, reliable service for dynamically fragmenting key ranges. The work distribution service provides exclusive ownership with rental right, load balancing, and routing information for owner discovery. By using a work allocation service, the service can easily achieve horizontal extension by fragmenting its workload.

Description

[0001] Prior Art

[0002] This application is a continuation of U.S. patent application No. 18 / 083,340, filed on December 16, 2022, the disclosure of which is hereby incorporated by reference herein. Background Art

[0003] Horizontal scaling refers to adding nodes or machines to a computing infrastructure to meet growing workload demands. For example, if the traffic to an application hosted on a first server exceeds the capacity of the first server, scaling can be achieved by adding a second server. Horizontal scaling is often achieved through sharding, which divides and stores data in a set of databases. However, sharding is not always easy to implement or may not be possible for some infrastructures. Summary of the invention

[0004] The present disclosure describes a work distribution service that implements sharding-as-a-service to support horizontal expansion. The work distribution service provides integration, flexibility and exclusivity. In terms of integration, the work distribution service simplifies multiple services so that these services no longer need to conduct leader elections. In terms of flexibility, the work distribution service makes advanced features such as routing information and dynamic re-sharding generally available to any service. Automatic re-sharding and rebalancing to adapt to automatic expansion services have been significantly improved over static solutions. In terms of exclusivity, the work distribution service provides exclusivity for task allocation, that is, tasks are only allocated to a single consumer, and the work distribution service does not allow duplicate allocation. In this regard, the integrated service can implement leader election. In addition to these advantages, new services can be implemented at low cost.

[0005] One aspect of the present disclosure provides a method for distributing workloads among multiple instances. The method may include: using one or more processors to receive one or more domain registrations from one or more consumers, each domain registration is used to execute an application in one of multiple instances in a region; using one or more processors to assign a key range associated with a distributed database to each instance, each key range defining a shard; using one or more processors to generate a routing table that maps each instance to its assigned key range; and publishing the routing table to multiple instances. Each instance can be executed in one region in a multi-region computing environment. Distributing the key ranges can include distributing a full set of available keys associated with the distributed database to instances in each region.

[0006] According to some embodiments, the method may further include determining a workload of each of the plurality of instances. Determining the workload may include polling the plurality of instances to obtain a number of active jobs in each instance, and / or monitoring performance and / or failures of the plurality of instances.

[0007] According to some embodiments, the method may further include determining, by one or more processors, whether to perform resharding, and when it is determined to perform resharding, redistributing the key range of the distributed database among the multiple instances, and updating the routing table to include an updated mapping of the multiple instances and the allocated key ranges. Determining whether to perform resharding may be based on determining whether instances are being added or deleted in one or more regions. Determining whether to perform resharding may also be based on the workload of one or more instances.

[0008] Another aspect of the present disclosure provides a system for distributing workloads among multiple instances. The system may include a memory and one or more processors configured to receive one or more domain registrations from one or more consumers, each domain registration for executing an application on one of multiple instances in a region, assigning a key range associated with a distributed database to each of the multiple instances, each key range defining a shard, generating a routing table that maps the multiple instances to their assigned key ranges, and publishing the routing table to the multiple instances.

[0009] The one or more processors may include one or more computing nodes including a leader and one or more coordinators. In addition, one or more agents may be configured to route communications between the multiple instances and the one or more coordinators.

[0010] According to some embodiments, each of the plurality of instances may be executed in a region of the multi-region computing environment. Distributing the key ranges may include distributing all available key sets associated with the distributed database to the plurality of instances in each region.

[0011] According to some embodiments, the one or more processors may be further configured to identify a workload of each of the plurality of instances, identifying the workload includes polling or monitoring the plurality of instances.

[0012] According to some embodiments, one or more processors may be further configured to determine whether to perform resharding, and when it is determined to perform resharding, redistribute the keys of the distributed database among multiple instances, and update the routing table to include an updated mapping of multiple instances and the allocated key ranges. Determining whether to perform resharding may be based on determining whether instances are being added or deleted in one or more regions. Determining whether to perform resharding may also be based on the workload of one or more instances.

[0013] Another aspect of the present disclosure provides a non-volatile computer-readable storage medium storing instructions executable by one or more processors for performing a method for distributing a workload among multiple instances, the method comprising receiving one or more domain registrations from one or more consumers, each domain registration being used to execute an application on one of multiple instances in a region, assigning a key range associated with a distributed database to each of the multiple instances, each key range defining a shard, generating a routing table that maps multiple instances to their assigned key ranges, and publishing the routing table to the multiple instances. The method may also include determining whether to perform resharding, and when it is determined that resharding is to be performed, reallocating the key range of the distributed database among the multiple instances, and updating the routing table to include an updated mapping of the multiple instances and the assigned key ranges. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a block diagram illustrating an architecture of a work distribution service node embodiment according to aspects of the present disclosure.

[0015] Figure 2 is a block diagram illustrating a multi-region deployment of a work assignment service embodiment according to aspects of the present disclosure.

[0016] Figure 3 is a block diagram illustrating an embodiment computing device for a work assignment service according to aspects of the present disclosure.

[0017] Figure 4-6 is a block diagram illustrating the operation of an embodiment of a work assignment service according to aspects of the present disclosure.

[0018] Figure 7 is a flow chart illustrating a sharding embodiment method according to aspects of the present disclosure.

[0019] Figure 8 is a flow chart illustrating an embodiment method of resharding according to aspects of the present disclosure. DETAILED DESCRIPTION

[0020] The present disclosure provides a work distribution service. The work distribution service is a multi-region service that uses a distributed database and key-value storage to define shard domains and perform leader elections. The work distribution service supports sharding as a service, enabling horizontal scaling even if the infrastructure does not provide sharding capabilities. The work distribution service may include a streaming remote procedure call (RPC) application programming interface (API) for shard ownership, with exclusive ownership of shards (implemented by leasing), load balancing of shards, and routing information for discovering owner shards. Load balancing of shards may include load balancing between clients. It may include a leader election API. It may support dynamic modification of work distribution, including moving work out of failed regions and redirecting work from failed deployments.

[0021] For multi-tenant environments, the work distribution service uses tenants and domains as namespaces. A tenant can be a service or collection of services of an organization. For example, a tenant of an online ordering and delivery platform can be a cloud printing service that manages receiving orders from remote client computing devices and printing the orders on a second computing device within the organization. A tenant can be composed of one or more domains that act as independent shard spaces. In some embodiments, a tenant can be a regional location of a server that provides a cloud service.

[0022] A domain can be a logical organization of connected network physical resources and users. A domain can include servers, printers, and other devices. It can also provide authorization and authentication mechanisms for users accessing the network, and administrators can control which resources users can access. Consumers register for a specific domain and receive shard allocations for that domain. Domains also enable users on a network to communicate.

[0023] Each domain can span multiple zones, each of which is a specific geographic location where resources (such as servers, printers, other computing devices, etc.) run. Each zone spans a universally unique identifier (UUID) range. The unit of allocation is a shard, which is a tuple that identifies a half-open UUID range for a domain and zone: (tenant, domain, zone, start, end)

[0024] UUID key ranges offer many advantages over discrete shard spaces such as integers. Examples of these advantages include support for splitting key ranges to arbitrary depths during resharding, support for automatic expansion of the number of shards, and natural avoidance of hotspots.

[0025] For a given domain, the work distribution service ensures that all shards for all regions of that domain are assigned to some consumer of that domain. It also provides the full list of assignments and receiver metadata to each connected receiver so that they can potentially communicate directly. All ranges for any specific key (tenant, domain, region, UUID) can be easily found.

[0026] The work distribution service may try to find a balanced distribution scheme taking into account zone affinity so that if the use case is related to distributed databases and key-value stores, the input / output (I / O) operations will be local to the zone if the receiver exists in that zone. This behavior can be configured for special domains such as leader election.

[0027] Applications utilizing the work distribution service may include, for example, a cloud printing service where a server monitors a group of printers. The monitoring server may periodically scan key ranges in a distributed database and key-value store for updates. In some embodiments, a minimum number of shards per region may be specified to ensure that scanning of the distributed database is feasible.

[0028] The work distribution service can provide client-side leader election functionality using domains with a single leader shard. In addition, the work distribution service can provide multiple application programming interfaces (APIs) for interacting with work balancing. For example, the work distribution service can be used to direct work away from a specific region, or to pause a domain entirely. Thus, in the event of a failure, work can be quickly directed away from the failed region or deployment.

[0029] Figure 1 An embodiment of a work distribution service running in multiple regions is shown. In this embodiment, the work distribution service 100 is deployed in region 1, region 2, and region 3. Each region contains one or more clients 160, 170, 180. Each client may include, for example, an instance, such as a server running an application. The application may run in a container, a virtual machine, or a similar environment. To simplify the description, a client may include an application running in one or more containers. According to some embodiments, each container may be grouped. For example, these groups may be one or more Pods (such as in Kubernetes) that share computing resources with the region. Each region may have the same or different number of clients, containers, or Pods. For example, as shown in the figure, region 1 contains three clients 160, while region 2 and region 3 each contain two clients 170, 180. It should be understood that these are merely embodiments and the number of clients in each region may vary. Similarly, although Figure 1 Three regions are shown in Figure 1, but it is important to understand that the work distribution service can be deployed in more or fewer regions.

[0030] The clients 160, 170, 180 of each region can receive the regional local shard allocation. The shard can be defined according to the key range, and the allocation can cover the full range of keys available in the distributed database and the key-value storage 150. For example, as shown in the figure, the full range of keys is represented as aaaaa... to fffff..., although these are merely examples of key range identifiers, they can also be represented using numerical values, alphanumeric values ​​or other formats. Therefore, each of the clients 160 in region 1 can be responsible for a portion of such keys. For example, the first client 160 can be responsible for aaaaa to bbbbb. The second client 160 can be responsible for bbbbbc to eeeee. The third client 160 can be responsible for eeeef to fffff.

[0031] Shard allocation can include balancing by allocating an approximately equal number of shards to each instance, while taking zone affinity into account. For example, if a given zone has 10 shards and 3 clients, two clients can each be assigned 3 shards, while the third client can be assigned 4 shards.

[0032] The routing table within each region can indicate the key range to which each client is assigned. Since the key range defines the shards, the routing table maps the shards to the clients in each region. The routing table can be stored, for example, in each client or Pod, or in any one or more resources that communicate with other resources. In this regard, each Pod knows which shards are assigned to which clients. Therefore, when an inbound request is received for a key in a shard, the Pod knows which client owns the shard and will handle it.

[0033] Sometimes, resharding is appropriate, such as if the shard allocation becomes unbalanced. For example, if clients are added or removed, the shard allocation may become unbalanced. Returning to the previous example of 3 clients and 10 shards, if a fourth client is added, it will have no shards and therefore appear unbalanced compared to the other clients that have been assigned 3 or 4 shards. Therefore, the work distribution service can shift allocations to redistribute the shards. For example, two of the four clients can each be reassigned two shards, while the other two clients can each be assigned three shards. This redistribution of shards can be achieved by reassigning key ranges. The redistribution can be done gradually or all at once.

[0034] According to some embodiments, the work distribution service may determine that resharding or reallocating shards can improve traffic to clients 160-180. For example, each pod may include a performance / fault monitoring mechanism for determining traffic metrics, such as task execution time, the number of tasks waiting in a queue, the number of task failures, etc. The pod may report traffic metrics to the work distribution service 100, or generate a notification for the work distribution service to otherwise indicate the performance situation or actions that the work distribution service should take. According to other embodiments, the work distribution service itself may include a monitoring mechanism for determining when resharding would be beneficial.

[0035] According to other embodiments, the workload of a service may increase over time, and the service may add more nodes to accommodate the workload. The work allocation service may redistribute shards between the added nodes, or add shards. As just one example, if a new service has 4 nodes and uses a work allocation service with 20 shards, each node may be assigned 5 shards. If over time, the service load increases, it can be expanded to 20 nodes. The work allocation service may redistribute the allocated shards so that each node is now assigned one shard. Even at a later point in time, the load of the service may increase further, and the service may be expanded to 30 nodes. In the case of 30 nodes, the 20 shards initially allocated to the service may no longer be sufficient, because 10 nodes will be idle. Therefore, the work allocation service can increase the number of shards without downtime, to 30, 50, 100, or any other number of shards. If the workload and / or the number of nodes later decreases, the work allocation service can also reduce the number of shards accordingly.

[0036] According to some embodiments, the work assignment service 100 may redefine the shards using different key ranges. For example, the work assignment service may assign a larger key range to the first client in the region and a smaller key range to the second client in the region. The work assignment service 100 may update the routing table accordingly and broadcast the updated routing table to the clients 160 in region 1.

[0037] By defining shards using key ranges, key ranges can be split arbitrarily when resharding. In addition, the number of shards can be expanded automatically. In addition, if UUIDs are used as identifiers, hot spots can be naturally avoided because the generation of such identifiers is random and distributed across the entire UUID space. Therefore, when clients populate their databases with UUID identifiers, no single shard served by the work distribution service will have a disproportionately high number of keys than other shards.

[0038] Figure 2An architecture of an embodiment of a work distribution service 100 is shown. The work distribution service 100 can be executed in a distributed system composed of nodes, which include one or more agents 110 and one or more coordinators 120. For example, each node can be a server running the work distribution service, and a coordinator / leader / worker process runs on the node. Although a specific number of components are shown in the figure, the number of components in an actual implementation may vary. A leader 130 can be elected from these processes, and the leader communicates with a distributed database 150. The work distribution service 100 can also include one or more workers (not shown). A bidirectional streaming API allows clients to connect to the work distribution service and receive exclusive shard allocations.

[0039] The leader 130 may be responsible for domain allocation and load distribution among all coordinators 120. The leader 130 may also be responsible for publishing a domain allocation table to facilitate direct communication between nodes and coordinators. For example, the domain allocation table may publish information for establishing a multiplexed bidirectional stream connection, such as using remote procedure calls (RPC), gRPC, HTTP, etc. This information may include the address, routing, or other connection information of a particular coordinator 120.

[0040] The leader 130 can be dynamically selected from multiple processes. For example, the leader 130 can be selected based on availability, rotation, randomness, self-recommendation confirmed by nodes, or any other leader election method. The coordinator 120 will communicate with the leader 130 through heartbeats to renew the leader's ownership lease. If the leader crashes, the new leader will take over and collect the existing allocations in the coordinator. If the coordinator 122, 124 crashes, the leader 130 will reassign the coordinator after its lease expires.

[0041] Each coordinator 122, 124 is responsible for a separate tenant. For the corresponding tenant, the coordinator can be responsible for shard allocation (including leasing and revocation) and load balancing. Shards can be defined based on regions, and each shard can include a series of addresses, such as UUIDs. Each domain participant or consumer connects to a specific coordinator responsible for its address through a proxy's bidirectional stream connection. The domain participant communicates heartbeats with the coordinator to maintain its lease. In some embodiments, if a domain participant wants to disconnect gracefully, it may give up the shard. This sub-lease semantics can ensure the exclusivity of the domain participant.

[0042] The architecture may also include one or more workers (not shown). Each worker connects to the leader and manages the lifecycle of one or more coordinators 120 according to the leader's instructions. The worker may update the allocation table used by the proxy to handle incoming client connections.

[0043] Each proxy 112, 114 may be responsible for forwarding messages from a recipient (e.g., a domain participant) to the coordinator responsible for that domain. Examples of such messages may include allocation requests, heartbeat messages, abandonment messages, etc. from a given domain. The proxy may maintain long-lived connections between nodes. Examples of such long-lived connections may include RPC streams or gRPC streams.

[0044] Distributed database 150 may be, for example, a geographically distributed Structured Query Language (SQL) database. Distributed database 150 may include a key-value store.

[0045] Although the above embodiment shows a specific number of agents, coordinators and other components, it should be understood that this is only one embodiment. Other embodiments may include various numbers of each type of component. In addition, the number of each component does not necessarily correspond to the number of other components. For example, a system may include more agents than coordinators, or more coordinators than agents, and so on.

[0046] Figure 3 An embodiment computing device 300 is shown, which can be configured as a leader or coordinator in a work allocation service. The computing device 300 includes one or more processors 320, a memory 330, and other components typically found in a computing device. The memory 330 can store information accessible to the one or more processors 320, including instructions 334 executable by the one or more processors 320.

[0047] The memory 330 may also include data 332 that may be retrieved, manipulated, or stored by the processor. The memory may be of any non-transitory type capable of storing information accessible by the processor, such as a hard disk, a memory card, a read-only memory (ROM), a random access memory (RAM), a digital video disk (DVD), a compact disk-read-only memory (CD-ROM), a writable memory, and a read-only memory.

[0048] Instructions 334 may be any set of instructions that may be directly executed by one or more processors (e.g., machine code), or indirectly executed (e.g., scripts). In this regard, the terms "instructions," "application," "steps," and "program" may be used interchangeably herein. Instructions may be stored in an object code format for direct processing by a processor, or in any other computing device language, including a collection of scripts or independent source code modules that are interpreted or compiled in advance when needed. Functions, methods, and routines in instructions are explained in more detail below.

[0049] Data 332 may be retrieved, stored, or modified by one or more processors 320 in accordance with instructions 334. For example, although the subject matter described herein is not limited to any particular data structure, the data may be stored in a computer register, as a relational database with tables having many different fields and records, or as an extensible markup language (XML) document. The data may also be formatted in any computing device readable format, such as, but not limited to, binary values, ASCII, or Unicode. In addition, the data may include any information sufficient to identify the relevant information, such as numbers, descriptive text, proprietary codes, pointers, references to data stored in other storage (such as other network locations), or information used by a function to calculate the relevant data.

[0050] The one or more processors 320 may be any conventional processor, such as a commercially available central processing unit (CPU). Alternatively, the processor may be a dedicated component, such as an application specific integrated circuit ("ASIC") or other hardware-based processor. Although not required, one or more of the computing devices 300 may include dedicated hardware components that perform specific computing processes.

[0051] The computing device 300 may include a communication interface 340 that enables the computing device 300 to communicate with other components (such as distributed databases, key-value stores, consumer systems, etc.). The computing device can communicate with other devices over a network and using various protocols and systems, so the network can be part of the Internet, the World Wide Web, a specific intranet, a wide area network, or a local area network. The network can use standard communication protocols, such as Ethernet, WiFi, HTTP, protocols described in IEEE 302.33, cellular technology (such as GSM, CDMA, UMTS, EV-DO, WiMAX, LTE, etc.) protocols (these protocols are proprietary to one or more companies), and various combinations of the above protocols. Although certain advantages can be obtained when transmitting or receiving information as described above, other aspects of the subject matter described herein are not limited to any particular information transmission method.

[0052] although Figure 3The processor, memory and other elements are functionally shown to be located in the same module, but in fact, the processor, computer, computing device or memory may include multiple processors, computers, computing devices or memories, which may or may not be stored in the same physical housing. Therefore, references to processors, computers, computing devices or memories should be understood to include references to sets of one or more processors, computers, computing devices or memories, which may or may not run in parallel. In addition, although some functions described below are indicated to be executed on a single computing device with a single processor, various aspects of the subject matter described herein can be implemented by multiple computing devices, for example, in the "cloud". Similarly, memory components located in different locations can store different parts of instructions 334 and together constitute a medium for storing instructions. Various operations described herein as being performed by a computing device can be performed by a virtual machine. For example, instructions 334 may be specific to a first type of server, but related operations may be performed by a second type of server running a hypervisor that simulates the first type of server. Operations can also be performed by containers, for example, a computing environment that does not rely on an operating system bound to a specific type of hardware.

[0053] Figure 4 An embodiment of the work allocation service 100 in operation is shown. At startup, the work allocation service 100 can identify two instances 460, 470 for execution. For example, the work allocation service 100 can receive communications from the instances 460, 470. These communications can include, for example, requests for shard allocations. According to some embodiments, the communications can be received at an agent of the work allocation service 100 and forwarded to the coordinator based on the instance that received the communication.

[0054] The work allocation service 100 can determine shard allocations for the instances 460, 470 based on communications received from the instances 460, 470. Initially, the shard allocations can be based on the number of instances. For example, if there are two instances, the work allocation service 100 can evenly distribute the available shards between the two instances. In other embodiments, the shard allocations can be based on the respective workloads of the instances 460, 470. For example, an instance with a larger workload can be allocated more shards than an instance with a smaller workload.

[0055] Shard allocation can include allocating key ranges or unique identifier ranges corresponding to resources in the distributed database 150. In this embodiment, the entire identifier range, such as the entire UUID space, is abstracted as AC. In the illustrated embodiment, the first instance 460 is assigned a key range AB, and the second instance 470 is assigned a key range BC. Key range AB can include, for example, resources A1, A2, A3, A4...An. Key range BC can include, for example, B1, B2, B3...Bn. Resources can include, for example, computing devices, printers, etc.

[0056] The work distribution service 100 may publish a routing table 490 indicating the key range assignments for each instance 460, 470. According to some embodiments, each instance 460, 470 may store the routing table 490, such as in a cache.

[0057] Once the allocation is complete, the instances 460, 470 can obtain shards from the distributed database 150. For example, the first instance 460 can send a request or command for the key range AB, while the second instance 470 can send a request or command for the key range BC. Figure 4 In the embodiment of FIG. 4 , in the first example 460, resource A1 has 5 jobs to perform and resource A2 has 8 jobs to perform. In the second example 470, resource B2 has 10 jobs to perform and B3 has 12 jobs to perform. It should be understood that these are merely examples provided for illustrative purposes.

[0058] Figure 5 An embodiment of using a work allocation service to provide data in a sliced ​​form is shown. In this diagram, different instances have different pieces of data and respond to requests from other services or users about this data. For example, any instance may receive a communication asking how many active jobs are running on a particular resource. The request can come from outside the work allocation service 100, such as from another service that wants to know resource information. If the resource being queried is assigned to the instance receiving the request, the instance can respond to the request. For example, if the first instance 460 receives a request asking how many active jobs are on resource A1, the first instance 460 can respond to the request, indicating that there are 5 active jobs on A1. If the resource being queried is assigned to an instance different from the instance receiving the request, the instance receiving the request can forward the request. For example, if the second instance 470 receives a request asking how many jobs are being executed on resource A1, the second instance 470 can forward the request to the first instance 460 to which resource A1 is assigned.

[0059] At times, the work distribution service may determine whether a resharding should be performed. This determination may be based on, for example, the number of jobs being executed in each instance. According to other embodiments, it may be determined based on the performance or failure of a resource. According to other embodiments, when instances are added or removed, a resharding may be performed to redistribute the full key range among the updated number of instances.

[0060] Figure 6 An embodiment of adding instances is shown. In this embodiment, a third instance 480 is added to a region containing a first instance 460 and a second instance 470. Therefore, the work assignment service 100 can decide to reshard. The work assignment service 100 can therefore redistribute the available keys between the first, second, and third instances 460-480. In the illustrated embodiment, the work assignment service 100 assigns the key range A-A5 to the first instance 460, the key range A6-B2 to the second instance 470, and the key range B3-C to the new third instance 480. The work assignment service 100 can update the routing table with the updated key range and publish the updated routing table to the instances 460-480.

[0061] Figure 7-8 Embodiment methods including work assignment service operations are shown. Although the operations are described in a particular order, the order may be changed, and / or the operations may be performed simultaneously. In addition, operations may be added or omitted.

[0062] Figure 7 A method 700 for assigning shards to instances is shown. In step 710, a work assignment service receives a domain registration from a consumer and sends assignment information to the consumer. For example, an application connects to a tenant as a consumer and can register one or more domains. Each application can run in an instance. Each instance can execute in one of a plurality of regions.

[0063] In step 720, the work distribution service determines the workload of each instance. Initially, this can be an estimate based on, for example, the type of application, the number of instances executing in each region, or other information related to the instances. In other embodiments, it can be initially assumed that the workload of all instances is approximately equal. In other embodiments, the work distribution service can determine the number of requests or the number of active jobs for each instance.

[0064] In step 730, the work distribution service assigns key ranges compatible with efficient operation of the distributed database and key-value store to each instance, each key range defining a shard. A key range can be a range of identifiers (such as UUIDs) corresponding to resources of the distributed database. The assignment of key ranges can be based on a workload determined for each instance. For example, a region or instance with a larger workload can be assigned a larger key range than other regions or instances with a smaller workload.

[0065] In step 740, the work assignment service generates a routing table that maps instances to assigned key ranges. The routing table can be published to each instance. Based on the routing table, each instance can determine the owner of the assigned key range.

[0066] Figure 8 A resharding method 800 is shown that can be performed periodically while a work distribution service is running and instances are executing in a multi-region environment.

[0067] In step 810, the work distribution service monitors the instances executing in each region. For example, the work distribution service may use performance and / or fault monitoring mechanisms. As another embodiment, the work distribution service may track the activity level of each instance, such as by measuring the messages received from each instance. As another example, the work distribution service may periodically poll the instances to obtain workload information. For example, the work distribution service may ask each instance how many active tasks they are processing. Such polling may be performed every minute, hour, day, week, or any other frequency. Monitoring also includes determining whether instances are being added or deleted. For example, the work distribution service may receive a request for a new instance to allocate a shard to it.

[0068] In step 820, it is determined whether to reshard based on the monitoring results, for example, to split the entire space of keys or identifiers (e.g., the entire UUID space) into a different number of shards. If a new instance is being added or an instance is being deleted, a decision to reshard may be made. As another embodiment, a decision to reshard may also be made if a given workload is overloaded or fails, or if resources are underutilized. If it is determined not to reshard, method 800 will continue to monitor the instance.

[0069] If a resharding is determined, then in step 830, the work distribution service reallocates key ranges between instances to achieve the resharding. If workload or performance issues are detected, or instances are added or removed, the key ranges allocated in one region can be changed. In other embodiments, the reallocation can change the key ranges allocated across multiple regions. By reallocating the key ranges, the work distribution service reallocates the resources allocated to each instance.

[0070] In step 840, the mapping table is updated according to the resharding result. The updated mapping table reflects the updated assignment of key ranges to instances. The updated mapping table can be published to each instance.

[0071] The work allocation service described herein can be used in a variety of applications. As an example only, it can be used in a cloud printing application that uses sharding for printer monitoring. For example, the work allocation service can be used to dynamically assign ownership of polling-based printers to discoverable nodes, and the protocol header can forward requests. All printer protocols can then be implemented as long-running connection protocols with in-memory state.

[0072] Another embodiment implementation is a queued message service where shards can be used to distribute region / key pair queues between workers within a job service so that no single worker needs to process all queues at once. A job distribution service can be used to maintain and propagate a shard ownership table to coordinate between jobs, allowing for simple and efficient communication.

[0073] Another embodiment implementation includes a keyed event queue where shards are used to distribute the queue among workers in the service to monitor the queue and whether messages have expired.

[0074] Although several example implementations are described, the work assignment service may be implemented in an endless variety of applications.

[0075] Unless otherwise indicated, the above alternative embodiments are not mutually exclusive, but can be implemented in various combinations to obtain unique advantages. Since these and other variations and combinations of the above features can be used without departing from the scope of the subject matter defined by the claims, the above description of the embodiments should be regarded as illustrative rather than limiting the scope of the subject matter defined by the claims. In addition, the embodiments described herein and the use of phrases such as "for example", "including", etc. should not be interpreted as limiting the subject matter of the claims to specific embodiments; on the contrary, the embodiments are intended to illustrate only one of many possible embodiments. In addition, the same reference numerals in different figures may identify the same or similar elements.

Claims

1. A method for distributing a workload among a plurality of instances, comprising: receiving, using one or more processors, one or more domain registrations from one or more consumers, each domain registration for executing an application in one of a plurality of instances within a region; Using one or more processors, assigning each instance a key range associated with the distributed database, each key range defining a shard; Generates a routing table using one or more processors that maps each instance to its assigned key range; as well as Publish routing tables to multiple instances.

2. The method according to claim 1, characterized in that Each instance executes in one zone of the multi-zone computing environment.

3. The method according to claim 2, characterized in that Distributing a key range involves assigning the entire set of available keys associated with the distributed database to the instances in each region.

4. The method according to claim 1, characterized in that Also included is identifying the workload for each of the multiple instances.

5. The method according to claim 4, characterized in that Identifying the workload involves polling multiple instances to obtain the number of active jobs in each instance.

6. The method according to claim 4, characterized in that Identifying workloads includes monitoring multiple instances for performance and / or failures.

7. The method according to claim 1, characterized in that Also includes: One or more processors determine whether to reshard; When resharding is determined to be necessary, redistribute the key range of the distributed database to multiple instances; And update the routing table to include the updated mapping of multiple instances and assigned key ranges.

8. The method according to claim 7, characterized in that Determining whether to reshard is based on determining whether instances are being added or removed in one or more zones.

9. The method according to claim 7, characterized in that: The determination of whether to reshard is based on the workload of one or more of the multiple instances.

10. A system for distributing workload among multiple instances, comprising: Memory; and one or more processors configured to: receiving one or more domain registrations from one or more consumers, each domain registration for executing an application in one of a plurality of instances within a region; Assigning a key range associated with the distributed database to each of the multiple instances, each key range defining a shard; Generate a routing table that maps each of the multiple instances to its assigned key range; as well as Publish routing tables to multiple instances.

11. The system according to claim 10, characterized in that The one or more processors include: a plurality of computing nodes including a leader and one or more coordinators.

12. The system according to claim 11, characterized in that Further included is one or more proxies configured to route communications between the plurality of instances and the one or more coordinators.

13. The system according to claim 10, characterized in that Each of the multiple instances executes in a zone of the multi-zone computing environment.

14. The system according to claim 13, characterized in that Distributing the key range involves distributing the entire set of available keys associated with the distributed database to multiple instances in each region.

15. The system according to claim 10, characterized in that The one or more processors are further configured to identify a workload of each of the plurality of instances, identifying the workload includes polling the plurality of instances or monitoring the plurality of instances.

16. The system according to claim 10, characterized in that The one or more processors are further configured to: Determine whether to reshard; When resharding is determined to be necessary, redistribute the keys of the distributed database to multiple instances; And update the routing table to include the updated mapping of multiple instances and assigned key ranges.

17. The system according to claim 16, characterized in that Determining whether to reshard is based on determining whether instances are being added or removed in one or more zones.

18. The system according to claim 16, characterized in that The determination of whether to reshard is based on the workload of one or more of the multiple instances.

19. A non-volatile computer-readable storage medium storing instructions executable by one or more processors for performing a method for distributing a workload among a plurality of instances, the method comprising: receiving one or more domain registrations from one or more consumers, each domain registration for executing an application in one of a plurality of instances within a region; Assigning a key range associated with the distributed database to each of the multiple instances, each key range defining a shard; Generate a routing table that maps each of the multiple instances to its assigned key range; as well as Publish routing tables to multiple instances.

20. The non-volatile computer-readable storage medium of claim 19, wherein: The method further includes: Determine whether to reshard; Redistribute the key ranges of the distributed database across multiple instances when resharding is determined to be necessary; and Update the routing table to include the updated mapping of multiple instances and assigned key ranges.

Images