Distributing rules across rule engine instances

Abstract

A method for distributing rules across rule engine instances is provided. The method includes determining a rule load for each rule set of a plurality of rule sets, each rule set being associated with a tenant among a plurality of tenants hosted on a multi-tenant system. The method includes aggregating the rule load for each rule set into an overall rule load. The method further includes distributing the plurality of rule sets across a set of rule engine instances such that a substantially equal percentage of the overall rule load is assigned to each rule engine instance of the set of rule engine instances.

Description

Technical Field

[0001] The present disclosure generally relates to the field of rule services, and more particularly to distributing rule execution across multiple independent instances of a rule engine.

Background Art

[0002] Modern rule engines maintain an in-memory model of a rule set and facilitate fast evaluation of rules. In a modern microservices environment, rule services typically deploy multiple instances of a rule engine simultaneously to further facilitate fast evaluation of rules.

Summary of the Invention

[0003] The present disclosure includes a method, a computer program product, and a system for distributing rules across independent instances of a rule engine. The method includes determining a rule load for each of a plurality of rule sets, where each rule set is associated with a tenant among a plurality of tenants hosted on a multi-tenant system. The method further includes integrating the rule loads into an overall rule load. The method further includes distributing the plurality of rule sets across a set of rule engine instances such that approximately the same proportion of the overall rule load is assigned to each rule engine instance of the set of rule engine instances.

[0004] The above summary of the invention is not intended to describe every embodiment or all implementations described in the present disclosure.

[0005] The drawings included in the present disclosure are incorporated into or form a part of the specification. These serve to illustrate embodiments of the present disclosure and to explain the principles of the disclosure in accordance with the description. The drawings illustrate only typical embodiments and do not limit the disclosure.

Brief Description of the Drawings

[0006]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

[0007] The embodiments described herein are subject to various changes and alternative forms. These details are shown by way of example in the drawings and are described in detail. However, it should be understood that the specific embodiments described are not to be treated in a limiting sense. Rather, the intention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the present invention.

[0008] Aspects of the present disclosure generally relate to the field of rule services, and more specifically to distributing the execution of rules across independent instances of a rule engine. The present disclosure is not necessarily limited to such applications, but various aspects of the present disclosure can be appreciated through discussion of various examples using this context.

[0009] A rule engine can be used to process a number of different types of requests and return an appropriate response. A rule engine can utilize a set of rules, each having a condition and an associated action, to process data and generate (or filter) a result (e.g., select an action based on the processed data). For example, a rule engine may be used as part of a search engine. The search engine performs a search for the particular information requested in a systematic manner. The results of the search are then returned to the requester. How and in what order the search results are presented, for example, is managed by a rule engine. More specifically, the rule engine applies a set of rules or multiple sets of rules to the results to determine how the results are presented to the requester. The set of rules or multiple sets of rules vary depending on the purpose of the entity generating the rules.

[0010] For example, in the case of an online retailer, it may be advantageous for the retailer to promote certain items for one of several reasons. For example, the item may have a limited shelf life, and the retailer may need to sell more of that item before it expires. Additionally, or alternatively, the retailer may receive certain benefits for selling items from a particular supplier, and thus may be motivated to sell more items from that source. Additionally, or alternatively, the retailer may wish to sell more of a particular item to create room for new or different items. In each of these exemplary situations, and in many other cases, it is advantageous for the retailer to be able to somewhat control how the list of search results is presented to consumers. For example, the retailer may have modified search results where a particular item is emphasized, shown first, or otherwise more readily brought to the consumer's attention. The retailer may exert such control via a set of rules or multiple sets of rules that a rules engine applies to the results of a consumer's search.

[0011] In a microservices environment, various services, referred to herein as microservices, are provided to tenants by a microservices provider. A tenant may be, for example, an online retailer. Microservices provided to a tenant may be, for example, obtaining search results and applying sets of rules. It is common for a single microservices provider to operate a multi-tenant system in which the microservices provider provides microservices to multiple tenants. Such a microservices provider may provide such services to tenants, for example, via a shared cloud system.

[0012] In the present disclosure, a microservice provider provides a service that applies a rule set to search results. Thus, the microservice provider maintains a rule execution microservice that executes rules about a tenant in response to a request. Each tenant within a multi-tenant system may have its own rules or rule set, which are provided to be applied during the execution of the microservice provider for the requested microservice by the microservice provider. Each rule set is stored in the memory of the microservice provider so that the microservice provider can quickly call and apply the rule set about the tenant in response to a request. Typically, in a multi-tenant system, to speed up the process of calling and applying different sets of rules for different tenants, the microservice provider concurrently executes multiple instances of a rule engine at the same time. Currently, each instance of the rule engine includes all the rules for all tenants. In this way, when multiple requests from tenants come in simultaneously or around that time, different requests can be sent to different instances of the rule engine to distribute the request load and return a response to the tenant as quickly as possible.

[0013] However, one drawback of this current system is the large number of duplicates of the same rules that should be stored in multiple locations. More specifically, if a multi-tenant microservices provider has 5,000 tenants and each tenant has an average of 100 rules to be applied, then 100 rules for 5,000 tenants need to be loaded into all instances of the microservices provider's rule engine. In other words, 500,000 rules need to be loaded into all instances of the rule engine. In some cases, the number of rules becomes high enough that the cost borne by a system that loads so many rules, such as memory consumption, exceeds the benefits provided by loading duplicates of each rule for each tenant on each instance of the rule engine.

[0014] One way to manage a large number of rules is to evenly divide the tenants among the instances and load only the rules for the tenants assigned to each instance onto each instance. For example, tenants 1 to 100 might be assigned to instance A of the rule engine, tenants 101 to 200 might be assigned to instance B of the rule engine, and tenants 201 to 300 might be assigned to instance C of the rule engine. Then, instance A stores and applies only the rules associated with tenants 1 to 100, instance B stores and applies only the rules associated with tenants 101 to 200, and instance C stores and applies only the rules associated with tenants 201 to 300. Thus, the rule-distributed microservice distributes requests from the microservices provider such that any incoming request from any of tenants 1 to 100 is routed to instance A, any incoming request from any of tenants 101 to 200 is routed to instance B, and any incoming request from any of tenants 201 to 300 is routed to instance C.

[0015] However, the execution of all rules does not require the same amount of resources. For example, executing a set of rules for some tenants may incur a greater actual execution time or load cost than executing a set of rules for other tenants. Therefore, even if a large number of tenants are evenly distributed across instances of the rule engine, the collective burden or load of all tenants is not necessarily evenly distributed across instances of the rule engine. Thus, dividing tenants evenly among instances may not be the most efficient or effective way to optimize the functionality and resources of the microservice provider.

[0016] Embodiments of the present disclosure can overcome the above and other problems by distributing the execution of rules across independent instances of a rule engine based on the load generated by each tenant. Thus, a microservice provider also maintains a rule distribution microservice that distributes rules, as described in more detail herein. The load generated by an associated tenant is referred to herein as the "rule load" generated by that particular tenant. The rule load is determined for each tenant by an algorithm that takes into account factors including, but not limited to, the number of rule evaluation requests by the tenant within a given time period, the average number of rule condition checks performed by the rule engine per request within this given time period, the average number of actions triggered by the rule engine per request within this given time period, the average rule evaluation context size per request within this given time period, the number of defined conditional facts per tenant, and a complexity metric value.

[0017] Once the rule loads for each tenant are known, the rule loads for all tenants may be integrated into the overall rule load. The tenants are then distributed across the instances of the rule engine such that each instance processes approximately the same proportion of the overall rule load. In other words, the tenants are distributed across the instances in such a way as to distribute the cumulative rule load of all tenants as evenly as possible. In this way, the functionality and resources of the microservice provider are optimized more efficiently and effectively.

[0018] It should be understood that the above advantages are exemplary advantages and should not be construed as limitations. Embodiments of the present disclosure may include all, some, or none of the above advantages while remaining within the spirit and scope of the present disclosure. Further, although the present disclosure discloses detailed applications of various embodiments in the context of a search engine, the embodiments may be directed to other applications of the rule engine, and it should be understood that the exemplary embodiments disclosed herein should not be construed as limitations.

[0019] Referring now to the drawings, FIG. 1 shows a block diagram of an exemplary rule execution system 100 according to an embodiment of the present disclosure. The exemplary rule execution system 100 includes a microservice provider 102, a rule distribution microservice 104, a plurality of rule engine instances 106A, 106B, 106C, 106D (collectively referred to as rule engine instances 106), a rule monitoring component 108, and a plurality of tenants T(n) (collectively referred to as tenants 110). In at least one embodiment of the present disclosure, the microservice provider 102 includes the rule distribution microservice 104. However, in other embodiments of the present disclosure, the microservice provider 102 and the rule distribution microservice 104 are separate and distinct components of the system 100.

[0020] Note that FIG. 1 is intended to show representative major components of an exemplary rule execution system 100. In some embodiments, however, the individual components may have more or less complexity than those represented in FIG. 1, there may be components other than or additional to those shown in FIG. 1, and the number, type, and configuration of such components may vary. For example, FIG. 1 shows a rule execution system 100 having four rule engine instances 106, but a suitable computing environment for implementing embodiments of the present disclosure may include any number of rule engine instances. Additionally, the various models, modules, systems, and components shown in FIG. 1, if they exist, may exist across multiple host devices and remote devices.

[0021] In some embodiments, the rule execution system 100 may be implemented within a cloud computing environment or using one or more cloud computing services. Consistent with various embodiments, a cloud computing environment may include a network-based distributed data processing system that provides one or more cloud computing services. Further, a cloud computing environment may be located in one or more data centers and may include a number of computers (e.g., hundreds or thousands of computers or more) configured to share resources via a network.

[0022] According to embodiments of the present disclosure, the microservice provider 102 provides microservices to a plurality of tenants. For purposes of illustration, in the exemplary embodiment shown in FIG. 1, the microservice provider 102 provides microservices to 4,000 tenants (shown in FIG. 1 as tenant T1, tenant T2, …, tenant T4000), and these tenants are collectively referred to as tenant 110. In alternative embodiments, the microservice provider 102 may provide more or fewer than 4,000 microservices. In the rules execution system 100 discussed herein, the microservice provider 102 operates at least in part as a rules engine and provides a rules execution service to each tenant 110. Thus, the microservice provider 102 comprises a set of tenant-specific rules from each of the tenants 110. Each set of tenant-specific rules may have a different number of rules, or may require a different amount of resources for the execution of these rules, or both.

[0023] To improve the efficiency of responding to rules execution requests received from tenant 110, the rules provided to the microservice provider 102 are distributed across the rules engine instances 106. Each of the rules engine instances 106 is at least a partial copy of a rules engine capable of storing, loading, and executing rules when receiving a rules execution request from a tenant. The distribution of rules is managed by the rules distribution microservice 104. In particular, the rules are distributed in such a way that all of the rules of a set of tenant-specific rules associated with a particular tenant 110 are assigned to a single rules engine instance 106. Each rules engine instance 106 is responsible for storing, loading, and executing the assigned rules.

[0024] According to an embodiment of the present disclosure, instead of all rules being assigned to each rule engine instance 106, a subset of the rule set is assigned to each rule engine instance 106. In particular, instead of the rule set for all tenants 110 being assigned to each rule engine instance 106, each rule engine instance 106 is assigned a rule set related to a subset of the tenants 110.

[0025] More specifically, referring to FIG. 2, how the rule set is assigned to the rule engine instance 106 is managed by the method 200. First, in operation 204, an in-memory rule evaluation model is generated for each tenant 110. The in-memory rule evaluation model may be referred to herein as a rule evaluation model, an in-memory model, or a rule model. In at least one embodiment of the present disclosure, the rule model is generated by the rule distribution microservice 104. In at least some embodiments of the present disclosure, each rule model specifies how the rule set is to be applied to a set of data for which it is provided.

[0026] In operation 208, a workload is calculated for each rule model. Since the rule model is generated for each tenant 110, the workload calculated for the rule model is also the workload of the associated tenant. The workload represents the impact or burden that executing the rule model places on the microservice engine. In this case, the microservice engine is the rule engine. The workload for each rule model is calculated based on many aspects of the rule model, such as the number of rules in the rule model, the number and complexity of the conditions associated with each rule, the expected average number of rule evaluation requests, and the complexity of the rule evaluation context.

[0027] According to at least one embodiment of the present disclosure, a specific algorithm for calculating the workload of the rule model is as follows: (Number 1) W(T) = r(t) * c(t) * a(t) * (x + s(t) / f)

[0028] In the algorithm, the variable W(T) represents the tenant's workload. The variable r(t) represents the number of rule evaluation requests received from the tenant by the rule engine within a given time period (t). According to at least one embodiment of the present disclosure, a given time period (t) may be, for example, 2 minutes. The variable c(t) represents the average number of rule conditions that the rule engine should consider to answer those requests. The variable a(t) represents the average number of actions triggered by the rule engine to answer these requests within a given time period (t). The variable s(t) represents the average rule evaluation context size of those requests within a given period. According to at least one embodiment of the present disclosure, the evaluation context may be, for example, a fact value. Thus, the average rule evaluation context size per request may be, for example, the number of fact values passed to the rule evaluation request within a given time period (t). The variable f represents the number of condition facts defined by the tenant. According to at least one embodiment of the present disclosure, the condition facts may include, without limitation, for example, user segments, geographical locations, referrers, types of client devices, genders, or ages. The variable x represents a configured value of complexity, which can be used to adjust the workload. According to at least one embodiment of the present disclosure, the default value of the variable x is 1.

[0029] According to at least one embodiment of the present disclosure, the rule monitoring component 108 is a tracking component that measures the rule load generated by a specific tenant on the rule engine. Thus, when the microservice system 100 is in operation, the rule monitoring component 108 receives data regarding: the number of rule evaluation requests per tenant within a predetermined time period, the average number of rule condition checks executed per request within a given time period, the average number of actions triggered per request within a given time period, the average rule evaluation context size per request or the number of defined conditional facts per tenant within a given period, or a combination thereof.

[0030] Once the workload of each rule model is determined, method 200 proceeds to operation 212, where these workloads are used to distribute the rule models among a set of orchestrated microservice engines (such as rule engine instance 106, for example) in order to optimize the functionality and resources of the microservice provider. More specifically, the workloads calculated for all tenants 110 are considered to generate an in-memory rule engine distribution model. Based on the in-memory rule engine distribution model, the rule sets are distributed across the rule engine instances 106 such that the workloads of all tenants 110 are distributed as evenly as possible across the rule engine instances 106. In other words, the number of tenants 110 or the associated rule sets assigned to each rule engine instance 106 may vary, but the workload assigned to each rule engine instance 106 is as equal as possible. Thus, each rule engine instance 106 processes approximately the same proportion of the overall workload. Therefore, the available system resources are optimized by being deployed as uniformly as possible across the rule engine instances 106.

[0031] For example, the execution of method 200 may result in the assignment of rules for tenants T1 - T30 and T700 - T2000 to rule engine instance A106A, the assignment of rules for tenants T31 - T699 and T2001 - T2500 to rule engine instance B106B, the assignment of rules for tenants T2501 - T3225 and T3900 - T4000 to rule engine instance C106C, and the assignment of rules for tenants T3226 - T3899 to rule engine instance D106D.

[0032] Note that the above is an exemplary distribution. Other distributions based on other factors (such as user settings, the relative processing power of nodes hosting various engines, etc.) are possible. For example, in at least some embodiments of the present disclosure, the distribution of the rule model may be determined based on the resources available to or the capabilities of, or both, individual rule engine instances. For example, a particular rule engine instance may be on a server with more memory. Thus, this particular rule engine instance may have access to greater memory resources than other rule engine instances. In this way, it may be advantageous to distribute the rule model such that a greater proportion is processed by a particular rule engine instance rather than distributing the rule model approximately evenly across all rule engine instances.

[0033] In at least one embodiment of the present disclosure, the distribution or redistribution of tenant 110 on rule engine instance 106 based on an in-memory rule engine distribution model may be achieved by changing the assignment of the tenant from the corresponding source rule engine instance 106 to the corresponding target rule engine instance 106. For example, changing the assignment of tenant T31 from rule engine instance A 106A to rule engine instance B 106B includes bootstrapping the rule model for tenant T31 on the target instance, in this case rule engine instance B 106B, and then modifying the corresponding entry in the mapping table. The rule model for tenant T31 is then removed from the source instance, in this case rule engine instance A 106A, and the corresponding entry in the mapping table is modified accordingly.

[0034] According to some embodiments of the present disclosure, when the workload is distributed across rule engine instances 106 by the method 200 or the like, the rule requests may be executed according to the method 300 shown in FIG. 3. Method 300 begins with operation 304, where an incoming rule execution request from tenant 110 is received. In at least one embodiment of the present disclosure, the incoming rule execution request is received by microservice provider 102, and rule distribution microservice 104 then receives the incoming rule execution request from microservice provider 102.

[0035] For illustration purposes, in one example, a consumer may search for widgets on a product website. Tenant T5 is associated with the product website and receives search results from a search service provider. Tenant T5 then sends these search results to microservice provider 102 along with a rule execution request. In this example, in operation 304, the incoming rule execution request is a request to microservice provider 102 to return a preferred search result representation for applying rules specific to Tenant T5 to the search results for presentation to the consumer.

[0036] In operation 308, the most suitable rule engine instance 106 for providing a response to the received request is selected based on the in-memory rule engine distribution model described above. In at least one embodiment of the present disclosure, the most suitable rule engine instance 106 is selected by rule distribution microservice 104. In at least one embodiment of the present disclosure, rule distribution microservice 104 analyzes the incoming request, applies the criteria used to determine the distribution of rule sets among rule engine instances 106, and selects the appropriate rule engine instance 106 to which the request should be sent.

[0037] In at least one alternative embodiment of the present disclosure, the most suitable rule engine instance 106 may be identified using the mapping table. For example, the most suitable rule engine instance 106 may determine which tenant is associated with the request, refer to the mapping table to identify which rule engine instance 106 is associated with that tenant, and then be identified by sending the request to the identified rule engine instance 106.

[0038] Continuing with the example of the description, the rule distribution microservice 104 selects a rule engine instance 106 that has a copy of the tenant-specific rules for tenant T5. In this example, the rule engine instance A 106A has a copy of the tenant-specific rules for tenant T5. Thus, in operation 308, the rule distribution microservice 104 selects the rule engine instance A 106A.

[0039] In operation 312, the rule execution request is routed to the selected rule engine instance 106. In at least one embodiment of the present disclosure, the rule execution request is routed by the rule distribution microservice 104. In the example of the description, the rule execution request from tenant T5 is routed to the selected rule engine instance 106A. In at least one embodiment of the present disclosure, the search results are sent to the selected rule engine together with the rule execution request.

[0040] In operation 316, the rule execution request is executed. In at least one embodiment of the present disclosure, the rule execution request is executed by the designated rule engine instance 106. In the example of the description, the rule engine instance 106A executes the requested rule execution request by applying the tenant-specific rules for tenant T5 to the provided search results. The tenant-specific rules for tenant T5 may include, for example, a rule that any result within a particular price range within the search results is ranked in a particular order in the presentation of the search results. The tenant-specific rules for tenant T5 may further include a rule that any result within a particular price range available within a particular geographic distance of a particular location is ranked in a particular order in the presentation of the search results.

[0041] In operation 320, the result of the executed rule execution request is returned to the microservice provider 102. In at least one embodiment of the present disclosure, the result of the executed rule execution request is returned to the microservice provider 102 by the rule engine instance 106 that executed the rule execution request. In the example for illustration, the result of the rule execution request executed by the rule engine instance 106A is returned to the microservice provider 102. These results include the prioritization of the search results by the two rules discussed above. The microservice provider 102 can then provide these prioritized results to tenant T5, and tenant T5 can present the prioritized results to the consumer.

[0042] In operation 324, an asynchronous message is sent to the rule monitoring component 108. In at least one embodiment of the present disclosure, the asynchronous message is sent by the rule engine instance 106 that executed the rule execution request. In at least one embodiment of the present disclosure, the asynchronous message includes rule metrics or information used to determine rule metrics. In the example for illustration, the asynchronous message is sent by the rule engine instance 106A to the rule monitoring component 108 and includes, but is not limited to, the number of rules applied during rule execution and the number of conditions or facts to be checked to execute these rules.

[0043] In operation 328, the rule metrics for the executed rule execution request are aggregated. In at least one embodiment of the present disclosure, the rule metrics are aggregated by the rule monitoring component 108. In the example for explanation, the data including the number of rules applied during rule execution and the number of conditions or facts to be checked to execute those rules is aggregated with other data similar to other rule execution requests received by the microservice provider 102.

[0044] These aggregated rule metrics may then be used to recalculate or update the workload for each rule model or tenant 110. For example, the data within the aggregated rule metrics may be input back into the algorithm provided in operation 208 of method 200. By updating the workload based on data collected during operation of system 100 and feeding the updated workload back into the in-memory rule engine distributed model, the distribution of rule models among rule engine instances may be further and repeatedly optimized. For example, in at least one embodiment of the present disclosure, the workload may be updated based on data from the aggregated rule metrics at regular intervals.

[0045] Once the workload is initially distributed across rule engine instances 106, such as by method 200 described above, according to some embodiments of the present disclosure, the distribution may be further optimized dynamically over time. Such further rule optimization and scaling may be performed by a system 400 as shown in FIG. 4.

[0046] More specifically, system 400 includes a rule distribution microservice 404 that is substantially similar to rule distribution microservice 104 described above with reference to FIG. 1, a plurality of rule engine instances 406 that are substantially similar to rule engine instances 106 described above with reference to FIG. 1, and a rule monitoring component 408 that is substantially similar to rule monitoring component 108 described above with reference to FIG. 1. System 400 further includes a rule distribution optimization and scaling component 412 and a rule store 416.

[0047] According to at least one embodiment of the present disclosure, the rule distribution optimization and scaling component 412 is configured to request aggregated rule metrics from the rule monitoring component 408. The request may be sent, for example, at regular time intervals. Further, the rule monitoring component 408 is configured to return the aggregated rule metrics to the rule distribution optimization and scaling component 412. The requested rule metrics may include data regarding which rules are executed by which tenants and the memory consumption of each rule and the CPU consumption of the average rule execution. In addition, the rule distribution optimization and scaling component 412 is communicatively coupled to the rule distribution microservice 404 such that the rule distribution microservice 404 can receive an updated rule engine distribution model from the rule distribution optimization and scaling component 412. Such an updated rule engine distribution model may then be used in the assignment of rule sets, such as by method 200, or in the subsequent execution of rule requests, such as by method 300, or both.

[0048] According to at least one embodiment of the present disclosure, the rule store 416 is configured to store all of the rules of the rule model. In other words, the rule store 416 is provided to a microservice provider (such as the microservice provider 102 shown in FIG. 1) and is configured to operate as a repository for rules that are distributed among rule engine instances 406.

[0049] An exemplary method 500 for further optimizing the distribution of a rule model among a set of assembled microservice engines is shown in FIG. 5. In at least some embodiments of the present disclosure, method 500 is implemented using system 400. Method 500 begins with a workload distributed across rule engine instances 106, such as by method 200 above.

[0050] In operation 504, aggregated rule metrics are requested from the rule monitoring component 408. In at least one embodiment of the present disclosure, the rule distribution optimization and scaling component 412 requests an aggregated rule metric from the rule monitoring component 408. In at least one embodiment of the present disclosure, the rule distribution optimization and scaling component 412 requests an aggregated rule metric at regular intervals. In particular, the requested aggregated rule metric may include, without limitation, which rules are executed by which tenants, the memory consumption of each rule, and the CPU consumption of the average rule execution.

[0051] In operation 508, the assignment between the rule model and the rule engine instance 406 is recalculated. In at least one embodiment of the present disclosure, the rule model is loaded from the rule store 416. In at least one embodiment of the present disclosure, the rule distribution optimization and scaling component 412 recalculates the assignment between the rule model (loaded from the rule store) and the rule engine instance 406 to achieve balanced central processing unit and memory consumption across the rule engine instances 406. In at least one embodiment of the present disclosure, if the expected memory and CPU consumption exceeds a predefined upper value, the rule distribution optimization and scaling component 412 instantiates an additional rule engine instance 406. However, if the memory and CPU consumption is below a predefined lower value, the rule distribution optimization and scaling component 412 shuts down the rule engine instance 406 and releases system resources.

[0052] In operation 512, the updated rule engine distribution model is sent to the rule distribution microservice 404. In at least one embodiment of the present disclosure, the rule distribution optimization and scaling component 412 sends the updated rule engine distribution model to the rule distribution microservice 404.

[0053] In operation 516, the request is sent to each rule engine instance 406 along with the identification information of the corresponding rules that each rule engine instance 406 needs to load into or make available in memory, or both. In at least one embodiment of the present disclosure, the rule distribution optimization and scaling component 412 sends the request to each rule engine instance 406 along with the identification of the rules that each rule engine instance 406 needs to load into or make available in memory.

[0054] In at least one embodiment of the present disclosure, machine learning can be used to predict the expected rule load per tenant. For example, based on time or another quantitative variable, the rule load of a tenant may vary. These variations may be detected by sensors configured to detect the clock or quantitative variables. Such predictions enable the system to proactively distribute or redistribute the rule model in a way that prevents high loads on specific rule engine instances.

[0055] In at least one embodiment of the present disclosure, the execution of rules may be analyzed to identify those that are executed together commonly. This factor can be used to manage the distribution of rules across rule engine instances.

[0056] Referring now to FIG. 6, there is shown a high-level block diagram of an exemplary computer system 601 that can be utilized in implementing one or more of the methods, tools, and modules and any related functions described herein (e.g., using one or more processor circuits or computer processors of a computer). In some embodiments, the main components of computer system 601 may include one or more CPUs 602, a memory subsystem 604, a terminal interface 612, a storage interface 614, an I / O (input / output) device interface 616, and a network interface 618, all of which are communicatively coupled, either directly or indirectly, for component-to-component communication via a memory bus 603, an I / O bus 608, and an I / O bus interface unit 610.

[0057] Computer system 601 includes one or more general-purpose programmable central processing units (CPUs) 602A, 602B, 602C, and 602D, which are referred to herein generically as CPU 602. In some embodiments, computer system 601 may include a multi-processor typical of relatively large-scale systems, however, in other embodiments, computer system 601 may alternatively be a single CPU system. Each CPU 602 executes instructions stored in memory subsystem 604 and may include one or more levels of on-board cache.

[0058] System memory 604 may include a computer system-readable medium in the form of volatile memory, such as random access memory (RAM) 622 or cache memory 624. Computer system 601 may further include other removable / non-removable volatile / non-volatile computer system storage media. By way of example, storage system 626 is provided for reading from and writing to a non-portable non-volatile magnetic medium, such as a "hard drive". Although not shown, a magnetic disk drive for reading from and writing to a removable non-volatile magnetic disk (e.g., a "floppy disk" (registered trademark)), or an optical disk drive for reading from and writing to a removable non-volatile optical disk, such as a CD-ROM, DVD-ROM, or other optical media, may be provided. Additionally, memory 604 may include flash memory, such as a flash memory stick drive or flash drive. The memory device may be connected to memory bus 603 by one or more data media interfaces. Memory 604 may include at least one program product having a set (at least one) of program modules configured to implement the functions of various embodiments.

[0059] One or more programs / utilities 628 may each have a set (at least one) of program modules 1030 and may be stored in memory 604. Programs / utilities 628 may include a hypervisor (also referred to as a virtual machine monitor), one or more operating systems, one or more application programs, other program modules, and program data. Each or some combination of the operating system, one or more application programs, other program modules, and program data may include an implementation of a networking environment. Program modules 1030 generally execute the functions or methodologies of various embodiments.

[0060] Although the memory bus 603 is shown in FIG. 6 as a single bus structure that provides a direct communication path between the CPUs 602, the memory subsystem 604, the I / O bus interface 610, and the memory bus 603 may, in some embodiments, include a plurality of different buses or communication paths, which may be arranged in any of various forms such as point-to-point links, multi-tier buses, parallel and dual paths, or any other suitable type of configuration in a hierarchical, star, or web configuration. Further, although the I / O bus interface 610 and the I / O bus 608 are shown as single respective units, the computer system 601 may, in some embodiments, include a plurality of I / O bus interface units 610, a plurality of I / O buses 608, or both. Further, although a plurality of I / O interface units that separate the I / O bus 608 from various communication paths to various I / O devices are shown, in other embodiments, some or all of the I / O devices may be directly connected to one or more system I / O buses.

[0061] In some embodiments, the computer system 601 may be a multi-user mainframe computer system, a single-user system, or a server computer, or a similar device having few or no user interfaces, but receives requests from other computer systems (clients). Further, in some embodiments, the computer system 601 may be implemented as a desktop computer, a portable computer, a laptop or notebook computer, a tablet computer, a pocket computer, a telephone, a smartphone, a network switch or router, or any other suitable type of electronic device.

[0062] Note that FIG. 6 is intended to depict the representative major components of an exemplary computer system 601. In some embodiments, however, the individual components may have more or less complexity than shown in FIG. 6, there may be components other than or in addition to those shown in FIG. 6, and the number, type, and configuration of such components may vary.

[0063] This disclosure includes a detailed description of cloud computing, but it should be understood in advance that the implementation of the teachings detailed herein is not limited to a cloud computing environment. Rather, embodiments of the invention are implementable in conjunction with any other type of computing environment now known or later developed.

[0064] Cloud computing is a service distribution model that enables convenient on-demand network access to a shared pool of configurable computing resources (such as networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with a service provider. This cloud model may include at least five characteristics, at least three service models, and at least four deployment models.

[0065] The characteristics are as follows.

[0066] On-demand self-service: Cloud consumers can unilaterally provision computer capabilities such as server time and network storage automatically and as needed, without the need for human interaction with a service provider.

[0067] Broadband network access: The capabilities are accessible over the network and are accessed via standard mechanisms that facilitate use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, PDAs).

[0068] Resource pooling: The provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, and various physical and virtual resources are dynamically assigned and reallocated according to demand. Consumers generally have a sense of location independence in that they do not manage or have knowledge of the exact location of the provided resources, but can specify location at a higher level of abstraction (e.g., country, state, or data center).

[0069] Rapid elasticity: The capabilities can be provisioned quickly and flexibly, and in some cases automatically, to scale out rapidly and can be released quickly to scale in rapidly. For the consumer, the provisioned available capabilities often appear to be unlimited externally and can be purchased in any quantity at any time.

[0070] Measured service: The cloud system automatically controls and optimizes resource use by leveraging metering capabilities at an appropriate level of abstraction for the type of service (e.g., storage, processing, bandwidth, number of active users). Resource usage is monitored, controlled, and reported to provide transparency to both the provider and the consumer of the utilized service.

[0071] The service model is as follows.

[0072] Software as a Service (SaaS): The ability provided to the consumer is to use the provider's application running on cloud infrastructure. The application is accessible from various client devices via a thin client interface such as a web browser (e.g., web-based email). The consumer does not manage or control the underlying infrastructure, including the network, servers, operating systems, storage, or even the individual application capabilities, except for potential exceptions of limited user-specific application configuration settings.

[0073] Platform as a Service (PaaS): The ability provided to the consumer is to deploy consumer-created or acquired applications, created using programming languages and tools supported by the provider, onto cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, or storage, but has control over the deployed applications and, in some cases, the configuration of the application hosting environment.

[0074] Infrastructure as a Service (IaaS): The ability provided to the consumer is to provide other basic computing resources, such as processing, storage, network, and the consumer can deploy and run any software that may include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure, but has control over the operating systems, storage, deployed applications, and, in some cases, limited control over selected networking components (e.g., host firewall).

[0075] The deployment model is as follows.

[0076] Private Cloud: The cloud infrastructure is used only for one organization. This may be managed by the organization or a third party and may exist on-premises or off-premises.

[0077] Community Cloud: The cloud infrastructure is shared by several organizations and supports a specific community with common concerns (e.g., mission, security requirements, policies, and compliance considerations). This may be managed by the organization or a third party and may exist on-premises or off-premises.

[0078] Public Cloud: The cloud infrastructure is available to the general public or a large industry group and is owned by an organization that sells cloud services.

[0079] Hybrid Cloud: The cloud infrastructure is a mix of two or more clouds (private, community, or public), which remain distinct entities but are joined by standardized or proprietary technologies (e.g., cloud bursting for load distribution between clouds) that enable data and application portability.

[0080] The cloud computing environment is service-oriented, emphasizing statelessness, loose coupling, modularity, and semantic interoperability. The core of cloud computing is an infrastructure that includes a network of multiple interconnected nodes.

[0081] Referring now to FIG. 7, an exemplary cloud computing environment 50 is shown. As illustrated, cloud computing environment 50 includes one or more cloud computing nodes 10, which may communicate with local computing devices used by cloud consumers such as, for example, a PDA or cellular telephone 54A, desktop computer 54B, laptop computer 54C, or automotive computer system 54N, or a combination thereof. Nodes 10 may communicate with one another. These may be physically or virtually grouped (not shown) in one or more networks such as private, community, public, or hybrid clouds as described above, or a combination thereof. This enables cloud computing environment 50 to provide infrastructure, platform, software, or a combination thereof as services, for which a cloud consumer need not maintain resources on a local computing device. The types of computing devices 54A - 54N shown in FIG. 7 are for illustrative purposes only, and it is understood that cloud computing nodes 10 and cloud computing environment 50 can communicate with any type of computerized device via any type of network, network addressable connection (e.g., using a web browser), or both.

[0082] Referring now to FIG. 8, a set of functional abstraction layers provided by cloud computing environment 50 (FIG. 7) is shown. It should be understood in advance that the components, layers, and functions shown in FIG. 8 are for illustrative purposes only and that embodiments of the invention are not limited thereto. As shown, the following layers and corresponding functions are provided.

[0083] The hardware and software layer 60 includes hardware and software components. Examples of hardware components include mainframe 61, server 62 based on RISC (Reduced Instruction Set Computer) architecture, server 63, blade server 64, storage device 65, and network and networking components 66. In some embodiments, the software components include network application server software 67 and database software 68.

[0084] The virtualization layer 70 provides an abstraction layer from which examples of virtualized entities such as virtualized server 71, virtualized storage 72, virtualized network 73 including virtual private network, virtualized applications and operating systems 74, and virtual client 75 are provided.

[0085] In one example, the management layer 80 may provide the functions described below. Resource provisioning 81 provides for the dynamic procurement of computing resources and other resources utilized to execute tasks within a cloud computing environment. Metering and pricing 82 provides for the tracking of the costs of resources utilized within the cloud computing environment and the billing or invoicing for the consumption of these resources. In one example, these resources may include licenses for application software. Security provides for the authentication of cloud consumers and tasks, and the protection of data and other resources. The user portal 83 provides access to the cloud computing environment for consumers and system administrators. Service level management 84 provides for the allocation and management of cloud computing resources to meet the required service levels. Planning and fulfillment of service level agreements (SLAs) 85 provides for the pre - placement and procurement of cloud computing resources for which future demands are anticipated, in accordance with the SLA.

[0086] The workload layer 90 provides examples of the functionality for which a cloud computing environment is utilized. Examples of the workloads and functions provided by this layer include mapping and navigation 91, software development and lifecycle management 92, virtual classroom education distribution 93, data analytics processing 94, transaction processing 95, mobile desktop 96.

[0087] In addition to the above embodiments, other embodiments having fewer operation steps, more operation steps, or different operation steps are contemplated. Also, in some embodiments, some or all of the above operation steps may be executed in a different order. Further, multiple operations may occur simultaneously, or may occur as an inner part of a larger process. Exemplary embodiments are enumerated and described in accordance with the embodiments, but this is not meant to indicate the necessity of a particular module, nor is it meant to indicate the exclusion of other potential modules (or functions / purposes applied to a particular module).

[0088] In the above, various embodiments have been referred to. However, it should be understood that the present disclosure is not limited to what is specifically described. Instead, any combination of the described features and elements is contemplated to implement and practice the present disclosure, regardless of whether they relate to different embodiments. Many modifications and variations may be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. Further, embodiments of the present disclosure may achieve advantages over other potential solutions or over the prior art, but whether a particular advantage is achieved by a given embodiment is not limiting of the present disclosure. Accordingly, the described aspects, features, embodiments, and advantages are merely exemplary and are not considered elements or limitations of the appended claims, unless expressly recited therein.

[0089] The present disclosure may be a system, method, or computer program product, or a combination thereof. The computer program product may include a computer-readable storage medium having thereon computer-readable program instructions for causing a processor to execute aspects of the present invention.

[0090] A computer-readable storage medium may be a tangible device that holds and stores instructions for use by an instruction execution device. The computer-readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of the computer-readable storage medium, by way of non-limiting example, include a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk®, a punch card, or a mechanically encoded device such as a raised structure within a groove having recorded instructions, and any suitable combination of the foregoing. The computer-readable storage medium, as used herein, is not itself a propagated signal such as a radio wave, a freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., an optical pulse passing through a fiber optic cable), or an electrical signal transmitted through a wire that is interpreted as a transient signal per se.

[0091] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to respective computers / processing devices or to an external computer or external storage device via a network such as, for example, the Internet, a local area network, a wide area network, or a wireless network, or combinations thereof. The network may include copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers or edge servers, or combinations thereof. A network adapter card or network interface in each computer / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions for storage on a computer-readable storage medium within each computing / processing device.

[0092] Computer-readable program instructions for performing the operations of the present invention may be written in assembly instructions, instruction set architecture (ISA) instructions, machine language instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, where the one or more programming languages include object-oriented languages such as Smalltalk®, C++, or the like, and conventional procedural languages such as the C programming language or similar programming languages. The computer-readable program instructions may be executed as a stand-alone software package, entirely on the user's computer, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, an electrical circuit may execute the computer-readable program instructions by utilizing the state information of the computer-readable program instructions to customize the electrical circuit in order to perform aspects of the present invention, and the electrical circuit may include, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA).

[0093] Aspects of the invention will be described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0094] These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart illustration and / or block, or blocks, thereof. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, or other device to function in a particular manner, such that the computer-readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement the function / act specified in the flowchart and / or block, or blocks, thereof.

[0095] The computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block, or blocks, thereof.

[0096] Flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions that includes one or more executable instructions for implementing a particular logical function. In some alternative implementations, the functions recited in the blocks may occur out of the order shown in the figures. For example, two blocks shown in succession may, in fact, be implemented as one step, executed simultaneously, substantially simultaneously, in a partially or wholly temporally overlapping manner, or the blocks may be executed in the reverse order depending on the functionality involved. Note that each block of a block diagram or flowchart diagram, or combinations of multiple blocks of a block diagram or flowchart diagram, or both, may be implemented by a special purpose hardware-based system that performs a particular function or action, or implements a combination of special purpose hardware and computer instructions.

[0097] The terms used in this specification are for the sole purpose of describing particular embodiments and are not intended to limit the various embodiments. As used in this specification, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Further, the term "include" or "including" or both, when used in this specification, specifies the presence of the described feature, integer, step, operation, element or component or combination thereof, and does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components or groups thereof or combinations thereof. In the description of the exemplary embodiments of the various embodiments described above, reference is made to the accompanying drawings (where like numerals represent like elements), which form a part of this specification, and which are shown to illustrate specific exemplary embodiments in which the various embodiments can be implemented. These embodiments have been described in sufficient detail so that those skilled in the art can implement the embodiments, but other embodiments may be used and logical, mechanical, electrical and other changes can be made without departing from the scope of the various embodiments. In the foregoing description, many specific details have been set forth in order to provide a thorough understanding of the various embodiments. However, the various embodiments can be practiced without these specific details. In some other instances, well-known circuits, structures and techniques have not been described in detail so as not to obscure the embodiments.

[0098] As used in this specification, "a number of", when used in reference to an item, means one or more items. For example, "a number of different networks" are a plurality of different types of networks.

[0099] If different reference numerals include a common numeral and subsequent different letters (e.g., 100a, 100b, 100c) or punctuation and subsequent different numerals (e.g., 100-1, 100-2 or 100.1, 100.2), use of only the reference numeral excluding the letter or subsequent numeral (e.g., 100, etc.) may refer to a group of elements as a whole, any subset of the group, or an exemplary specimen of the group.

[0100] Furthermore, when the phrase "at least one of" is used with a list of items, it means that one or more different combinations of the listed items are used, and only one of each item in the list may be required. In other words, the phrase "at least one of" means that any combination of items and the number of items may be used from the list, however, not all items in the list are necessarily required. An item may be a particular object, thing, or category.

[0101] For example, without limitation, "at least one of item A, item B or item C" may include item A, item A and item B, or item B. This example may also include item A, item B and item C, or item B and item C. Of course, any combination of these items may exist. In some illustrative examples, "at least one of" may be, for example, without limitation, two of item A; one of item B, ten of item C, four of item B and seven of item C, or other suitable combinations.

[0102] As used in this specification, different examples of the word "embodiment" do not necessarily refer to the same embodiment, but they may. Any data and data structures illustrated or described in the specification are merely examples, and in other embodiments, different amounts of data, data types, fields, numbers and types of fields, field names, numbers and types of rows, records, entries, or organization of data may be used. Additionally, any data may be combined with logic so as not to require a separate data structure. The foregoing detailed description should, therefore, not be construed in a limiting sense.

[0103] Descriptions of various embodiments of the present disclosure have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein are chosen to best explain the principles of the embodiments, the practical application, or technical improvements found in the marketplace, or to enable other ordinary skill in the art to understand the embodiments disclosed herein.

[0104] Although the invention has been described in connection with specific embodiments, changes and modifications thereto will be apparent to those of ordinary skill in the art. Accordingly, the following claims are intended to be construed to cover all changes and modifications that fall within the true spirit and scope of the invention.

Claims

1. 1. A method, comprising: determining a rule load for each rule set of a plurality of rule sets, each rule set being associated with a tenant of a plurality of tenants hosted on the multi-tenant system; aggregating said rule loads into a total rule load; distributing the plurality of rule sets across the set of rule engine instances such that the overall rule load is distributed as evenly as possible across the set of rule engine instances; A method for performing.

2. The method of claim 1 , wherein the plurality of rule sets is maintained by a microservice provider.

3. The method of claim 2 , wherein the set of rule engine instances is deployed by the microservice provider.

4. Determining the rule load for each of the rule sets comprises: determining a number of requests made by each tenant; determining an average number of actions triggered per request; The method according to any one of claims 1 to 3, comprising:

5. Determining the rule load for each of the rule sets comprises: determining a number of requests made by each tenant; determining the number of conditions to be considered for responding to each request; The method according to any one of claims 1 to 4, comprising:

6. The number of requests made by each tenant and the number of conditions considered for responding to each request are sent to a rule monitoring component; 6. The method of claim 5, wherein the rule monitoring component aggregates these counts for each of the rule sets.

7. The method further comprises the step of: obtaining the aggregated number from the rule monitoring component; redistributing the plurality of rule sets across the set of rule engine instances based on the aggregated numbers such that a substantially equal percentage of the overall rule load is assigned to each rule engine instance of the set of rule engine instances; The method of claim 6, further comprising:

8. A computer program product causing a processor to carry out the steps of the method according to any one of claims 1 to 7.

9. A computer readable storage medium having recorded thereon the computer program product of claim 8.

10. A rule distribution system, comprising: Memory, a processor communicatively coupled to the memory; wherein the processor is configured to execute a method, the method comprising: determining a rule load for each rule set of a plurality of rule sets, each rule set being associated with a tenant of a plurality of tenants hosted on the multi-tenant system; aggregating said rule loads into a total rule load; distributing the plurality of rule sets across the set of rule engine instances such that the overall rule load is distributed as evenly as possible across the set of rule engine instances; A rule distribution system including:

11. The rule distribution system of claim 10 , wherein the plurality of rule sets are maintained by a microservice provider.

12. The rule distribution system of claim 11 , wherein the set of rule engine instances is deployed by the microservice provider.

13. Determining the rule load for each of the rule sets comprises: determining a number of requests made by each tenant; determining an average number of actions triggered per request; The rule distribution system according to any one of claims 10 to 12, comprising:

14. Determining the rule load for each of the rule sets comprises: determining a number of requests made by each tenant; determining the number of conditions to be considered for responding to each request; The rule distribution system according to any one of claims 10 to 13, comprising:

15. The method further comprises: aggregating the number of requests made by each tenant and the number of conditions considered for responding to each request; redistributing the plurality of rule sets across the set of rule engine instances based on the aggregated count such that a substantially equal percentage of the overall rule load is assigned to each rule engine instance of the set of rule engine instances; 15. The rule distribution system of claim 14, comprising:

Images