Optimizing communication in distributed computing networks

Abstract

A computing system includes a distributed computing cluster comprising a plurality of computing nodes interconnected via an interconnection network, wherein the computing nodes in the plurality of computing nodes communicate with each other via message passing via the interconnection network. The computing nodes are configured with a first parameter governing message transmission by the computing nodes via the interconnection network. The computing nodes are configured to accumulate messages for transmission as a group according to the first parameter, and the computing system is configured to limit injection of computing requests into the distributed computing cluster according to a second parameter. A controller is configured to receive at least one predetermined service level requirement and control a value of the second parameter and a value of the first parameter to control the computing throughput of the distributed computing cluster while complying with the at least one service level requirement.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 027,008, filed May 19, 2020, the contents of which are incorporated herein in their entirety. Background Art

[0003] This specification relates to managing and optimizing communications in distributed computing networks.

[0004] Distributed computing systems use clusters of computing nodes working together in a coordinated manner to solve computational problems. Computing nodes are typically located on separate networked computers that communicate via an interconnect. In operation, the computational problem is divided into multiple tasks, each of which is solved by one or more (individual) computing nodes in the distributed computing system. The solution of the computational problem is coordinated by passing messages between tasks (and computing nodes) via the interconnect. Messages are passed over the interconnect for a variety of reasons, including providing computational results to downstream tasks and replicating data. Summary of the Invention

[0005] In a distributed computing system, when the state of a node in the network changes (e.g., a node receives an instruction from an external source or generates the result of a task), the change can be notified to other computing nodes in the distributed computing system. To notify other computing nodes of the change, messages containing information related to network transactions are communicated between the nodes. Because message communication between computing nodes can be frequent, it is desirable to maintain efficiency and high throughput when communicating messages.

[0006] One strategy for achieving efficiency and high throughput involves bundling together several messages from different transactions and sending the message bundle in a single network transaction. The bundling strategy increases throughput on the interconnect because the CPU cost and network overhead of sending a single message are largely fixed, and the CPU cost of sending subsequent messages in the same transaction is negligible relative to the CPU cost of sending a single message. However, the bundling strategy may also result in undesirably large network delays due to the system waiting for messages to fill the bundle before sending the bundle. Another parameter that affects throughput and latency in a distributed computing system is the rate at which requests are injected into the distributed computing system. For example, an injection rate that is too high may cause congestion, and thereby reduce efficiency and / or increase latency. Aspects described herein relate to strategies for controlling the above-mentioned parameters of a distributed computing system to balance latency and throughput in order to achieve desired overall network performance.

[0007] In a general aspect, a computing system includes: a distributed computing cluster comprising: a plurality of computing nodes interconnected by an interconnection network, computing nodes in the plurality of computing nodes communicating with each other via message passing via the interconnection network. At least some of the computing nodes are each configured with a first parameter that governs message transmission by the computing nodes via the interconnection network. The at least some computing nodes are configured to accumulate messages for transmission as a group of messages according to the first parameter, and the computing system is configured to limit injection of computing requests into the distributed computing cluster according to a second parameter. The computing system also includes: a controller configured to receive at least one predetermined service level requirement and control a value of the second parameter and a value of the first parameter to control the computing throughput of the distributed computing cluster while complying with the at least one service level requirement.

[0008] Aspects may include one or more of the following features.

[0009] A computing system may include one or more interface components, each configured to receive input data, inject requests into a distributed computing cluster to process the input data, receive processing results from the distributed computing cluster, and generate output data based on the processing results. The interface components may be configured to limit the rate at which requests are injected into the distributed computing system based on a second parameter. At least some of the one or more interface components may be executed on a client system separate from the system executing the distributed computing cluster. At least some of the received input data may be associated with a priority level, and the one or more interface components may be configured to manage injection of the input data into the distributed computing cluster based on the priority level.

[0010] The distributed computing cluster may include a rate limiter configured to limit a rate at which requests are injected into the distributed computing system based on a second parameter. Each of the one or more service level requirements may specify a maximum duration within which the distributed computing cluster must satisfy or respond to a request. The computing system may include a monitor configured to observe a computational throughput of the distributed computing cluster and a processing latency of the distributed computing system. The observed computational throughput may be measured by monitoring a CPU load of one or more computing nodes.

[0011] The observed computational throughput can be inferred from the number of messages present in the interconnected network. The observed latency can be measured at the service request level. The observed latency can be measured by determining the time difference between injecting a request into the distributed computing cluster and receiving a response to the request from the distributed computing cluster. The requests injected into the distributed computing cluster and the results output by the distributed computing cluster can be communicated through a monitor.

[0012] The controller may be configured to process the observed computational throughput and observed processing latency of the distributed computing cluster to adjust the value of the second parameter and the value of the first parameter. The controller may repeatedly process the observed computational throughput and observed processing latency of the distributed computing cluster to adjust the value of the second parameter and the value of the first parameter while the distributed computing cluster is processing requests. The controller may implement a feedback loop to process the observed computational throughput and observed processing latency of the distributed computing cluster to adjust the value of the second parameter and the value of the first parameter. The controller may use an optimization algorithm to control the value of the first parameter.

[0013] The first parameter may include a delay parameter that governs a minimum time interval between transmissions of messages by a computing node over the interconnect network. At least some of the computing nodes may be configured to accumulate messages for transmission as a group of messages during the minimum time interval. The second parameter may include an input rate parameter that is configured to limit the rate at which computing requests are injected into the distributed computing cluster. Controlling the computing throughput of the distributed computing cluster while complying with at least one service level requirement may include maximizing the computing throughput of the distributed computing cluster without violating any of the at least one service level requirement. The first parameter may govern the number of messages accumulated between transmissions of messages by the computing nodes over the interconnect network.

[0014] In another general aspect, a method includes limiting injection of computational requests into a distributed computing cluster based on a second parameter, wherein the distributed computing cluster includes a plurality of computing nodes interconnected by an interconnection network, wherein computing nodes in the plurality of computing nodes communicate with each other via message passing via the interconnection network. At least some of the computing nodes are each configured with a first parameter that governs message transmission by the computing nodes via the interconnection network. The method also includes accumulating messages for transmission as a group of messages at the at least some of the computing nodes based on the first parameter, receiving at least one predetermined service level requirement at a controller, and controlling, using the controller, a value of the second parameter and a value of the first parameter to maximize computational throughput of the distributed computing cluster while complying with the at least one service level requirement.

[0015] In another general aspect, software stored in non-transitory form on a computer-readable medium includes instructions for causing a computing system to: limit a rate at which computing requests are injected into a distributed computing cluster based on a second parameter. The distributed computing system includes: a plurality of computing nodes interconnected by an interconnection network, wherein computing nodes in the plurality of computing nodes communicate with each other through the interconnection network using message passing. At least some of the computing nodes are each configured with a first parameter that governs the transmission of messages by the computing nodes through the interconnection network. The instructions further cause the computing system to accumulate messages for transmission as a group of messages at at least some of the computing nodes based on the first parameter, receive at least one predetermined service level requirement at a controller, and control, using the controller, a value of the second parameter and a value of the first parameter to maximize the computing throughput of the distributed computing cluster while complying with the at least one service level requirement.

[0016] In another general aspect, a computing system includes means for limiting the rate at which computing requests are injected into a distributed computing cluster based on a second parameter. The distributed computing system includes a plurality of computing nodes interconnected by an interconnection network, computing nodes in the plurality of computing nodes communicating with each other via message passing via the interconnection network, at least some of the computing nodes each configured with a first parameter that governs the transmission of messages by the computing nodes via the interconnection network. The computing system also includes means for accumulating messages for transmission as a group of messages at at least some of the computing nodes based on the first parameter, means for receiving at least one predetermined service level agreement at a controller, and means for controlling, using the controller, a value of the second parameter and a value of the first parameter to maximize the computing throughput of the distributed computing cluster while complying with the at least one service level requirement.

[0017] Aspects can include one or more of the following advantages.

[0018] A distributed computing cluster, comprising many interconnected computing nodes, is capable of receiving and processing large amounts of information, and subsequently communicating the processed information outside the network. Typically, while jobs entering a distributed computing cluster can be designated as "high priority" or "low priority," there is rarely a concept of "priority" within the computing cluster where all information is processed equally. The concept of "priority" associated with jobs entering a distributed computing cluster is related to the desired service level agreement (SLA). However, meeting the SLA requires not only controlling jobs entering the system based on their priority, but also controlling the throughput and latency of jobs within the computing cluster. While controlling one of throughput and latency is relatively simple, controlling both to meet the SLA can be challenging.

[0019] A difficulty arises in controlling both throughput and latency because it is impossible to know a priori how many jobs will enter the system at a given time. Adapting the compute cluster to the volume of jobs entering it at any given time eliminates the possibility of jobs flooding the system, allowing the compute cluster to control not only the internal throughput and latency of the jobs being processed in the system, but also the SLAs for those jobs—simply put, if the system cannot meet a particular SLA for an incoming job, the job is not accepted by the compute cluster.

[0020] In terms of processing jobs within a distributed computing cluster, the system's ability to meet SLAs may be hampered by mismanagement of the throughput and latency of jobs within the system. In practice, high throughput and low latency cannot always be achieved at once - a straightforward approach to achieving high throughput within a distributed computing cluster involves collecting a large number of messages in a compute node's buffer before communicating the contents of the buffer to other nodes. The problem then arises that, while this achieves high throughput, it may take a long time to fill the buffer, thus resulting in high latency. Similarly, the computing cluster can be configured to send messages immediately upon receipt in order to achieve low latency, however this will also result in low throughput. The aspects described herein are configured to address the problem of balancing desired throughput and latency in a manner that meets the SLA required for the job.

[0021] In some aspects, optimized communications in a distributed computing network implement a "call cluster component" that acts as a gatekeeper at the entry point of a job into the distributed computing cluster. Various aspects use the call cluster component to prevent a job from entering the system if the SLA associated with the job cannot be met. Furthermore, in some examples, the call cluster component advantageously exhibits a preference for allowing high priority jobs to enter the system over lower priority jobs. Given that once there is no concept of "priority" for a job within the system, a way for a high priority job to miss its SLA is if the system is overwhelmed by lower priority jobs. By providing a preference for high priority jobs upon entry into the system, the system is more likely to meet the SLA for that job.

[0022] Certain aspects advantageously implement synchronized internal clocks across all compute nodes in a distributed computing cluster. To achieve the desired throughput and latency for jobs within the system relative to SLAs, the system instructs its compute nodes to communicate their content to other compute nodes after a certain time interval indicated by the internal clock. In doing so, the system advantageously achieves the desired throughput while ensuring that its latency never exceeds the predetermined time interval.

[0023] Other features and advantages of the invention will become apparent from the following description and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is an exemplary data processing system.

[0025] Figure 2 It is a depiction of the execution environment including the distributed computing cluster and the controller.

[0026] Figure 3 is an example of a control algorithm implemented by a controller.

[0027] Figure 4 In the first example of the operation of the execution environment Figure 2 The initial state of the execution environment.

[0028] Figure 5 yes Figure 2 The first step in a first example of operation of an execution environment.

[0029] Figure 6 yes Figure 2 The execution environment is operated in the second step of the first example.

[0030] Figure 7 yes Figure 2 The execution environment is operated as the third step in the first example.

[0031] Figure 8 yes Figure 2The fourth step of the first example of the operation of the execution environment.

[0032] Figure 9 yes Figure 2 The fifth step of the first instance of operation of the execution environment.

[0033] Figure 10 In the second example of the operation of the execution environment Figure 2 The initial state of the execution environment.

[0034] Figure 11 yes Figure 2 The first step in a second example of the operation of the execution environment.

[0035] Figure 12 yes Figure 2 The second step in the second example of the operation of the execution environment.

[0036] Figure 13 yes Figure 2 The third step of the second example of the operation of the execution environment. DETAILED DESCRIPTION

[0037] Figure 1 An example of a data processing system 100 in which communication optimization techniques may be used is shown. System 100 includes data source 102, which may include one or more data sources such as a storage device or a connection to an online data stream, each of which may store or provide data in any of a variety of formats, such as database tables, spreadsheet files, flat text files, or native formats used by mainframe computers.

[0038] The execution environment 104 includes a distributed computing cluster 112 (sometimes referred to as an "active data cluster"), as described in more detail below, that can be used by one or more client systems 106 to perform computing tasks. An example of a distributed computing cluster is the distributed computing cluster described in U.S. Patent Application No. 16 / 175,133, which is incorporated herein by reference. The execution environment 104 can be hosted on one or more general-purpose computers, for example, under the control of a suitable operating system (such as a version of the UNIX operating system). For example, the execution environment 104 can include a multi-node parallel computing environment that includes a configuration of a computer system using multiple central processing units (CPUs) or processor cores, the multiple central processing units or processor cores being local (e.g., a multi-processor system such as a symmetric multiprocessing (SMP) computer) or locally distributed (e.g., multiple processors coupled as a cluster or massively parallel processing (MPP) system), or remote or remotely distributed (e.g., multiple processors coupled via a local area network (LAN) and / or a wide area network (WAN)), or any combination thereof.

[0039] In some examples, execution environment 104 reads data from data source 102 and stores the results back in data source 102 or provides the results to other downstream systems for further processing. The storage providing data source 102 can be local to execution environment 104, for example, stored on a storage medium (e.g., hard drive 108) connected to the computer hosting execution environment 104, or can be remote to execution environment 104, for example, hosted on a remote system (e.g., mainframe 110) that communicates with the computer hosting execution environment 104 via a remote connection (e.g., provided by a cloud computing infrastructure).

[0040] The data storage system 116 may also be accessed by a development environment 118 in which a developer 120 may specify a computer program for execution in the execution environment (e.g., by the distributed computing cluster 112 via one or more client systems 106). In some implementations, the development environment 118 is a system for developing applications as data flow graphs, which include vertices (representing data processing components or data sets) connected by directed links between the vertices (representing flows of job elements (i.e., data)). Such an environment is described in more detail, for example, in U.S. Publication No. 2007 / 0011668, entitled “Managing Parameters for Graph-Based Applications,” which is incorporated herein by reference. A system for performing such graph-based computations is described in U.S. Patent No. 5,966,072, entitled “Executing Computations Expressed as Graphs,” which is incorporated herein by reference. Data flow graphs produced according to this system provide methods for transferring information into and out of the various processes represented by the graph components, for moving information between processes, and for defining the order in which the processes are run. The system includes algorithms for selecting an inter-process communication method from any available methods (e.g., the communication paths along the graph's links may use TCP / IP or UNIX domain sockets, or use shared memory to pass data between processes).

[0041] In some examples, data source 102 maintains data in any number of different forms of database systems. The data can be organized into records with values ​​(including possible null values) for corresponding fields (also called "attributes" or "columns"). When first reading data from a data source, execution module 104 typically starts with some initial format information about the records in the data source. In some cases, the record structure of the data source may not be known initially, but can be determined after analyzing the data source or the data. The initial information about the record may include, for example, the number of bits representing different values, the order of the fields within the record, and the type of value represented by the bits (e.g., string, signed / unsigned integer).

[0042] 1. Overview of the Execution Environment

[0043] refer to Figure 2, a general overview of one example of an execution environment 104 includes a distributed computing cluster 112, a controller 110, a throttle 117, and a monitor 119. Very generally, the controller 110 and the distributed computing cluster 112 operate in a feedback loop to process a stream of input computing requests 111 provided by an upstream component (e.g., a “calling cluster component” or “interface component” not shown) for injection into the cluster 112, and generate a stream of output data elements 113, which may be responses generated by the nodes 105 of the distributed computing cluster 112 in response to the computing requests 111 injected into the distributed computing cluster 112, in a manner that maximizes the computing throughput of the distributed computing cluster 112 while complying with any service level agreements or requirements associated with the distributed computing cluster 112.

[0044] In some examples, controller 110 receives data from monitor 119 that characterizes the processing latency and output rate R associated with distributed computing cluster 112. out The controller 110 uses the processing delay and output rate R out To determine the communication delay d (sometimes referred to as the “first parameter”) of messages sent within the distributed computing cluster 112 and the rate R at which requests are injected into the distributed computing cluster 112 in (Sometimes referred to as a "second parameter"). The output rate may be the rate at which cluster 112 outputs output data elements 113, which may be responses from node 105 to injected requests 111. The output rate R may be measured by monitor 119. out The communication delay d is provided to the distributed computing cluster 112 and used to configure the computing nodes 105 in the distributed computing cluster, as described in more detail below. in is provided to the throttle 117 and is used to limit the rate at which new requests are injected into the distributed computing cluster 112 .

[0045] In some examples, input request 111 comprises a request to process data according to a procedure defined in a component (e.g., a calling cluster component) external to distributed computing cluster 112. Distributed computing cluster 112 processes input request 111 and provides a result 113 of the processing (a response from node 105 to the injected computing request 111) to a component external to distributed computing cluster 112 (e.g., back to the calling cluster component).

[0046] The throttle 117 is configured to limit the rate R of injection of requests into the distributed computing cluster 112. inVery generally, the request injection rate parameter is adjustable to ensure that requests are not injected into the distributed computing cluster 112 if those requests are unlikely to complete within the amount of time specified by the SLA associated with the request.

[0047] 2Distributed Computing Cluster

[0048] Distributed computing cluster 112 includes a plurality of computing nodes 105 connected to one another via an interconnection network 107 (or simply, the "interconnect"). Interconnect 107 facilitates data communication between computing nodes 105 for processing data in a distributed manner. For example, interconnect 107 includes an Internet Protocol (IP)-based and / or Ethernet-based network. In general, processing requests are injected into distributed computing cluster 112 by one or more external components (not shown). Distributed computing cluster 112 processes the requests using computing nodes 105 and interconnection network 107 to generate result data (e.g., responses from nodes 105 to the injected computing requests), which is provided back to the external components.

[0049] In the process of processing requests, messages are frequently passed between computing nodes 105 via interconnect 107. Communication delay d indicates how long computing nodes 105 in distributed computing cluster 112 wait between transmitting messages to each other via interconnect 107. Larger values ​​of communication delay cause computing nodes 105 to wait for longer durations between transmissions. While computing nodes 105 wait, they accumulate a group of messages for later transmission (e.g., in a single network packet). Therefore, larger values ​​of communication delay are associated with more messages sent per transmission, resulting in greater throughput for distributed computing cluster 112 (i.e., due to reduced network overhead associated with packet formation). However, longer wait durations also result in greater computational latency. Similarly, smaller values ​​of communication delay d are associated with lower throughput for the distributed computing cluster (i.e., due to increased network overhead associated with the increased number of packets being formed), but are also associated with less computational latency.

[0050] 3 controllers

[0051] As described above, the controller 110 implements a feedback control mechanism to maximize throughput in the distributed computing cluster 112 while ensuring compliance with service level agreements (SLAs) or other commitments associated with the requests. The controller 110 uses the monitor 119 to monitor the processing latency and output rate R associated with the distributed computing cluster 112. out Controller 110 receives delay, output rate R out and the service level agreement SLA as input, and processes these inputs to determine the input rate R of the distributed computing cluster 112 in and communication delay d.

[0052] The input rate R determined by the controller 110 in is provided to the throttle 117, which maintains the rate at which requests 111 are injected into the distributed computing cluster 112 at R in The communication delay d determined by the controller 110 is provided to the distributed computing cluster 112 , which maintains a time interval between message transmissions over the interconnect 107 of the distributed computing cluster 112 in accordance with the communication delay d.

[0053] 3.1 Control Algorithm

[0054] refer to Figure 3 , an example of a control algorithm implemented by the controller 110 begins at line 1 by starting a while loop. On line 3, the first operation of the process implemented in the while loop calls the adjust(d) function. The adjust(d) function defined on lines 13-16 modifies the value of the communication delay d while monitoring the output rate R out , the goal is to reach the value d*, which is the value obtained by further increasing d without increasing R out The lowest value of d in the case of .

[0055] After the adjust(d) function returns, the algorithm implemented by the controller 110 proceeds to line 4 where a test is performed to determine the output rate R from the distributed computing cluster. out Is it equal to the rate R at which requests are injected into the distributed computing cluster? in (ie, does the distributed computing cluster currently have sufficient throughput to keep up?), and whether the difference between the SLA and the latency is greater than zero (ie, is the distributed computing cluster exceeding the SLA?).

[0056] In line 5 of the algorithm, if the conditions set forth in line 4 are met, the controller 110 will request an injection rate R in At line 6 of the algorithm, if one or more of the conditions set forth in line 4 of the algorithm are not met, the controller 110 will request an injection rate R in That is, if the distributed computing cluster 112 is currently keeping up with and exceeding the SLA, the algorithm attempts to increase the input rate R in Otherwise, if the distributed computing cluster 112 fails to keep up with its processing load or fails to meet the SLA, the controller 110 reduces the input rate R in .

[0057] Then, loop with R in The updated version in the previous iteration of the loop is repeated. The adjust(d) function call in the current iteration of the loop modifies d to reach the lowest value of d where Rin An updated version of this, where further increasing d at this lowest value of d does not increase R out If the adjust(d) function obtains a value of d, where the value of d makes R out Equal to R in The algorithm increases R again if the updated value of R is set and the difference between SLA and delay is greater than zero. in Otherwise, the algorithm reduces R in .

[0058] By monitoring the latency and throughput of the distributed computing cluster 112, Figure 3 The algorithm maximizes throughput in the distributed computing cluster 112 while ensuring that a service level agreement (SLA) or other commitment associated with the request is not violated.

[0059] 4 Example 1

[0060] refer to Figure 4-Figure 9 , a first exemplary operation of the execution module 104 illustrates the control of the distributed computing cluster 112 by the controller 110 .

[0061] refer to Figure 4 , in the initial state, the rate R of injecting requests into the distributed computing cluster 112 in is equal to 2000 (eg, messages / second), and the output rate R of the distributed computing cluster 112 is out Equal to 2000 (e.g., messages / second). in and R out Equality indicates that the distributed computing cluster 112 has sufficient throughput to keep up with the request injection rate R in The monitored latency of distributed computing cluster 112 is 40ms, and the communication delay d is equal to 20ms. Controller 110 must maintain a 70ms SLA for request 111.

[0062] refer to Figure 5 , in the first iteration of the control algorithm, the controller 110 executes the adjust(d) function at line 2 of the control algorithm, and d is determined to be already at a minimum value where further increasing d does not increase R out The controller 110 then proceeds to line 4 of the control algorithm, where it determines R out Equal to R in (ie, 2000=2000) and the delay is less than the SLA (ie, 40ms<70ms). Since the condition in line 4 is met, the controller 110 proceeds to line 5 of the control algorithm and sets R in Increased from 2000 to 3000. R in The increase in R outIncreased to 2500 (ie, the current configuration of the distributed computing cluster 112 does not have sufficient throughput to keep up with the increased request injection rate R in ).

[0063] refer to Figure 6 , in the second iteration of the control algorithm, the controller 110 executes the adjust(d) function at line 2 of the control algorithm and increases d to 29ms, which results in R out After d increases, the distributed computing cluster 112 is configured to have sufficient throughput to keep up with the increased request injection rate R in The increase in d also causes the latency of the distributed computing cluster to increase to 60ms.

[0064] refer to Figure 7 , the second iteration of the control algorithm proceeds to line 4 of the control algorithm, where it is determined to be R out Equal to R in (ie, 3000=3000) and the delay is less than the SLA (ie, 60ms<70ms). Since the condition in line 4 is met, the controller 110 proceeds to line 5 of the control algorithm and sets R in Increased from 3000 to 4000. R in The increase in R out Increased to 3500 (ie, the current configuration of the distributed computing cluster 112 does not have sufficient throughput to keep up with the increased request injection rate R in ).

[0065] refer to Figure 8 , in the third iteration of the control algorithm, the controller 110 executes the adjust(d) function and increases d to 38ms, which results in R out After d increases, the distributed computing cluster 112 is configured to have sufficient throughput to keep up with the increased request injection rate R in The increase in d also causes the latency of the distributed computing cluster to increase to 80ms.

[0066] refer to Figure 9 , in the third iteration of the control algorithm, the controller 110 proceeds to line 4 of the control algorithm and determines at line 4 that R out Equal to R in (i.e., 3000=3000) and the delay is greater than the SLA (i.e., 80ms>70ms), thus violating the SLA. Since the condition in line 4 is not met, the controller 110 proceeds to line 8 of the control algorithm and sets R in Reduced from 4000 to 3000. R outIt decreases to 300, and the delay is reduced to 60ms (no longer violating the SLA).In the fourth iteration of the control algorithm, the controller 110 proceeds to line 3 and executes the adjust(d) function, which reduces d to 29ms.

[0067] The controller 110 continues to execute the above Figure 3 to maximize throughput in the distributed computing cluster 112 while ensuring that a service level agreement (SLA) associated with the request 111 is not violated.

[0068] 5Example 2

[0069] refer to Figure 10-13 In the second example, the configuration of the distributed computing cluster 112 changes, and the controller 110 executes Figure 3 The algorithm can adapt to the configuration changes. For example, refer to Figure 10 , two of the computing nodes 105 fail, resulting in a reduction in the computing capacity of the distributed computing cluster 112.

[0070] As the number of computing nodes 105 in the distributed computing cluster 112 decreases from five to three, the output rate R out From 3000 to 2000, while the input rate R in Keep it at 3000. The delay increases to 85ms.

[0071] refer to Figure 11 , the controller 110 executes line 3 of the control algorithm, executes the adjust(d) function and increases d to 100ms, which results in R out After d increases, the distributed computing cluster 112 still does not have enough throughput to keep up with the increased request injection rate R in The increase in d also causes the latency of the distributed computing cluster to increase to 120ms.

[0072] refer to Figure 12 , the controller 110 proceeds to line 4 of the control algorithm and determines at line 4 that R out Not equal to R in (i.e. 2500!=3000) and the delay is greater than the SLA (i.e. 120ms>70ms), violating the SLA. Since the condition in line 4 is not met, the controller 110 proceeds to line 8 of the control algorithm and sets R in Reduced from 3000 to 2000. R out to 2000 (i.e., the current configuration of the distributed computing cluster 112 has sufficient throughput to keep up with the increased request injection rate R in ). The latency is reduced to 90ms.

[0073] refer to Figure 13 , in subsequent iterations of the while loop, the controller 110 executes line 3 of the control algorithm, executes the adjust(d) function, and again reaches a value of 40ms for d, which causes R out Keep it at 2000. Figure 13 The distributed computing cluster is configured with sufficient throughput to keep up with the reduced request injection rate R in The latency of the distributed computing cluster is reduced to 60ms.

[0074] exist Figure 13 In the configuration of , controller 110 has adapted the system to maximize throughput in the new configuration of distributed computing cluster 112 while ensuring that a service level agreement (SLA) associated with request 111 is not violated.

[0075] 6 Alternatives

[0076] exist Figures 6 to 13 In the description of FIG, the controller 110 has adapted the system to maximize the throughput in the new configuration of the distributed computing cluster 112 while ensuring that the service level agreement (SLA) associated with the request 111 is not violated. However, instead of maximizing the throughput, the throughput may be controlled so that when it is determined that the service level arrangement is not violated and when it is determined that R in and R out When equal (indicating that the distributed computing cluster 112 has sufficient throughput to keep up with the request injection rate R in ), no further increase in throughput is made (e.g., Figure 5 in the scenario shown).

[0077] also, Figure 3 The control algorithm in is only exemplary and adjusts R at a coarse granularity. in and d, but it should be noted that in some examples, the control algorithm uses finer granularity adjustments, and in some cases variable granularity, to adjust R in and d.

[0078] The adjust(d) function described above may be implemented using any of a number of known optimization techniques, such as gradient descent or stochastic optimization techniques.

[0079] While the above examples illustrate injecting requests from only a single external component interacting with the distributed data cluster (e.g., a calling cluster component), it's possible that several external components will be interacting with the distributed data cluster simultaneously. In this case, the controller monitors the latency and throughput of many or all external components and balances their access to the distributed data cluster's resources to ensure that throughput is maximized and the SLAs for requests from the external components are met.

[0080] In some examples, input data arriving at an external component (eg, a call cluster component) is prioritized, and requests are injected according to the priority level.

[0081] In some examples, latency is measured at the service request level (e.g., remote procedure call latency), or simply by the time it takes between injecting a request into a distributed data cluster and receiving a response.

[0082] In some examples, throughput is measured by monitoring CPU load or the number of messages "in flight."

[0083] In some examples, each compute node maintains a timer and decides when to send a message over the interconnect based on the timer and a communication delay d. In other examples, a centralized clock periodically distributes a trigger signal, where the period is defined as the communication delay d. When the trigger signal is received at a compute node, the trigger signal causes the compute node to send a message.

[0084] Although not shown in the figures, in some examples, the output of throttle 117 is fed through monitor 119 to help determine processing latency of distributed computing cluster 112 .

[0085] As described above, a distributed computing cluster interacts with one or more external components (sometimes referred to as "call cluster components" or "interface components"). The external components can be associated with corresponding data processing processes. In some examples, the external components receive input data for processing according to their corresponding data processing processes. The external components use the input data to form processing requests and provide the processing requests to the distributed computing cluster. The distributed computing cluster processes the requests using the control scheme described above and generates outputs. The outputs are provided back to the external components. The external components process the outputs to generate output data, which the external components send to one or more downstream components.

[0086] 7 Implementation

[0087] The above methods can be implemented, for example, using a programmable computing system that executes appropriate software instructions, or can be implemented in suitable hardware (such as a field programmable gate array (FPGA)), or in some hybrid form. For example, in a programming method, the software can include processes in one or more computer programs executed on one or more programming or programmable computing systems (which can be of various architectures, such as distributed, client / server, or grid), each computing system including at least one processor, at least one data storage system (including volatile and / or non-volatile memory and / or storage elements), at least one user interface (for receiving input using at least one input device or port, and for providing output using at least one output device or port). The software can include, for example, one or more modules of a larger program that provides services related to the design, configuration, and execution of data flow graphs. The modules of the program (e.g., elements of the data flow graph) can be implemented as data structures or other organized data that conform to a data model stored in a data repository.

[0088] The software may be stored in a non-transitory form (such as embodied in a volatile or non-volatile storage medium or any other non-transitory medium) for a period of time (e.g., the time between refresh cycles of a dynamic memory device (such as dynamic RAM)) using the physical properties of the medium (e.g., surface pits and lands, magnetic domains, or charge). The software may be provided on a tangible, non-transitory medium (such as a CD-ROM or other computer-readable medium (e.g., readable by a general-purpose or special-purpose computing system or device)) in preparation for loading instructions, or on a tangible, non-transitory medium that may be delivered (e.g., encoded in a propagating signal) over the communication medium of a network to the computing system where it is executed. Some or all of the processing may be performed on a dedicated computer or using dedicated hardware (such as a coprocessor or field programmable gate array (FPGA) or a dedicated application-specific integrated circuit (ASIC)). The processing may be implemented in a distributed manner (in which different parts of the computation specified by the software are performed by different computing elements). Each such computer program is preferably stored or downloaded onto a computer-readable storage medium (e.g., solid-state memory or medium, or magnetic or optical medium) of a storage device accessible by a general or special purpose programmable computer for configuring and operating the computer when the computer reads the storage device medium to perform the processes described herein. The system of the present invention may also be considered to be implemented as a tangible, non-transitory medium configured with a computer program, wherein the medium so configured causes the computer to operate in a specific and predefined manner to perform one or more of the process steps described herein.

[0089] Several embodiments of the present invention have been described. However, it should be understood that the foregoing description is intended to illustrate and not to limit the scope of the present invention, which is defined by the scope of the appended claims. Therefore, other embodiments are also within the scope of the appended claims. For example, various modifications may be made without departing from the scope of the present invention. In addition, some of the above steps may be sequence-independent and, therefore, may be performed in an order different from that described.

Claims

1. A computing system (104), comprising: A distributed computing cluster (112) comprising: a plurality of computing nodes (105) interconnected by an interconnection network (107), computing nodes of the plurality of computing nodes communicating with each other via message passing via the interconnection network, at least some of the computing nodes each being configured with first parameters governing message transmission by the computing nodes via the interconnection network, wherein said at least some computing nodes are configured to accumulate messages for transmission as a group of messages according to said first parameter, wherein the computing system is configured to limit injection of computing requests into the distributed computing cluster according to a second parameter; The computing system further includes: a controller configured to receive at least one predetermined service level requirement and control a value of the second parameter and a value of the first parameter to control a computational throughput of the distributed computing cluster while complying with the at least one predetermined service level requirement, wherein the distributed computing cluster comprises a rate limiter (117) configured to limit a rate at which the computing request is injected into the distributed computing cluster according to the second parameter, and The computing system further comprises a monitor (119), which is configured to observe the output rate of the distributed computing cluster and the processing delay of the distributed computing cluster.

2. The computing system of claim 1, wherein: Each of the one or more service level requirements specifies a maximum allowable time difference between injecting a computing request into the distributed computing cluster and receiving a response to the injected computing request from the distributed computing cluster.

3. The computing system of claim 1 , wherein: The observed processing latency is measured by determining a time difference between injecting a computation request into the distributed computation cluster and receiving a response to the injected computation request from the distributed computation cluster.

4. The computing system of claim 1, wherein: Requests injected into the distributed computing cluster and results output by the distributed computing cluster are passed through the monitor.

5. The computing system of claim 1 , wherein: The computational throughput is observed using the monitor by measuring the rate at which data elements are output by the cluster. The computing system according to claim 1 , wherein: The controller is configured to process observed computational throughput and observed processing latency of the distributed computing cluster to adjust a value of the second parameter and a value of the first parameter.

7. The computing system of claim 6, wherein: The controller repeatedly processes the observed computation throughput and the observed processing latency of the distributed computing cluster to adjust the value of the second parameter and the value of the first parameter while the distributed computing cluster is processing the injected computation request.

8. The computing system of claim 6, wherein: The controller implements a feedback loop with the distributed computing cluster to process observed computing throughput and observed processing latency of the distributed computing cluster, thereby adjusting the value of the second parameter and the value of the first parameter.

9. The computing system according to any one of claims 1 to 8, wherein: The controller uses an optimization algorithm to control the value of the first parameter and / or the value of the second parameter to maximize the computing throughput of the distributed computing cluster while complying with the at least one predetermined service level requirement.

10. The computing system according to any one of claims 1 to 8, wherein: The first parameters include a delay parameter that governs a minimum time interval between transmissions of messages by the computing nodes over the interconnection network.

11. The computing system of claim 10, wherein: The at least some computing nodes are configured to accumulate messages for transmission as a group of messages during the minimum time interval.

12. The computing system according to any one of claims 1 to 8, wherein: The second parameter includes an input rate parameter configured to limit a rate at which computing requests are injected into the distributed computing cluster.

13. The computing system according to any one of claims 1 to 8, wherein: Controlling the computational throughput of the distributed computing cluster while complying with the at least one predetermined service level requirement includes maximizing the computational throughput of the distributed computing cluster without violating any of the at least one predetermined service level requirement.

14. The computing system according to any one of claims 1 to 8, wherein: The first parameter governs the number of messages that accumulate between transmissions of messages by the compute node through the interconnection network.

15. The computing system according to any one of claims 1 to 8 further includes one or more interface components, each interface component being configured to receive input data, inject computing requests into the distributed computing cluster to process the input data, receive processing results from the distributed computing cluster, and generate output data based on the processing results, and the interface component being configured to limit the rate at which computing requests are injected into the distributed computing cluster according to the second parameter.

16. The computing system of claim 15, wherein: At least some of the one or more interface components execute on a client system (106) that is separate from a system executing the distributed computing cluster.

17. The computing system of claim 15, wherein: At least some of the received input data is associated with a priority level, and the one or more interface components are configured to manage injection of the input data into the distributed computing cluster according to the priority level.

18. A method for operating a computing system, the computing system comprising a distributed computing cluster, the method comprising: Injection of computing requests into the distributed computing cluster is restricted according to a second parameter, wherein the distributed computing cluster comprises: a plurality of computing nodes (105) interconnected by an interconnection network (107), computing nodes of the plurality of computing nodes communicating with each other via message passing via the interconnection network, at least some of the computing nodes each being configured with a first parameter governing message transmission by the computing nodes via the interconnection network, accumulating, at said at least some computing nodes, messages for transmission as a group of messages according to said first parameter, receiving at least one predetermined service level requirement at a controller, and controlling, using the controller, a value of the second parameter and a value of the first parameter to control a computational throughput of the distributed computing cluster while complying with the at least one predetermined service level requirement, The method further includes using a rate limiter to limit the rate of injecting the computing request into the distributed computing cluster according to the second parameter, and using a monitor to observe the output rate of the distributed computing cluster and the processing delay of the distributed computing cluster.

19. A computer-readable medium storing software in non-transitory form, the software comprising instructions for causing a computing system to: The rate at which computing requests are injected into the distributed computing cluster is limited according to a second parameter, wherein Distributed computing clusters include: a plurality of computing nodes (105) interconnected by an interconnection network (107), computing nodes of the plurality of computing nodes communicating with each other via message passing via the interconnection network, at least some of the computing nodes each being configured with a first parameter governing message transmission by the computing nodes via the interconnection network, accumulating, at said at least some computing nodes, messages for transmission as a group of messages according to said first parameter, receiving at least one predetermined service level requirement at a controller, and controlling, using the controller, a value of the second parameter and a value of the first parameter to control a computational throughput of the distributed computing cluster while complying with the at least one predetermined service level requirement, The software further includes instructions for using a rate limiter to limit the rate of injecting the computing request into the distributed computing cluster according to the second parameter, and using a monitor to observe the output rate of the distributed computing cluster and the processing delay of the distributed computing cluster.

20. A computing system comprising: means for limiting a rate of injection of computing requests into a distributed computing cluster according to a second parameter, wherein the distributed computing cluster comprises: a plurality of computing nodes (105) interconnected by an interconnection network (107), computing nodes of the plurality of computing nodes communicating with each other via message passing via the interconnection network, at least some of the computing nodes each being configured with a first parameter governing message transmission by the computing nodes via the interconnection network, means for accumulating, at said at least some computing nodes, messages for transmission as a group of messages in accordance with said first parameter, means for receiving at the controller at least one predetermined service level agreement, means for controlling, using the controller, a value of the second parameter and a value of the first parameter to control a computational throughput of the distributed computing cluster while complying with at least one service level requirement, and A component for observing the output rate of the distributed computing cluster and the processing delay of the distributed computing cluster.

21. A computer program product comprising a computer program, which, when executed by a processor, implements the steps of the method according to claim 18.

Images