System and method for input / output distribution

Abstract

Embodiments of the present disclosure relate to systems and methods for input / output schedule dispatch. The decoupling computing system includes layers of computing resources of the same type, and includes assigning layers, such as input and output (I / O) streams, for assigning tasks from one layer to another. The I / O stream may be assigned to particular computing resources of a layer based on a weighted moving average of performance data for that layer. When traffic is high, assigning may include randomly assigning to some or all of the computing resources in the layer. The I / O stream may be divided into read-dense streams and write-dense streams, wherein more read-dense streams are assigned based on a selection ratio.

Description

Technical Field

[0001] The present disclosure relates to systems and methods for dispatching input and output (I / O) requests in a distributed computing system. Background Art

[0002] Even in distributed computing systems, resources tend to be coupled to each other, such as in the form of servers that each include memory, a central processing unit (CPU), a graphics processing unit (GPU), etc. The coupling of resources can limit flexibility and scalability because resources are scaled as a complete coupled system that includes each of the resource classes. Scaling using coupled units does not necessarily correspond to demand for a particular individual resource type. In addition, queuing of input and output (I / O) requests is often based on fairness principles or other such heuristics and / or a priori determinations of performance. Summary of the invention

[0003] The present disclosure relates to systems and methods for dispatching input and output (I / O) requests in a distributed computing system.

[0004] By allocating resources into decoupled pooling layers of the same type of resources, resources can be efficiently assigned to corresponding computing tasks in a manner that can be viewed as flexible and scalable. By prioritizing read-intensive I / O streams over write-intensive I / O streams, the impact of write-intensive I / O streams on latency can be relatively reduced, thereby maintaining the order of operations and improving the performance of the computing system. By using a weighted moving average within a window to determine the performance for making dispatch decisions, the most relevant data can be used to make dispatch decisions and reduce the computational overhead caused by attempting to consider outdated performance data. By adding constrained and / or unconstrained random assignments of I / O streams to resources based on traffic levels, large volumes and peaks in traffic can be efficiently assigned without excessively overloading individual resources.

[0005] In one embodiment, a distributed computing system includes a first plurality of decoupled computing resources of a first type and a second plurality of decoupled computing resources of a second type, the first plurality of decoupled resources being a first layer of distributed computing resources. The second plurality of decoupled resources are a second layer of the distributed computing system. The second layer is configured to receive operations provided by the first layer. The distributed computing system also includes a dispatching layer. The dispatching layer is configured to assign at least some of the operations provided by the first layer to at least some of the computing resources of the second plurality of decoupled computing resources of the second layer based on a weighted moving average of the states of at least some of the second plurality of decoupled computing resources over time.

[0006] In one embodiment, the first layer is a client layer and the second layer is a cache layer. The second plurality of decoupled computing resources are corresponding storage memories. The operation is an input / output (I / O) stream. In one embodiment, the corresponding storage memory includes a dynamic random access memory (DRAM). In one embodiment, the corresponding storage memory is a solid state drive (SSD). In one embodiment, the dispatch layer is configured to divide the I / O stream into a read-intensive queue and a write-intensive queue based on the write percentage of each I / O stream in the I / O stream. In one embodiment, the dispatch layer is configured to assign the I / O stream of the read-intensive queue and the I / O stream of the write-intensive queue according to a pick ratio, and the pick ratio is the ratio of multiple I / O streams of the read-intensive queue to be assigned for each I / O stream in the I / O stream of the write-intensive queue to be assigned.

[0007] In one embodiment, the weighted average is an exponentially weighted moving average.

[0008] In one embodiment, the dispatch layer is configured to determine an amount of incoming operational traffic. In one embodiment, the dispatch layer is configured to compare the determined amount of incoming operational traffic to a first threshold and a second threshold. When the amount of incoming operational traffic is above the first threshold and below the second threshold, the dispatch layer is configured to randomly assign operations provided by the first layer to a subset of the second plurality of computing resources, the subset being selected based on a weighted moving average of states of the second plurality of decoupled computing resources over time. In one embodiment, the one or more processors are configured to randomly assign operations provided by the first layer to the entire second plurality of computing resources when the amount of incoming operational traffic is above the second threshold.

[0009] In one embodiment, a method for assigning operations to computing resources includes obtaining a weighted average of states over time for multiple decoupled computing resources, receiving one or more operations to be assigned to the multiple decoupled computing resources, and assigning the one or more operations to at least some of the multiple decoupled computing resources based on the weighted average of states over time for the multiple decoupled computing resources.

[0010] In one embodiment, the weighted average is an exponentially weighted average.

[0011] In one embodiment, the operation is an input / output (I / O) stream, and the method further includes dividing the I / O streams into a read-intensive queue and a write-intensive queue based on a write percentage of each of the I / O streams. In one embodiment, the I / O streams of the read-intensive queue and the I / O streams of the write-intensive queue are assigned according to a pick ratio, and the pick ratio is a ratio of a plurality of I / O streams of the read-intensive queue to be assigned to each of the I / O streams in the write-intensive queue to be assigned.

[0012] In one embodiment, the method further includes determining an amount of incoming operational traffic.

[0013] In one embodiment, the method further comprises comparing the amount of incoming operational traffic to a first threshold and a second threshold. When the amount of incoming operational traffic is above the first threshold and below the second threshold, the method comprises randomly assigning operations provided by the first layer to a subset of the second plurality of computing resources, the subset being selected based on a weighted moving average of states of the second plurality of decoupled computing resources over time. In one embodiment, when the amount of incoming operational traffic is above the second threshold, the operations provided by the first layer are randomly assigned to the entire second plurality of computing resources.

[0014] In one embodiment, a data center includes a first plurality of decoupled computing resources of a first type, a second plurality of decoupled computing resources of a second type, and a dispatch layer configured to assign at least some operations provided by the first layer to at least some computing resources of the second plurality of decoupled computing resources of the second layer based on a weighted moving average of states of at least some of the second plurality of decoupled computing resources over time.

[0015] In one embodiment, the second type of decoupled computing resource is a cache storage memory.In one embodiment, the operation is an I / O stream, and the dispatch layer is configured to divide the I / O stream into a read-intensive queue and a write-intensive queue based on a write percentage of each I / O in the I / O stream. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of a distributed computing system according to one embodiment is shown.

[0017] Figure 2 A schematic diagram of cache and cloud storage layers is shown according to one embodiment.

[0018] Figure 3 A schematic diagram of a dispatcher of a dispatch layer according to one embodiment is shown.

[0019] Figure 4 A flow chart of a method for determining performance volatility according to one embodiment is shown.

[0020] Figure 5 A flow chart for determining and responding to flow levels according to one embodiment is shown.

[0021] Figure 6 A flow chart of a method for assigning operations to computing resources according to one embodiment is shown.

[0022] Figure 7 A flow chart of a method for scaling operations according to one embodiment is shown.

[0023] Figure 8 A flow chart of a method for determining peak variability according to one embodiment is shown.

[0024] Fig. 9 A flow chart of a method for determining a scale of an operation according to one embodiment is shown.

[0025] Fig.10 A schematic block diagram of an example computer system suitable for implementing an electronic device, arranged in accordance with at least some embodiments described herein, is shown. DETAILED DESCRIPTION

[0026] The present disclosure is directed to systems and methods for dispatching input and output (I / O) requests in a distributed computing system.

[0027] As used herein, "decoupled" refers to computing resources that are not directly connected to other types of computing resources in a particular computing device (such as a server). Decoupled computing resources can be indirectly connected to other types of computing resources through connections between layers, etc.

[0028] As used herein, "same type" means that all computing resources share a particular functionality, such as a CPU, GPU, storage memory, cache memory, dispatcher, etc. In one embodiment, the computing resources referred to as "same type" may include multiple different components with the same functionality, such as different models of CPUs or GPUs, different sizes or types of memory for storage or cache memory, etc. In one embodiment, at least some of the computing resources referred to as "same type" may be identical to each other, such as the same models of CPUs, GPUs, memory devices, etc.

[0029] Figure 1 A schematic diagram of a distributed computing system according to one embodiment is shown. Distributed computing system 100 includes client layer 102, dispatch layer 104, cache layer 106, and cloud storage layer 108. Although illustrated as discrete components, various components may be divided into additional components, combined into fewer components, or eliminated entirely while being considered within the scope of the disclosed subject matter. It will be understood by those skilled in the art that each function and / or operation of a component may be implemented individually and / or collectively by various hardware, software, firmware, or any combination thereof.

[0030] Distributed computing system 100 is a data center that includes decoupled resources that are pooled into corresponding layers. The layers can each include the same type of computing resources. Distributed computing system 100 can provide any suitable functions of a data center, including, as a non-limiting example, storing and processing data.

[0031] The client layer 102 is a layer that includes client devices 110 that interface with the distributed computing system 100. The client device 110 can interface with the distributed computing system 100 through one or more suitable libraries and / or application programming interfaces (APIs). The client device can connect to the nearest proxy provided in the distributed computing system 100 through one or more domain name servers (DNS), which are configured to enable discovery of the nearest proxy. In one embodiment, the DNS can provide multiple proxy addresses, allowing the client to switch proxies, for example, in the event of a proxy timeout. In one embodiment, the DNS can support load balancing and / or failure recovery for the proxy.

[0032] Figure 1 In the illustrated example embodiment, the dispatching layer 104 is configured to direct I / O streams from the client device 110 to specific cache resources of the cache layer 106. The dispatching layer 104 includes a plurality of dispatchers 112. Each dispatcher in the dispatchers 112 is configured to receive an I / O stream from one or more of the client devices 110 and direct the I / O stream to a specific cache device 116 of the cache layer 106. In one embodiment, the I / O stream is directly assigned to the cache device 116 of the cache layer 106. In one embodiment, the I / O stream may be assigned to a queue for the I / O stream, through which the I / O stream proceeds to the specific cache device 116 of the cache layer 106. In one embodiment, the dispatching layer 104 may be provided between appropriate layers of the distributed computing system 100, so that any operation assigned by one layer may be assigned to resources of another appropriate layer. Dispatching by the dispatching layer 104 may allow the cache layer 106 to provide a stateless cache. In one embodiment, such dispatching can simplify the cache protocol, thereby reducing computational overhead in the distributed computing system 100. In one embodiment, the dispatching layer 104 includes a cache layer monitor 114, which is configured to obtain status for some or all of the cache devices 116. The cache layer monitor can be configured to obtain status periodically, for example, at regular intervals (such as every minute or every five minutes). Periodically obtaining status can reduce the computational overhead associated with obtaining the status of the cache devices 116 while providing sufficient information for the operation of the dispatcher 112.

[0033] The cache layer 106 is a layer that provides cache resources. The cache layer 106 may include multiple cache devices 116. The cache layer may use virtualization and / or pooling of cache devices to provide cache storage for the distributed computing system 100. Pooling and virtualization may improve the utilization of the cache device 116. The cache device 116 may be any storage medium suitable for cache operations, non-limiting examples of which include memory, such as dynamic random access memory (DRAM), persistent memory (PMEM), solid-state drive (SSD), such as NVMe protocol SSD, etc. The cache performed in the cache layer may be performed according to any suitable cache algorithm, non-limiting examples of which include least recently used segments, first-in-first-out, L2ARC cache, etc. In one embodiment, data eviction in the cache layer 106 may be performed according to any suitable method, such as a side-by-side or hierarchical method. The cache layer 106 may be where read and write operations are processed first. In one embodiment, when cache device 116 becomes full, at least some data from the full cache device may be persisted when provided to cloud storage tier 108. In one embodiment, when a request for data from client tier 102 cannot be satisfied by data in cache tier 106, the requested data may be obtained from cloud storage tier 108.

[0034] The cloud storage layer 108 includes one or more persistent memory devices, such as PMEM, solid-state drives (such as NVMe SSDs), hard disk drives (HDDs), etc. The persistent memory devices can be virtualized and / or pooled. Data eviction in the cloud storage layer 108 can be performed according to any suitable method, such as a side-by-side or tiered method. As described above, the cloud storage layer 108 can provide persistent storage for data from the cache layer 106, such as when the cache device 116 becomes full. In one embodiment, when the requested data is not available from the cache layer 106, the cloud storage 108 can provide the requested data.

[0035] Figure 2 A schematic diagram of cache and cloud storage tiers is shown according to one embodiment. Cache tier 200 includes DRAM 202, PMEM 204, and SSD 206. Cloud storage tier 208 includes PMEM 210, SSD 212, and HDD 214.

[0036] Cache layer 200 is used to provide a distributed computing system (such as described above and in Figure 1 The distributed computing system 100 shown in FIG. 1 provides cache storage. The cache layer may be as described above and in Figure 1200. The cache layer 200 may include different types of storage devices, including, for example, DRAM 202, PMEM 204, and SSD 206. The storage devices in the cache layer may be arranged and / or utilized according to the respective capacity, bandwidth, and latency of the respective storage devices. DRAM 202 is a dynamic random access memory and has the highest bandwidth and the lowest latency, but has the lowest capacity. PMEM 204 provides intermediate capacity, bandwidth, and latency, while SSD 206 provides the highest capacity but the lowest bandwidth and the highest latency in the cache layer 200, as shown in FIG. Figure 2 shown.

[0037] The cloud storage layer 208 is a distributed computing system (such as the above and Figure 1 The distributed computing system 100 shown in FIG. 1 provides persistent storage. The cloud storage layer 208 may be as described above and in Figure 1 2. The cloud storage layer 108 shown in FIG. The cloud storage layer 208 may include different types of storage devices, such as PMEM 210, SSD 212, and HDD 214. The devices of the cloud storage layer 208 may be arranged and / or utilized according to the corresponding capacity, bandwidth, and latency of the corresponding storage devices. Among the devices used in the cloud storage layer 208, the PMEM 210 may provide the highest bandwidth and the lowest latency while having the lowest capacity, wherein the SSD 212 provides relatively moderate capacity, bandwidth, and latency, and the HDD 214 provides the largest capacity as well as the lowest bandwidth and the highest latency among the devices included in the cloud storage layer 208, such as Figure 2 shown.

[0038] Figure 3 A schematic diagram of a dispatcher of a dispatch layer according to one embodiment is shown. The dispatcher 300 includes a read / write comparison module 302 , a read intensive queue 304 , a write intensive queue 306 , a dispatch output 308 , and a monitor 310 .

[0039] The dispatcher 300 may be a member of a dispatching layer, such as described above and described in Figure 1 3. Dispatcher 300 may be a proxy to which a client, such as client device 110, connects, the connection being facilitated by a DNS directing the client to dispatcher 300. The dispatcher may receive a requested operation, such as a requested I / O stream, from a client device.

[0040] The read / write comparison module 302 receives the requested I / O stream and determines the relative read and write intensity of the I / O stream. The read / write comparison module 302 may include any suitable hardware, software, or combination thereof capable of determining the relative read and write intensity for the received I / O stream. The relative read and write intensity may be, for example, a read percentage, a write intensity, a read / write ratio, or any other suitable metric for determining the relative read and write intensity for the I / O stream. The relative read and write intensity may be compared to a threshold value that is selected so that the I / O stream is directed to a queue corresponding to its read or write intensity. As a non-limiting example, where the relative read and write intensity is measured as the write percentage of the I / O stream, when the write percentage is less than 60%, the I / O stream may be assigned to the read-intensive queue 304, and when the write percentage of the I / O stream is 60% or higher, the I / O stream may be assigned to the write-intensive queue 306.

[0041] The read-intensive queue 304 is a queue of I / O requests that the read / write comparison module determines to be read-intensive. When more I / O requests arrive than can be output from the dispatch output 308, the read-intensive queue 304 may be filled with read-intensive I / O requests. The write-intensive queue 306 is a queue of I / O requests that the read / write comparison module determines to be write-intensive. When more I / O requests arrive than can be output from the dispatch output 308, the write-intensive queue 306 may be filled with write-intensive I / O requests.

[0042] The dispatch output 308 is configured to dispatch I / O requests from the read-intensive queue 304 and the write-intensive queue 306 to appropriate resources, such as described above and in Figure 1 304 and 306. The dispatch output can select the I / O request to be dispatched from the read-intensive queue 304 and the write-intensive queue 306 according to the pick ratio. The pick ratio can be any suitable ratio for efficiently assigning requests to resources, such as based on the relative resource consumption of read operations and write operations. In one embodiment, the pick ratio can be a 1:m ratio, where for every m read-intensive I / O requests taken out of the read-intensive queue 304 and assigned to the resource, one write-intensive I / O request is taken out of the write-intensive queue 306 and will be assigned to the resource. M can be a value greater than 1. Making m greater than 1 gives priority to read-intensive I / O streams, which generally have higher performance requirements than write-intensive I / O streams. This is because write operations are generally much slower than read operations, and streams with a higher proportion of write-intensive requests can slow down concurrent read-intensive streams, thereby protecting the fairness of the allocation of resources to process each request and improving the performance of executing all corresponding I / O requests.

[0043] The I / O request may be assigned to one of the multiple destinations by the dispatch output 308. The destination may be, for example, a specific disk of a cache node, a cache node including one or more disks, or any other such suitable storage device of a cache layer. The target destination may be selected based on the performance volatility of the destination. Performance volatility may be a quantification of how an I / O request negatively affects other I / O flows at the potential destination. Performance volatility may be determined by taking a weighted average of historical deceleration values ​​and current deceleration values ​​measured in recent sampling. The weighted average may be based on any suitable weighting, such as, as a non-limiting example, an exponentially weighted moving average. The weighting may emphasize recent results so that older results have a reduced impact on the determination of performance volatility. The performance deceleration used to determine performance volatility may be calculated based on the average response time of the destination. The average response time may be the sum of the response times for all I / O flows being processed by the destination. In one embodiment, the average response time may be the ratio of the sum of the response times of all I / O flows hosted on the destination during the most recent sampling period to the sum of the workloads of all I / O flows hosted on the destination during the most recent sampling period. For example, a performance slowdown may be calculated by dividing the response time for a destination from the most recent sample by the response time for the destination from the previous sample. A higher value may indicate a greater slowdown, such as due to load on the destination or other impairment to performance. The assignment of I / O requests to destinations by dispatch output 308 may be to the destination with the lowest performance volatility at the dispatch time, or any other suitable relationship between performance volatility and the selection of a destination. In one embodiment,

[0044] In one embodiment, the assignment of I / O requests to resources may also be based on the level of operational traffic, wherein random assignment to a selected subset of resources of a tier or random assignment to any resource of a tier is described below and as Figure 5 is executed as shown.

[0045] Monitor 310 can be used to obtain the status of some or all devices included in the layer of the dispatch output 308 assignment operation, such as the cache device of the cache layer. Monitor 310 can be configured to obtain the status periodically, for example, at regular intervals, such as every minute or every five minutes. Periodically obtaining the status can reduce the computational overhead associated with obtaining the device status while providing sufficient information for the operation of the dispatcher.

[0046] Figure 4A flow chart of a method for determining performance volatility according to one embodiment is shown. The method 400 includes obtaining an average response time for an I / O stream at 402, obtaining a workload of the I / O stream at 404, determining a weighted average response time for multiple I / O streams at 406, determining a slowdown of a disk or cache node at 408, and determining performance volatility of the disk or cache node at 410.

[0047] Method 400 is a method by which performance volatility for a particular resource (such as a cache node or a disk or other storage medium thereof) can be determined, for example, for use as an input to a dispatching decision, such as that performed by dispatcher 300, dispatcher 112 of dispatching layer 104, etc. Figure 1 and Figure 3 As shown and described above.

[0048] The response time(s) for the resource are obtained at 402. The response time(s) for the resource may be the sum of the response times for one or more I / O flows hosted on the resource during the most recent response time sampling period, such as periodically monitored by cache layer monitor 114 or monitor 310, as described above and in Figure 1 and 3 Those shown in .

[0049] The workload of the resource is obtained at 404. The workload of the resource may be defined, for example, as a workload size (e.g., footprint) for the resource, as a working set size (e.g., in bytes) for the resource, or as a number of I / O operations being performed by the resource.

[0050] A weighted average response time for the resource is determined at 406. The weighted average response time may be determined based on the average response time and workload for the resource over time, including the most recent sample and the previous sample within a window. The weighted average may be calculated based on the samples taken within the sampling window. The weighting may favor the most recent data, such as, within the sampling window, the most recent sample is more favorable than the previous sample. The weighting may utilize any suitable weighting scheme, such as linear weighting, exponential weighting, etc. In one embodiment, the weighting is exponential weighting.

[0051] The slowdown of the disk or cache node is determined at 408. The slowdown can be determined based on a change in the weighted average response time of the resource. For example, the slowdown can be calculated at 408 based on the average response time of the resource during the most recent sampling period divided by the average response time of the resource during one or more previous sampling periods.

[0052] The performance volatility of the disk or cache node is determined at 410. The determined performance volatility may be determined based on a slowdown of the disk or cache node over time, such as a change in the slowdown as determined at 408.

[0053] For the following functions in this article, the terms are provided in the following table:

[0054] Table I:

[0055]

[0056] In one embodiment, assignment of I / O streams to specific disks may be performed based on the following function:

[0057] in

[0058]

[0059] By reducing the number of terms, the above function (1) can be made more efficient. Specifically, as provided in function (1) May be truncated based on a limit to the number of previous time periods considered.

[0060] Function (1) can be expressed as When it is truncated to q periods, function (1) can be simplified to function (4) as follows:

[0061]

[0062] Figure 5 5. The cache monitor 500 provides information to the flow calculation module 502. The output from the flow calculation module 502 is compared with various thresholds at the condition check module 504. Based on the comparison made by the condition check module 504, the assignment module 506 assigns the flow to the computing resources according to the selected policy.

[0063] The cache monitor 500 is configured to monitor the traffic status of the cache resources of the cache layer. The cache monitor 500 may be as described above and in Figure 1 Cache layer monitor 114 of dispatch layer 104 in distributed computing system 100 is shown in FIG. Cache monitor 500 may also control policies used for assignment of operations to resources, such as assigning I / O flows to cache resources based on traffic conditions.

[0064] The traffic calculation module 502 is configured to determine the traffic level for the resources of the cache layer. The module may include any suitable hardware, software, and combination thereof to determine the traffic level for the resources of the cache layer. In one embodiment, the traffic level may be defined based on the ratio of the workload in the write-intensive queue to the workload in the read-intensive queue, such as described above and in Figure 3 304 and write-intensive queues 306 are shown in FIG. 304. As more write-intensive I / O flows join the queue, the ratio may increase, indicating burst traffic. The use of such a ratio may reduce the computational overhead of making traffic determinations compared to other traffic determinations, thereby reducing the impact of determining traffic on overall system performance.

[0065] The condition checking module 504 can be used to select a strategy for the assignment of operations to resources (such as I / O flows to cache resources) based on traffic conditions. In one embodiment, the condition checking module 504 is configured to compare the calculated traffic from the traffic calculation module with a threshold associated with a specific assignment strategy in response to the size of the traffic burst. In one embodiment, there are two thresholds, the first threshold indicating when it should be assigned to a subset of resources, and the second threshold indicating when it should be assigned to any or all resources. In an embodiment where the traffic is determined as the ratio of write-intensive operations to read-intensive operations in the corresponding queue as described above, the threshold can be 70% write-intensive as a first threshold, and 90% write-intensive as a second threshold.

[0066] The assignment module 506 is configured to assign operations to resources according to one of the policies selected by the condition checking module 504. Figure 5 In the illustrated embodiment, the assignment module 506 may assign operations to resources using one of three strategies.

[0067] When the condition check module 504 determines that the traffic is in a low condition, such as by being below a first threshold, the assignment module can select an individual resource to assign operations to. For example, a resource can be selected based on being the resource with the lowest determined performance volatility, such as based on Figure 4 Determination of performance volatility of method 400 is shown and described above.

[0068] When the condition check module 504 determines that the traffic is in a moderate condition, such as by being between a first threshold and a second threshold, a subset of resources of the layer can be selected as eligible for assignment, and operations are randomly assigned to the resources in the subset. The size of the subset can be any size that is suitable for the system architecture, the type of computing devices in the layer, the degree of traffic peaks in the range between the first and second thresholds, or any other suitable factor or combination thereof. The size of the subset can be predetermined or dynamically determined. Performance volatility can be determined for the resources, and the resources with the lowest performance volatility can be selected up to the size selected for the subset. Operations can then be randomly assigned to the resources included in the subset. Random assignment prevents individual resources from being overloaded by becoming target destinations during identified periods of increased traffic, while constrained random assignment continues to exploit knowledge of resource status to efficiently and fairly assign operations to resources. Random assignment of a constrained set of resources can be performed according to the following function:

[0069]

[0070] When the condition check module 504 determines that traffic is in a high condition, such as by being above a second threshold, a completely random strategy can be used to assign operations to any resource in the tier's resources. The completely random strategy uses all of the available resources to respond to the highest traffic condition.

[0071] The assignment strategy based on traffic conditions can be expressed as a piecewise function, for example according to the following function:

[0072]

[0073] Figure 6 A flow chart of a method for assigning operations to computing resources according to one embodiment is shown. The method 600 includes obtaining a weighted average of states over time for decoupled computing resources at 602, receiving an operation to be assigned to the decoupled computing resources at 604, and assigning the operation to the decoupled computing resources at 606. Optionally, the method 600 also includes determining whether the operation is read-intensive or write-intensive at 608. Optionally, the method 600 includes determining an amount of incoming operation traffic 610, and optionally randomly assigning the operation to a subset of the decoupled computing resources at 612, or optionally randomly assigning the operation to a set of decoupled computing resources at 614.

[0074] A weighted average of the state over time for the decoupled computing resources is obtained at 602. The decoupled computing resources may be resources of a given layer of a distributed computing system, such as described above and described in Figure 1106 of the distributed computing system 100 shown in FIG. The decoupled computing resources may be computing resources of the same type, such as all of the computing resources of the same type being, for example, one of a CPU, a GPU, a memory device, etc. The weighted average may be determined for some or all of the decoupled computing resources in the layer to which the operation is to be assigned. For example, the weighted average may be determined based on Figure 4 The determination of the weighted average for response time 406 as shown and described above is determined. Based on the time at which the state will be recorded, the average of the state for each decoupled computing resource is weighted, with the more recent state of the computing resource having a higher weight. The weighted average can be weighted according to any suitable weighting scheme, such as linear weighting, exponential weighting, or any other suitable weighting. In one embodiment, the weighted average is an exponentially weighted average. The state of the decoupled computing resource for which the weighted average is taken over time can be any suitable metric for evaluating the state of a particular computing resource, such as, for example, utilization level, performance fluctuations, latency, capacity, power consumption, etc.

[0075] Operations to be assigned to the decoupled computing resources are obtained at 604. These operations can be any operations suitable for decoupled computing resources. In one embodiment, these operations are I / O streams. These operations can be obtained from any suitable source, such as from a client device. In one embodiment, these operations can be obtained from a client device through an API, a library, etc.

[0076] The operations are assigned to the decoupled computing resources at 606. The operations may be assigned by a dispatch layer, such as dispatch layer 106 or a processor including the above-described Figure 1 and Figure 3 The layers of the dispatcher 300 shown in FIG. 606 can be assigned based on the weighted average of the state of the decoupled computing resources obtained at 602 over time. The assignment of operations can be based on, for example, the performance volatility of the resources, such as based on Figure 4 The method 400 shown in and described above determines performance volatility.

[0077] Optionally, whether each operation is read-intensive or write-intensive can be determined at 608. This optional step can be performed when the operation is an I / O stream. The determination can be made by determining the relative read and write intensity (such as the read or write percentage of the I / O stream) and comparing the relative read and write intensity to a suitable threshold. Based on the threshold, the I / O stream can be assigned to a read-intensive or write-intensive queue, such as described above and in Figure 3 6. The assignment of operations at 604 can be performed in part by extracting from the read-intensive and write-intensive queues according to a pick ratio, such as the 1:m pick ratio discussed above, for assigning one write-intensive operation to each of multiple read-intensive operations.

[0078] Optionally, the amount of incoming operational traffic can be determined at 610. The determination can be based on any suitable metric for the operational traffic to be assigned to the computing devices in the tier. For example, the amount of incoming operational traffic can be determined based on the ratio of write-intensive flows in the write-intensive queue to read-intensive flows in the read-intensive queue. An example function for determining the amount of incoming operational traffic at 610 can be the following:

[0079]

[0080] Among them B [τ] The burstiness of the I / O flow during period τ is defined based on the ratio of the workload of the write-intensive queue to the total workload of the write-intensive queue and the read-intensive queue during the current period. [WIQ,τ] and W [RIQ,τ] They represent the workload of the write-intensive queue and the read-intensive queue during period τ, respectively. A higher burstiness indicates that more write-intensive I / O flows join the queue compared to the previous period.

[0081] Optionally, operations may be randomly assigned to subsets of decoupled computing resources at 612. The assignment of subsets of decoupled computing resources at 612 may be based on incoming operation traffic between a first threshold and a second threshold. The subsets may be determined based on a subset size and a state of the resources, wherein each operation is randomly assigned to one of the resources within the subset size. The determination of the size of the subset and the selection of resources for the subset may be based on the selection of the subsets as described above and in Figure 5 The assignment strategy for medium traffic conditions is shown in .

[0082] Optionally, operations can be randomly assigned to the decoupled computing resource set at 614. The random assignment at 614 is any resource randomly assigned to the resources of the layer. When a high amount of incoming operation traffic is determined at 610, the random assignment at 614 can be performed. The random assignment at 614 can be based on the random assignment for the resource set described above and in Figure 5 The assignment strategy for high traffic conditions is shown in .

[0083] In one embodiment, a background monitoring system may be provided to evaluate the performance of a distributed computing system in response to workload demands. The background monitor may evaluate the workload and evaluate the consistency and / or magnitude of peaks in such workload. The distributed computing system may be scaled based on determinations made by evaluating the utilization states of multiple queues.

[0084] Figure 7 A flow chart of a method for scaling operation according to one embodiment is shown. Figure 7As shown, a background monitoring thread is initialized and checks whether all I / O queues are fully utilized. If not all I / O queues are utilized, the method can bypass additional steps. When all I / O queues are utilized, the consistency of the peak in utilization is determined. In one embodiment, the consistency of the peak can be determined based on Figure 8 The method shown is determined. If the peaks are not consistent, the method can bypass other steps; when the peaks are consistent, it can be determined whether the peaks are adequately handled by the distributed computing system. If the system is effectively managing the peaks, the method can bypass additional steps; if the peaks are causing the system to become bogged down, a scaling decision function can be called. In one embodiment, the scaling decision function can be based on Fig. 9 The method shown. The method may then proceed based on the scaling decision from the scaling decision function by performing a recommended scaling or bypassing scaling when it is determined by the scaling decision function not to perform scaling. In one embodiment, the scaling decision is provided to one or more system administrators as a recommendation. In one embodiment, the scaling decision may be implemented automatically.

[0085] The background monitoring thread can operate autonomously, running in the background while the main system continues its regular process. Therefore, background monitoring can alleviate overload conditions and facilitate seamless execution during fluctuating workloads while efficiently scaling resources.

[0086] Figure 8 A flow chart of a method for determining the variability of a peak value according to one embodiment is shown. As described above, Figure 7 The determination of the variability of the peak values ​​shown in can be integrated into Figure 7 In the method for scaling operations shown in . The variability of the peak value can be determined based on, for example, the coefficient of variation (CV). The CV can be the ratio of the standard deviation to the mean in the data set. The CV can be expressed as a percentage. In order to determine the CV, historical data related to the I / O request pattern can be systematically accumulated over a predetermined time frame. From the historical data, the frequency of the I / O request peak can be quantified. Then, the mean and standard deviation of the I / O request counts within a predefined time interval can be determined based on the frequency data about the I / O request peak. The CV can then be calculated based on the mean and standard deviation. In one embodiment, the calculation of the CV involves dividing the standard deviation by the mean and then multiplying by 100 to express the result as a percentage. This numerical output effectively encapsulates the degree of difference between the peak value and the average. The CV can be compared to a predetermined threshold value, which is used as a benchmark, and the peak values ​​below this benchmark are described as uniform. An exemplary threshold value can involve treating the peak value as uniform when the CV is less than 20%.

[0087] Scaling decisions may be made based on an evaluation of the total cost of ownership and the performance benefits provided by scaling and the manner in which it is provided. Scaling decisions may include whether to scale, whether scaling should be performed by scaling up or down, etc. Scaling decisions may be presented as a choice for a system administrator to make the final scaling decision on his own. Scaling decisions may be based on whether the service cost rate is equal to or lower than the service cost rate of the current configuration. The service cost rate may be defined as the ratio of the total cost of ownership to the maximum load capacity. In one embodiment, the difference in the service cost rate must meet or exceed a predefined threshold to trigger a decision to scale up or down.

[0088] Fig. 9 A flow chart of a method for determining a scale of an operation according to one embodiment is shown. Fig. 9 The method begins with determining the additional load required ("MaxLoad_scale"). The additional load can be determined by determining the extent to which the load exceeds the existing capacity. The total cost of ownership for expanding the capacity to accommodate the additional load is then determined ("TCO_scale"). The TCO_scale value can be added to the current total cost of ownership value ("TCO_prev") to obtain a new total cost of ownership for the combined current and increased capacity ("TCO_new"). MaxLoad_scale is added to the current load to obtain the total capacity of the scaled system ("MaxLoad_new"). The service cost rate for the combined current and increased capacity can then be determined by dividing TCO_new by MaxLoad_new. The current and previous service cost rates can be used to determine the marginal difference in the service cost rate ("ServCostRate_new"), which can then be compared to a predetermined threshold. Based on the comparison, a determination can be made whether to perform scaling to meet the additional load. When it is determined that scaling is performed to meet the additional load, a function can be called to determine a scaling plan for the scaling to be performed.

[0089] Fig.10 A schematic block diagram of an example computer system suitable for implementing an electronic device is shown, which is arranged according to at least some of the embodiments described herein. It should be understood that Fig.10 The computer system shown in FIG. 1 is for illustration only and is not intended to limit the functionality and applicability of the embodiments described herein.

[0090] As shown, the computer system 1000 may include a central processing unit (CPU) 1005. The CPU 1005 may perform various operations and processes based on a program stored in a read-only memory (ROM) 1010 or a program loaded from a storage device 1040 to a random access memory (RAM) 1015. The RAM 1014 may also store various data and programs required for the operation of the system 1000. The CPU 1005, the ROM 1010, and the RAM 1020 may be connected to each other via a bus 1020. An input / output (I / O) interface 1025 may also be connected to the bus 1020.

[0091] Components connected to the I / O interface 1025 may also include input devices 1030, including keyboards, mice, digital pens, drawing boards, etc.; output devices 1035, including displays such as liquid crystal displays (LCDs), speakers, etc.; storage devices 1040, including hard disks, etc.; and communication devices 1045, including network interface cards (such as LAN cards, modems, etc.). The communication device 1045 can perform communication processing via a network (such as the Internet, WAN, LAN, LIN, cloud, etc.). In one embodiment, a drive 1050 may also be connected to the I / O interface 1025. Removable media 1055 (such as magnetic disks, optical disks, magneto-optical disks, semiconductor memories, etc.) may be installed on the drive 1050 as needed, so that a computer program read from the removable media 1055 may be installed in the storage device 1040.

[0092] aspect:

[0093] It should be understood that any one of aspects 1 to 10 may be combined with any one of aspects 11 to 17.

[0094] Aspect 1. A distributed computing system, comprising:

[0095] a first plurality of decoupled computing resources of a first type, the first plurality of decoupled computing resources being a first tier of distributed computing resources;

[0096] a second plurality of decoupled computing resources of a second type, the second plurality of decoupled resources being a second tier of the distributed computing system, the second tier being configured to receive operations provided by the first tier; and

[0097] A dispatching layer is configured to assign at least some of the operations provided by the first layer to at least some of the computing resources of a second plurality of decoupled computing resources of the second layer based on a weighted moving average of the states of at least some of the decoupled computing resources of the second plurality over time.

[0098] Aspect 2. The distributed computing system according to aspect 1,

[0099] The first layer is the client layer.

[0100] wherein the second layer is a cache layer and the second plurality of decoupled computing resources are corresponding storage memories, and

[0101] Where operations are input / output (I / O) streams.

[0102] Aspect 3. A distributed computing system according to aspect 2, wherein the corresponding storage memory comprises a dynamic random access memory (DRAM).

[0103] Aspect 4. A distributed computing system according to aspect 2, wherein the corresponding storage memory is a solid state drive (SSD).

[0104] Aspect 5. A distributed computing system according to any one of Aspects 2 to 4, wherein the dispatching layer is configured to divide the I / O streams into a read-intensive queue and a write-intensive queue based on a write percentage of each of the I / O streams.

[0105] Aspect 6. The distributed computing system according to aspect 5,

[0106] wherein the dispatching layer is configured to assign the I / O flows of the read-intensive queue and the I / O flows of the write-intensive queue according to the pickup ratio, and

[0107] The pick ratio is a ratio of a plurality of I / O flows of the read-intensive queue to be assigned to each of the I / O flows of the write-intensive queue to be assigned.

[0108] Aspect 7. A distributed computing system according to any one of aspects 1 to 6, wherein the weighted average is an exponentially weighted moving average.

[0109] Aspect 8. A distributed computing system according to any one of aspects 1 to 7, wherein the dispatch layer is configured to determine the amount of incoming operational traffic.

[0110] Aspect 9. The distributed computing system according to aspect 8, wherein the dispatching layer is configured to:

[0111] comparing the determined amount of incoming operational traffic to a first threshold and a second threshold, and

[0112] When the amount of incoming operational traffic is above a first threshold and below a second threshold, operations provided by the first layer are randomly assigned to a subset of a second plurality of decoupled computing resources, the subset being selected based on a weighted moving average of states of the second plurality of decoupled computing resources over time.

[0113] Aspect 10. A distributed computing system according to aspect 9, wherein the one or more processors are configured to randomly assign operations provided by the first layer to the entire second plurality of computing resources when the amount of incoming operational traffic is above a second threshold.

[0114] Aspect 11. A method for assigning operations to computing resources, comprising:

[0115] Obtaining a weighted average of states over time for a plurality of decoupled computing resources;

[0116] receiving one or more operations to be assigned to a plurality of decoupled computing resources; and

[0117] One or more operations are assigned to at least some of the plurality of decoupled computing resources based on a weighted average of states over time for the plurality of decoupled computing resources.

[0118] Aspect 12. The method according to aspect 11, wherein the weighted average is an exponentially weighted average.

[0119] Aspect 13. The method according to any one of aspects 11 to 12,

[0120] where the operation is an input / output (I / O) stream, and

[0121] The method further includes dividing the I / O streams into a read-intensive queue and a write-intensive queue based on a write percentage of each I / O stream in the I / O stream.

[0122] Aspect 14. The method according to aspect 13,

[0123] wherein the I / O streams of the read-intensive queue and the I / O streams of the write-intensive queue are assigned according to a pick ratio, and

[0124] The pick ratio is a ratio of a plurality of I / O flows of the read-intensive queue to be assigned to each of the I / O flows in the write-intensive queue to be assigned.

[0125] Aspect 15. The method according to any one of aspects 11 to 14 further includes determining the amount of incoming operational traffic.

[0126] Aspect 16. The method according to aspect 15 further includes:

[0127] comparing the amount of incoming operational traffic to a first threshold and a second threshold, and

[0128] When the amount of incoming operational traffic is above a first threshold and below a second threshold, operations provided by the first layer are randomly assigned to a subset of the second plurality of computing resources, the subset being selected based on a weighted moving average of states of the second plurality of decoupled computing resources over time.

[0129] Aspect 17. The method of aspect 16, wherein when the amount of incoming operational traffic is above a second threshold, operations provided by the first layer are randomly assigned to the entire second plurality of computing resources.

[0130] Aspect 18. A data center comprising:

[0131] a first plurality of decoupled computing resources of a first type;

[0132] a second plurality of decoupled computing resources of a second type; and

[0133] A dispatch layer is configured to assign at least some operations provided by the first layer to at least some of the computing resources of a second plurality of decoupled computing resources of the second layer based on a weighted moving average of the states of at least some of the decoupled computing resources of the second plurality over time.

[0134] Aspect 19. The data center of aspect 18, wherein the second type of decoupled computing resource is a cache storage memory.

[0135] Aspect 20. The data center of aspect 19, wherein the operations are I / O streams, and the dispatching layer is configured to divide the I / O streams into a read-intensive queue and a write-intensive queue based on a write percentage of each of the I / O streams.

[0136] The examples disclosed in this application should be considered in all aspects as illustrative and not restrictive. The scope of the invention is indicated by the appended claims rather than the foregoing description; and all changes within the equivalent meaning and scope of the claims are intended to be encompassed.

Claims

1. A distributed computing system, comprising: a first plurality of decoupled computing resources of a first type, the first plurality of decoupled computing resources being a first tier of the distributed computing resources; a second plurality of decoupled computing resources of a second type, the second plurality of decoupled resources being a second tier of the distributed computing system, the second tier being configured to receive operations provided by the first tier; as well as A dispatching layer is configured to assign at least some of the operations provided by the first layer to at least some of the computing resources of the second plurality of decoupled computing resources of the second layer based on a weighted moving average of the states of at least some of the second plurality of decoupled computing resources over time.

2. The distributed computing system according to claim 1, The first layer is the client layer. wherein the second layer is a cache layer, and the second plurality of decoupled computing resources are corresponding storage memories, and The operation is an input / output I / O stream.

3. The distributed computing system of claim 2, wherein the corresponding storage memory comprises a dynamic random access memory (DRAM).

4. The distributed computing system of claim 2, wherein the corresponding storage memory is a solid state drive (SSD).

5. The distributed computing system of claim 2, wherein the dispatching layer is configured to divide the I / O streams into a read-intensive queue and a write-intensive queue based on a write percentage of each of the I / O streams.

6. The distributed computing system according to claim 5, wherein the dispatching layer is configured to assign the I / O flow of the read-intensive queue and the I / O flow of the write-intensive queue according to a pick ratio, and The pick ratio is a ratio of a plurality of I / O flows of the read-intensive queue to be assigned to each of the I / O flows of the write-intensive queue to be assigned.

7. The distributed computing system of claim 1, wherein the weighted average is an exponentially weighted moving average.

8. The distributed computing system of claim 1, wherein the dispatch layer is configured to determine an amount of incoming operational traffic.

9. The distributed computing system of claim 8, wherein the dispatch layer is configured to: comparing the determined amount of incoming operational traffic to a first threshold and a second threshold, and When the amount of incoming operational traffic is above the first threshold and below the second threshold, the operations provided by the first layer are randomly assigned to a subset of the second plurality of decoupled computing resources, wherein the subset is selected based on a weighted moving average of the states of the second plurality of decoupled computing resources over time.

10. The distributed computing system of claim 9, wherein the one or more processors are configured to randomly assign the operations provided by the first layer to the entire second plurality of computing resources when the amount of incoming operational traffic is above the second threshold.

11. A method for assigning operations to computing resources, comprising: Obtaining a weighted average of states over time for a plurality of decoupled computing resources; receiving one or more operations to be assigned to the plurality of decoupled computing resources; as well as The one or more operations are assigned to at least some of the plurality of decoupled computing resources based on the weighted average of the states over time for the plurality of decoupled computing resources. The method of claim 11 , wherein the weighted average is an exponentially weighted average.

13. The method according to claim 11, where the operation is an input / output I / O stream, and The method further includes dividing the I / O streams into a read-intensive queue and a write-intensive queue based on a write percentage of each of the I / O streams.

14. The method according to claim 13, wherein the I / O stream of the read-intensive queue and the I / O stream of the write-intensive queue are assigned according to a pick ratio, and The pick ratio is a ratio of a plurality of I / O streams of the read-intensive queue to be assigned to each of the I / O streams in the write-intensive queue to be assigned.

15. The method of claim 11, further comprising determining an amount of incoming operational traffic.

16. The method according to claim 15, further comprising: comparing the amount of incoming operational traffic to a first threshold and a second threshold, and When the amount of incoming operational traffic is above the first threshold and below the second threshold, the operations provided by the first layer are randomly assigned to a subset of the second plurality of computing resources, the subset being selected based on a weighted moving average of the states of the second plurality of decoupled computing resources over time.

17. The method of claim 16, wherein when the amount of incoming operational traffic is above the second threshold, the operations provided by the first layer are randomly assigned to the entire second plurality of computing resources.

18. A data center, comprising: a first plurality of decoupled computing resources of a first type; a second plurality of decoupled computing resources of a second type; as well as A dispatching layer is configured to assign at least some operations provided by the first layer to at least some of the computing resources of the second plurality of decoupled computing resources of the second layer based on a weighted moving average of the states of at least some of the second plurality of decoupled computing resources over time.

19. The data center of claim 18, wherein the second type of decoupled computing resource is cache storage memory.

20. The data center of claim 19, wherein the operations are I / O streams, and the dispatch layer is configured to divide the I / O streams into a read-intensive queue and a write-intensive queue based on a write percentage of each of the I / O streams.

Images