Distributed resource management by increasing cluster diversity

Abstract

A distributed resource management method is disclosed, which ensures fairness in a distributed computing system by prioritizing diversity of low-level hardware resources, the method comprising: determining usage of respective hardware resources by applications to generate usage metrics, and allocating the applications to a cluster of hardware resources to optimize diversity of usage of the hardware resources in the cluster and improve utilization of the hardware resources by applications running in the cluster. The diversity of usage of the hardware resources is determined from respective usage metrics of the respective applications running in the cluster. The diversity of usage of the hardware resources in the cluster is optimized by: allocating the applications to a diversified pool of hardware resources, wherein the diversified pool of hardware resources is used to minimize interference when applications allocated to the diversified pool of hardware resources access the hardware resources in the diversified pool; and allocating applications from different diversified pools to the cluster of hardware resources.

Description

[0001] Related applications cross-application

[0002] This application claims priority and benefit to U.S. Provisional Application No. 62 / 769,596, filed November 20, 2018, entitled “Distributed Resource Management by Enhancing Cluster Diversity,” which is incorporated herein by reference. Technical Field

[0003] This application relates to parallel and distributed computing, and more particularly to improving resource management in distributed resource clusters without impacting application performance in data centers and distributed resource clusters by identifying and enhancing the diversity of low-level resources used by applications. Background Technology

[0004] Resource management (RM) is a technology that maps resources with limited availability to their users and application frameworks. In the cloud, RM refers to the management of compute, network, and storage resources in large clusters or data centers. A data center's profitability depends directly on the utilization of individual compute nodes. However, most data centers and large clusters of hardware components have very low utilization (less than 50%) and typically operate at a fraction of their processing power. Data centers and the cloud hosting multiple applications from the same or different tenants often face low resource utilization, which increases energy costs and reduces profit margins. Low resource utilization is often a side effect of isolating applications to prevent performance degradation caused by other co-running applications or due to a lack of understanding of the applications themselves and their performance requirements.

[0005] Different resource management systems (RMs) have been proposed to address these resource management problems. Some RM systems are dedicated to handling different jobs or user requests (scheduling) for a specific framework, while others attempt to handle multiple frequently invoked frameworks. In either case, RM systems typically strive to optimize system functionality and performance through the managed resources. Functionality management involves resource billing performed at a coarse-grained level. This management rarely addresses inefficiencies in underlying hardware and applications. Similarly, performance management typically involves the system making resource allocation decisions, rather than the underlying framework. Furthermore, existing RM systems focus on CPU / memory access and do not maximize the diversity of low-level resource usage across machines or hardware clusters by allocating resources to applications, because RM systems are often unaware of the characteristics of application hardware usage. Therefore, RM systems typically allocate applications to resources based on high-level CPU and memory scheduling, without optimizing the efficiency of resource usage across other systems.

[0006] An application's performance largely depends on the efficient use of multiple hardware modules such as pipelines, caches, buses, and memory bandwidth. Since these hardware modules are shared by all applications using a given machine, they cannot be directly allocated to different applications. Therefore, failing to optimize the efficiency of low-level hardware resource usage is problematic. Different applications rely on using different hardware modules to minimize interference and achieve good performance. However, strictly allocating hardware modules to applications to minimize interference with hardware resource access does not optimize the use of underlying hardware resources. Summary of the Invention

[0007] This document describes various examples to illustrate the selection of concepts in a simplified form, which will be further described in the embodiments below. The summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

[0008] The aim is to develop a distributed resource management system that determines at least one of cache coherence, memory bandwidth, multilevel cache performance, processor bus performance, processor pipeline performance, cache bandwidth, cache hit rate, and inconsistent memory access latency, and allocates applications to be packaged within the same hardware component cluster. These applications have different characteristics in their low-level resource requirements, thereby increasing the diversity of their resource needs. The system and method described herein prioritize the diversity of low-level resources and package applications within machines such that applications allocated to a given machine use different low-level resources, ensuring that all possible low-level resources are utilized.

[0009] The first aspect relates to a computer implementation method for distributed resource management in a distributed computing system by prioritizing the diversity of hardware resources. The method includes the steps of: determining the usage of corresponding hardware resources by applications to generate usage metrics; and allocating the applications to a hardware resource cluster to optimize the diversity of hardware resource usage in the cluster and improve the utilization of hardware resources by applications running in the cluster. In an example embodiment, the diversity of hardware resource usage in the cluster is determined based on the corresponding usage metrics of the respective applications running in the cluster. This method improves overall resource utilization and efficiency without sacrificing the performance of the applications themselves. The method also improves the resource utilization of hardware clusters and data centers, reduces the average latency of application execution, reduces energy consumption, and increases cost savings due to the reduced time spent by each application. This efficiency improvement has a significant impact on the energy and cost of data centers, making them more affordable, profitable, and environmentally friendly.

[0010] According to the first aspect, in a first implementation, determining the application's usage of the corresponding hardware resources includes: using a hardware performance counter to determine the application's usage of the corresponding hardware resources in order to generate the usage metric.

[0011] According to the first aspect or any of the foregoing implementations of the first aspect, in a second implementation, the usage of the corresponding hardware resources by the application is determined to determine whether the usage metric is executed when the application is running offline.

[0012] According to the first aspect or any of the foregoing implementations of the first aspect, in a third implementation, the usage of the corresponding hardware resources by the application is determined to determine that the usage metric is executed in real time when the application is executed by the hardware resources.

[0013] According to the first aspect, in a fourth implementation, the method further includes: characterizing the corresponding hardware resources through micro-benchmarking to identify performance-critical hardware resources.

[0014] According to the first aspect or any of the foregoing implementations of the first aspect, in the fifth implementation, the step of allocating the application to a hardware resource cluster to optimize the diversity of the usage of the hardware resources in the cluster includes: pushing the hardware resources requested by the application to the capabilities of each hardware resource, and allocating the application to a diversified pool of hardware resources based on the usage metric using a fairness algorithm to maximize the diversity of the usage of the hardware resources in the diversified pool.

[0015] According to the first aspect or any of the foregoing implementations of the first aspect, in a sixth implementation, the step of allocating the application to a hardware resource cluster to optimize the diversity of the use of the hardware resources in the cluster includes: allocating the application to a diversified pool of hardware resources, wherein the diversified pool of hardware resources is configured to minimize interference when an application allocated to the diversified pool of hardware resources accesses the hardware resources in the diversified pool; and allocating applications from different diversified pools to the hardware resource cluster.

[0016] According to the first aspect or any of the foregoing implementations of the first aspect, in the seventh implementation, allocating the application to a hardware resource cluster to optimize the diversity of hardware resource usage in the cluster includes: classifying the application and other applications into a diversified pool of hardware resources using the fairness algorithm.

[0017] According to the first aspect or any of the foregoing implementations of the first aspect, in the eighth implementation, the step of allocating the application to a hardware resource cluster to optimize the diversity of the use of the hardware resources in the cluster includes: selecting the hardware resource cluster with the lowest diversity and allocating the application from the corresponding diversity pool to the hardware resource cluster with the lowest diversity for processing.

[0018] According to the first aspect or any of the foregoing implementations of the first aspect, in the ninth implementation, the method further includes: determining the performance of a module of hardware resources by determining at least one of cache coherence, memory bandwidth, multilevel cache performance, processor bus performance, processor pipeline performance, cache bandwidth, cache hit rate, and non-coherent memory access latency.

[0019] The second aspect relates to an apparatus for distributed resource management in a distributed computing system by prioritizing the diversity of hardware resources. The apparatus includes: at least one hardware performance counter for generating usage metrics of applications on corresponding hardware resources; and a resource manager for allocating the applications to a hardware resource cluster based on the usage metrics, thereby optimizing the diversity of hardware resource usage in the cluster and improving the utilization of hardware resources by applications running in the cluster. In an example embodiment, the diversity of hardware resource usage in the cluster is determined based on the corresponding usage metrics of the respective applications running in the cluster. The apparatus improves overall resource utilization and efficiency without sacrificing the performance of the applications themselves. The apparatus also improves the resource utilization of hardware clusters and data centers, reduces the average latency of application execution, reduces energy consumption, and increases cost savings due to the reduced time spent by each application.

[0020] According to the second aspect, in a first implementation, determining the application's usage of the corresponding hardware resources includes: using a hardware performance counter to determine the application's usage of the corresponding hardware resources in order to generate the usage metric.

[0021] According to the second aspect or any of the foregoing implementations of the second aspect, in the second implementation, the usage of the corresponding hardware resources by the application is determined to determine whether the usage metric is executed when the application is running offline.

[0022] According to the second aspect or any of the foregoing implementations of the second aspect, the usage of the corresponding hardware resources by the application is determined to determine that the usage metric is executed in real time when the application is executed by the hardware resources.

[0023] According to the second aspect or any implementation thereof, in a fourth implementation, the method further includes: characterizing the corresponding hardware resources through micro-benchmarking to identify performance-critical hardware resources.

[0024] According to the second aspect or any of the foregoing implementations of the second aspect, in the fifth implementation, the step of allocating the application to a hardware resource cluster to optimize the diversity of the use of the hardware resources in the cluster includes: pushing the hardware resources requested by the application to the capabilities of each hardware resource, and allocating the application to a diversified pool of hardware resources based on the usage metric using a fair algorithm to maximize the diversity of the use of the hardware resources in the diversified pool.

[0025] According to the second aspect or any of the foregoing implementations of the second aspect, in the sixth implementation, the step of allocating the application to a hardware resource cluster to optimize the diversity of the use of the hardware resources in the cluster includes: allocating the application to a diversified pool of hardware resources, wherein the diversified pool of hardware resources is configured to minimize interference when an application allocated to the diversified pool of hardware resources accesses the hardware resources in the diversified pool; and allocating applications from different diversified pools to the hardware resource cluster.

[0026] According to the second aspect or any of the foregoing implementations of the second aspect, in the seventh implementation, the step of allocating the application to a hardware resource cluster to optimize the diversity of the use of the hardware resources in the cluster includes: classifying the application and other applications into a diversified pool of hardware resources using the fairness algorithm.

[0027] According to the second aspect or any of the foregoing implementations of the second aspect, in the eighth implementation, the step of allocating the application to a hardware resource cluster to optimize the diversity of the use of the hardware resources in the cluster includes: selecting the hardware resource cluster with the lowest diversity and allocating the application from the corresponding diversity pool to the hardware resource cluster with the lowest diversity for processing.

[0028] According to the second aspect or any of the foregoing implementations of the second aspect, in the ninth implementation, the method further includes: determining the performance of a module of hardware resources by determining at least one of cache coherence, memory bandwidth, multilevel cache performance, processor bus performance, processor pipeline performance, cache bandwidth, cache hit rate, and non-coherent memory access latency.

[0029] According to a third aspect of this disclosure, a non-transitory computer-readable medium storing computer instructions is provided for distributed resource management in a distributed computing system by prioritizing the diversity of hardware resources. When the instructions are executed by one or more processors, the following method is implemented: determining the usage of corresponding hardware resources by an application to generate a usage metric; and allocating the application to a hardware resource cluster based on the usage metric to optimize the diversity of hardware resource usage in the cluster and improve the utilization of hardware resources by applications running in the cluster. In an example embodiment, the diversity of hardware resource usage in the cluster is determined based on the corresponding usage metric of the corresponding application running in the cluster. This medium improves overall resource utilization and efficiency without sacrificing the performance of the application itself. The medium also improves the resource utilization of hardware clusters and data centers, reduces the average latency of application execution, reduces energy consumption, and increases cost savings due to the reduced time spent by each application.

[0030] According to the third aspect, in the first implementation, determining the application's usage of the corresponding hardware resources includes: using a hardware performance counter to determine the application's usage of the corresponding hardware resources in order to generate the usage metric.

[0031] According to the third aspect or any of the foregoing implementations of the third aspect, in the second implementation, the usage of the corresponding hardware resources by the application is determined to determine whether the usage metric is executed when the application is running offline.

[0032] According to the third aspect or any of the foregoing implementations of the third aspect, in the third implementation, the usage of the corresponding hardware resources by the application is determined to determine that the usage metric is executed in real time when the application is executed by the hardware resources.

[0033] According to the third aspect or any implementation thereof, in a fourth implementation, the method further includes: characterizing the corresponding hardware resources through micro-benchmarking to identify performance-critical hardware resources.

[0034] According to the third aspect or any of the foregoing implementations of the third aspect, in the fifth implementation, allocating the application to a hardware resource cluster to optimize the diversity of the use of the hardware resources in the cluster includes: pushing the hardware resources requested by the application to the capabilities of each hardware resource, and allocating the application to a diversified pool of hardware resources based on the usage metric using a fairness algorithm to maximize the diversity of the use of the hardware resources in the diversified pool.

[0035] According to the third aspect or any of the foregoing implementations of the third aspect, in the sixth implementation, allocating the application to a hardware resource cluster to optimize the diversity of hardware resource usage in the cluster includes: allocating the application to a diversified pool of hardware resources, wherein the diversified pool of hardware resources is configured to minimize interference when an application allocated to the diversified pool of hardware resources accesses the hardware resources in the diversified pool; and allocating applications from different diversified pools to the hardware resource cluster.

[0036] According to the third aspect or any of the foregoing implementations of the third aspect, in the seventh implementation, allocating the application to a hardware resource cluster to optimize the diversity of hardware resource usage in the cluster includes: classifying the application and other applications into a diversified pool of hardware resources using the fairness algorithm.

[0037] According to the third aspect or any of the foregoing implementations of the third aspect, in the eighth implementation, the step of allocating the application to a hardware resource cluster to optimize the diversity of the use of the hardware resources in the cluster includes: selecting the hardware resource cluster with the lowest diversity and allocating the application from the corresponding diversity pool to the hardware resource cluster with the lowest diversity for processing.

[0038] According to the third aspect or any of the foregoing implementations of the third aspect, in the ninth implementation, the method further includes: determining the performance of a module of hardware resources by determining at least one of cache coherence, memory bandwidth, multilevel cache performance, processor bus performance, processor pipeline performance, cache bandwidth, cache hit rate, and non-coherent memory access latency.

[0039] As described in the following detailed embodiments, the apparatus and method create a mechanism to improve the efficiency of applications allocated to hardware clusters or data centers in using lower-level resources.

[0040] The method can be executed by the device, and the instructions on the computer-readable medium can be processed by the device. Further features of the method and the instructions on the computer-readable medium arise from the functionality of the device. Furthermore, the explanations provided for each aspect and its implementation also apply to other aspects and corresponding implementations. Different embodiments can be implemented in hardware, software, or any combination thereof. Moreover, any of the above examples can be combined with any one or more of the other examples above to create new embodiments within the scope of this disclosure. Attached Figure Description

[0041] In the accompanying drawings, which are not necessarily drawn to scale, similar figures may be used to describe similar components in different figures. The drawings illustrate various embodiments discussed in this document by way of example and not limitation.

[0042] Figure 1 This illustrates a distributed resource management system that distributes applications to a cluster of nodes to improve diversity in an example embodiment;

[0043] Figure 2 An example set of applications is shown that uses the techniques described in this paper to be distributed to a node cluster to improve diversity;

[0044] Figure 3 An example is shown of assigning a sample set of applications to a cluster of nodes using traditional fine-grained resource counting techniques;

[0045] Figure 4 An example is shown of allocating a sample set of applications to a node cluster using a diverse resource management system, according to an example embodiment.

[0046] Figure 5 A flowchart illustrating a variety of resource management algorithms according to an example embodiment is shown; and

[0047] Figure 6 Typical general-purpose network components suitable for implementing one or more embodiments of the apparatuses, methods, and computer-readable storage media disclosed herein are shown. Detailed Implementation

[0048] First, it should be understood that although illustrative implementations of one or more embodiments are provided below, in conjunction with... Figures 1 to 6 The disclosed systems and / or methods described can be implemented by any number of techniques, whether currently known or existing. The invention should in no way be limited to the illustrative embodiments, drawings, and techniques set forth below, including the designs and embodiments illustrated and described herein, but can be modified within the full scope of the appended claims and their equivalents.

[0049] The systems and methods described herein improve cluster utilization without impacting application performance in data centers and large hardware clusters by identifying and enhancing the diversity of low-level resources used by applications allocated to a cluster. In example embodiments, the resource manager packages applications onto the same machine or machine cluster to place various types of different tasks (e.g., CPU-intensive and memory-intensive applications) on the same computer, thereby improving overall resource utilization and efficiency without sacrificing the performance of the applications themselves. This is achieved by determining the usage of corresponding low-level hardware resources by applications to generate usage metrics, and allocating the applications to a hardware resource cluster to optimize the diversity of usage of these same hardware resources by applications running on that cluster. The diversity of hardware resource usage in the cluster is determined based on the usage metrics of the corresponding applications running on that cluster.

[0050] Such usage metrics are established by identifying performance-critical hardware resources using hardware performance counters and hardware and application characterizations based on micro-benchmarks. These tools can be purchased for testing specific resources or developed as custom products for specific hardware resources within a specific hardware cluster. As those skilled in the art know, hardware performance counters are a set of dedicated registers built into a microprocessor to store counts of hardware-related activities within a computer system. These counters are typically used to perform low-level performance profiling or tuning. Each hardware performance counter can be programmed with an index of the type of event to be monitored, such as an L1 cache miss or branch error. Software analyzers perform similar functions, but typically require modification of the source code, which is generally undesirable. Furthermore, some processors use instruction-based sampling (IBS) techniques to implement performance counters at the corresponding ends of the superscalar pipeline for extracting and manipulating samples.

[0051] On the other hand, micro-benchmarking involves running a computer program or a set of programs or other operations on a hardware system to evaluate the relative performance of corresponding components of that hardware system. The computer program or other operations typically perform numerous standard tests and trials to test the hardware and determine, for example, performance boundaries. Benchmarking is often associated with performance characteristic evaluations, such as the floating-point performance evaluation of a CPU, but micro-benchmarking can also be applied to other hardware components and software to allow the performance of various hardware subsystems and basic components to be compared across different chip and system architectures. For example, micro-benchmarking is often used to automatically detect computer hardware parameters such as the number of registers, cache size, memory latency, etc.

[0052] The resource manager then uses the acquired usage metrics to perform resource allocation to maximize the diversity of hardware resource usage by applications processed by a given node (machine). Applications can be allocated to resources based on usage metrics collected while the application is running offline (e.g., usage metrics collected for familiar applications in a private cloud), but usage metrics for new and unpredictable applications, as well as applications dynamically reassigned while the application is running, can also be collected in real time. Applications can then be allocated or reassigned to nodes for processing based on the collected usage metrics.

[0053] These technologies allocate applications to the machine based on its different low-level resources. Instead of strictly allocating applications according to the required CPU or memory cycles, it packages applications with different characteristics together to ensure that all low-level resources in the computer are fully utilized.

[0054] As used in this article, "application" refers to a software application that requires computing system resources to run. Such computing system resources include those managed by traditional resource managers, such as high-level resources like CPU, memory capacity, and disk quotas, as well as low-level resources that determine application performance, such as CPU pipelines, cache coherency, memory bandwidth, multi-level caches, and processor buses. These lower-level resources are typically shared among multiple CPUs and cannot be directly accessed.

[0055] Those skilled in the art will recognize that acquiring high-level resources such as CPU and memory does not guarantee efficiency and performance, especially regarding the use of other computing system resources. The systems and methods described herein address this problem by prioritizing the diversity of low-level resources when allocating applications to computing system resources, ensuring that all available low-level resources are utilized.

[0056] Implementing diversity is a proactive resource management approach that dedicates expensive hardware resources to specific applications that use those resources, since meeting resource requirements is different from achieving performance. Traditional resource management systems such as Mesos and Quasar, which use centralized scheduling, are modified to allocate resources to these applications according to the scalable algorithm described in this paper, maximizing resource diversity rather than relying strictly on CPU and memory scheduling. The algorithm becomes scalable by dividing machines into sub-clusters for application allocation.

[0057] In an example embodiment, applications are allocated to a hardware resource cluster to optimize the diversity of hardware resource usage within the cluster by: pushing the hardware resources requested by the application to the capacity of each hardware resource; and allocating the application to a diverse pool of hardware resources based on a fairness algorithm (e.g., round-robin or proportional fairness algorithm) according to each application's hardware usage metric to maximize the diversity of hardware resource usage within the pool. Round-robin algorithms are typically used by process and network schedulers to allocate time slices to each process in an equally distributed manner, allowing all processes to be processed without prioritization. Proportional fairness, on the other hand, is a compromise-based scheduling algorithm that balances the attempt to maximize overall throughput with allowing all users (applications) at least a minimum service level by assigning each application a scheduling priority inversely proportional to its expected resource consumption. While prioritizing hardware utilization and performance, fairness is still guaranteed by imposing constraints on hardware allocation. In different configurations, the priorities of the corresponding applications may differ, the hardware may be characterized differently, and fairness guarantees may vary. However, the overall approach still works seamlessly to provide a holistic approach to maximizing resource utilization, reducing energy costs, and improving performance by packaging applications together.

[0058] It should be understood that, for certain private clouds and applications, it is desirable to package applications together based on hardware usage metrics, as companies may want to run different applications together and observe compatibility (e.g., without disruption) before implementing the usage-based resource allocation described herein. However, the advantage of the techniques described herein is that they can work in both public and private clouds without manual experimentation or specific expertise.

[0059] Figure 1 This illustrates a distributed resource management system that distributes applications to a cluster of nodes (actual machines or virtual machines) to improve diversity in an example embodiment. As shown, a cluster 104 with multiple applications 100 assigned to nodes 106 is configured by resource manager 102. In the example embodiment, a hardware performance counter 108 is associated with each application 100 to measure hardware usage metrics of low-level resources used by each application 100 and to categorize the hardware modules / resources emphasized by each application 100. While the applications are running in cluster 104, the hardware performance counter 108 can measure usage metrics for new and / or unpredictable applications in real time, and / or can obtain usage metrics offline for familiar applications in a private cloud or computer node simulating the operation of the applications in cluster 104. See below for more details. Figure 5To further explain, the resource manager 102 implements a distributed resource management algorithm 110 to allocate the applications 100 to a diverse pool using a fair algorithm such as a circular sorting algorithm, wherein the allocation is performed based on usage metrics for each application 100. In an example embodiment, metrics are used to measure the application's usage of the node's hardware resources, such as memory boundaries 112, CPU pipeline boundaries 114, disk boundaries 116, and cache coherence boundaries 118. In an example embodiment, other usage metrics may also be measured, such as processor pipeline utilization or occupancy, cache bandwidth, cache hit rate, cache pollution, memory bandwidth, non-consistent memory access latency, consistent traffic, etc.

[0060] To explain Figure 1 Operation of a distributed resource management system Figure 2 An example set of applications is shown for use in cluster 104 assigned to node 106 to enhance diversity through the techniques described herein. Table 200 shows application characteristics, including memory boundaries 202, CPU pipeline boundaries 204, disk boundaries 206, and cache coherence boundaries 208 measured for the respective application 100. As shown in Table 200, the respective applications 100 are identified as constrained by their memory usage characteristics (M1, M2, M3), by their CPU pipeline characteristics (C1, C2, C3), by their disk usage characteristics (D1, D2, D3), or by their cache coherence characteristics (CC1, CC2). Of course, the respective applications may be constrained by other low-level hardware characteristics and defined accordingly.

[0061] Figure 3 A sample set of applications 100 is shown, which allocate resources to cluster 104 using techniques such as conventional fine-grained resource counting to node 106. Figure 3 In the example, because the conventional resource management system is unaware of this characteristic of the corresponding application, when assigning application 100 to node 106 in cluster 104, it does not consider the usage metrics of low-level resources such as memory boundaries 202, CPU pipeline boundaries 204, disk boundaries 206, and cache coherence boundaries 208. Instead, application 100 is assigned to node 100 based on the usage of high-level resources (e.g., CPU cycles, memory access cycles, etc.), as described in Table 300. As shown in Table 300, assigning application 100 to node 106 regardless of the boundaries of the corresponding application characteristics on the corresponding node results in each machine being saturated under different pressures.

[0062] on the other hand, Figure 4An example of a cluster of nodes using a diverse resource management system, according to an example embodiment, is shown. In the allocation shown in Table 400, the applications are allocated to corresponding nodes 106 by taking into account the performance characteristics of each application 100 determined by hardware performance counter 108 and the hardware characteristics of the corresponding node 106. In particular, the applications 100 are allocated to maximize the diversity of resource usage on the corresponding nodes 106 to avoid performance boundaries for each application 100. In the example allocation in Table 400, no node has multiple applications with the same type of hardware boundaries. Generally, the applications 100 are allocated to corresponding nodes to maximize this diversity.

[0063] Typically, the CPU pipeline is the fastest low-level resource. Others are slower, and therefore can be saturated by having fewer CPU cores than are available. This is taken into account when assigning application 100 to the appropriate node (machine), where the allocation of application 100 depends on the different low-level resources of the node. Figure 4 In the example, applications 100 with different characteristics are packaged on the same node 106. The limits of resources available to application 100 on any node 106 are obtained by extrapolating the resources required by the applications to the hardware capabilities of the respective nodes, wherein the hardware capabilities are determined by micro-benchmark software 120 (…). Figure 1 The software 120 measures the maximum utilization capacity of each node for a specific resource, as characterized by [the following]. Figure 5 As explained, when assigning the corresponding application 100 to node 106 for processing, the hardware characteristics of the corresponding node 106 are also taken into account.

[0064] Figure 5 The example embodiment shows a resource manager 102 ( Figure 1 The flowchart illustrates a diversified resource management algorithm 500. The task of the diversified resource management algorithm 500 is to identify key performance characteristics by overlaying the application's hardware resource usage with the capabilities of the corresponding nodes (machines) 106 available for application allocation. Based on the critical hardware usage, the application 100 is added to an appropriate diversified pool. Figure 5In step 502, hardware performance counter 108 monitors the hardware resource usage of new application 100 in real time, or provides offline hardware resource usage metrics collected for application 100 as application history 504 to central resource manager 506 to establish application diversity classification (e.g., hardware resource boundary characteristics of the application) for allocating application 100 to diversity pool 508. At step 510, application 100 is classified into diversity pool 508 based on the current allocation (if application 100 is already running on node 106), or allocated to node 106 using a fair algorithm such as a round-robin routine, lowest allocation priority algorithm, dominant resource fairness algorithm, etc., to classify application 100 based on the eligibility of measured low-level resources 100.

[0065] Once the application 100 is categorized into diversity pool 508 at 510, the most qualified application from different diversity pools 508 is selected for each node 106 at 512. The hardware characteristics 514 of the respective node 106 available for application allocation within cluster 104, along with the diversity score of the respective node 106, are used as inputs to the application allocation step 512. As described above, the hardware characteristics 514 can be determined using micro-benchmarking techniques to identify the maximum utilization of each machine's hardware for a given resource. Typically, the goal is to select the node 106 with the least diversity and available hardware resources based on the diversity score, and to select from the diversity pool 508 applications constrained by that hardware resource but whose hardware resource usage, according to the measured metric of the application 100's usage of that hardware resource, will not saturate that hardware resource. If there are insufficient hardware resources to allocate the application 100, in 516, applications 100 are selected from each end of the diversity pool 508, as appropriate, selecting the application 100 using the most or least resources. The restrictions on excess resource allocation prevent unfair allocation and ensure fairness compared to other applications (which have lower resource eligibility). However, in this process, certain diversified combinations (CPU + memory + disk) can be selectively prioritized over other combinations (CPU + L2 cache + L3 cache) to further reduce interference. If no application from a different diversified pool 508 exists or an application from a different diversified pool 508 is ineligible, the most eligible application 100 is selected based on resource usage metrics.

[0066] Once application 100 has been allocated based on its usage characteristics to maximize the diversity of low-level hardware resource usage relative to the available hardware resources on the corresponding node 106, the application 100 and node database 520 are updated in 518 to reflect the diversity score of the node 106 to which application 100 has been allocated. Based on the updated application usage metrics and diversity scores provided in the updated database 520 of the nodes and based on the hardware characteristics 514 of the corresponding node 106, the diversity resource management algorithm 500 determines the resources available for allocation for each node 106 in 522. Then, in 524, a diversity score is calculated for each node 106 in the cluster 104. The diversity score is a function of the hardware characteristics 514 and the monitored usage metrics of the corresponding application 100 allocated to the corresponding node 106. The updated diversity score is used as further input to the node selection step 512 to identify the node with the lowest diversity still available for application allocation based on the existing hardware usage. After assigning application 100, the diversity score of the nodes can also be fed back as input to the fairness algorithm 510 to help classify the applications into the diversity pool 508. This process is repeated as long as there are applications 100 available for assignment. Figure 5 The process is shown in the diagram.

[0067] Therefore, unlike traditional resource application systems that fail to recognize performance saturation due to their lack of awareness of application characteristics, the system and method described herein identify performance saturation and allocate applications to maximize performance, thereby avoiding such performance limits as much as possible. Furthermore, since different hardware machines saturate under varying pressures, machine characteristics are similarly monitored to improve the efficiency of allocating applications to appropriate nodes. Through this approach, the performance of nodes executing various applications is improved by over 60%. Those skilled in the art will understand that by increasing resource utilization using the techniques described herein, most CPU-constrained applications also have room for improvement.

[0068] Those skilled in the art will understand that, as long as the overall approach to increasing hardware usage diversity is maintained, the techniques described herein can be used to prioritize applications differently, characterize hardware differently, and modify fairness guarantees, etc. Of course, the techniques described herein do not need to be used exclusively, but can complement conventional resource management systems that allocate resources based on the usage of high-level system elements such as CPU cycles and memory access cycles. It will also be understood that the techniques described herein can reduce energy costs and improve performance by intelligently packaging applications together to, for example, maximize the use of low-level components based on energy usage.

[0069] Those skilled in the art will also understand that the techniques described herein are particularly suitable for use in private clouds and applications, as companies or individuals maintaining private clouds have the ability to experimentally run different applications together and observe compatibility. However, the techniques described herein, by monitoring and measuring hardware usage metrics of new applications in real time or offline, are applicable to both public and private clouds, as described herein, without the need for manual experimentation / specialized skills. Therefore, the systems and methods described herein can improve resource utilization in clusters and data centers, reduce average latency in application execution, and improve energy and cost savings due to reduced time spent per application.

[0070] Figure 6 A block diagram of a circuit in the form of a processing system is shown, used to implement, for example... Figures 1 to 5 The example embodiments illustrate systems and methods for providing diverse resource management in a host computer cluster. Not all components need to be used in all embodiments. One example computing device in the form of computer 600 may include processing unit 602, memory 603, removable storage 610, and non-removable storage 612. While the example computing device is described as computer 600, the computing device may take different forms in different embodiments. For example, the computing device may also be a smartphone, tablet, smartwatch, or other device including... Figure 6 Computing devices with the same or similar elements shown and described. Devices such as smartphones, tablets, and smartwatches are generally referred to as mobile devices or user devices. Furthermore, although various data storage elements are described as part of the computer 600, the memory may also or optionally include cloud-based memory accessible via a network, such as memory accessed via the Internet or a server.

[0071] The memory 603 may include volatile memory 614 and non-volatile memory 608. The computer 600 may also include or have access to a computing environment comprising various computer-readable media, such as volatile memory 614 and non-volatile memory 608, removable memory 610, and non-removable memory 612. Computer memory includes random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CDROM), digital versatile disc (DVD) or other optical disc storage, magnetic tape, magnetic tape, disk storage or other magnetic storage devices, or any other medium capable of storing computer-readable instructions.

[0072] The computer 600 may include or access a computing environment including an input interface 606, an output interface 604, and a communication interface 616. The output interface 604 may include a display device, such as a touchscreen, which may also be used as an input device. The input interface 606 may include a touchscreen, touchpad, mouse, keyboard, camera, one or more device-specific buttons, one or more sensors integrated into the computer 600 or coupled to the computer 600 via a wired or wireless data connection, and one or more other input devices. The computer 600 can operate in a networked environment using a communication connection to connect to one or more remote computers, which may include a personal computer (PC), server, router, network PC, peer device, or other public DFD network switch, etc. The communication connection may include a local area network (LAN), a wide area network (WAN), cellular, Wi-Fi, Bluetooth, or other networks. According to one embodiment, various components of the computer 600 are connected to a system bus 620.

[0073] Computer-readable instructions, such as program 618, stored on a computer-readable medium, may be executed by processing unit 602 of computer 600. In some embodiments, program 618 includes software that, when executed by processing unit 602, performs diverse resource management operations according to any embodiments included herein. Hard disks, CD-ROMs, and RAM are some examples of articles that include non-transitory computer-readable media such as storage devices. The terms "computer-readable medium" and "storage device" do not include a carrier wave, such that a carrier wave is considered transient. Memory may also include network memory, such as a storage area network (SAN). The computer program 618 may also include instruction modules that, when processed, cause processing unit 602 to perform one or more methods or algorithms described herein.

[0074] In an example embodiment, the computer 600 includes: a usage determination module for determining the usage of corresponding hardware resources by an application to generate usage metrics; and a cluster allocation module for allocating the application to a hardware resource cluster to optimize the diversity of hardware resource usage in the cluster and improve the utilization of hardware resources by applications running in the cluster, wherein the diversity of hardware resource usage in the cluster is determined based on the corresponding usage metrics of the respective applications running in the cluster. In some embodiments, the computer 600 may include other or additional modules for performing any one or a combination of steps described in the embodiments. Furthermore, as shown in any of the drawings or described in any claim, any additional or alternative embodiments or aspects of the method may also include similar modules.

[0075] While some embodiments have been described in detail above, other modifications are possible. For example, the logical flow shown in the figures does not require a specific order or sequence to achieve the desired result. Additional steps may be provided from the flow, or steps may be removed from the flow, and other components may be added to or removed from the system. Other embodiments are within the scope of the following claims.

[0076] It should also be understood that software comprising one or more computer-executable instructions may be installed in and sold with one or more computing devices consistent with this disclosure, said one or more computer-executable instructions facilitating the processing and operation described above with reference to any or all of the steps of this disclosure. Alternatively, said software may be obtained and loaded into one or more computing devices, including obtaining the software via physical media or a distribution system, including, for example, obtaining the software from a server owned by the software creator or from a server not owned by the software creator but used by the software creator. For example, the software may be stored on a server for distribution via the Internet.

[0077] Furthermore, those skilled in the art will understand that this disclosure, in its application, is not limited to the details of the structure and arrangement of the components described in the embodiments or shown in the drawings. The embodiments herein may be other embodiments and can be practiced or performed in various ways. Furthermore, it will be understood that the wording and terminology used herein are for descriptive purposes only and should not be considered limiting. The terms “comprising,” “including,” or “having,” and variations thereof, as used herein, are intended to include items listed herein and their equivalents, as well as additional items. Unless otherwise limited, the terms “connection,” “coupling,” “installation,” and variations thereof are used broadly herein and include direct and indirect connections, couplings, and installations. Moreover, the terms “connection” and “coupling,” and variations thereof, are not limited to physical or mechanical connections or couplings.

[0078] The components of the illustrative devices, systems, and methods employed according to the illustrated embodiments may be implemented at least in part in digital electronic circuits, analog electronic circuits, or computer hardware, firmware, software, or combinations thereof. For example, these components may be implemented as computer program products, such as computer programs, program code, or computer instructions tangibly embodied in an information carrier or machine-readable storage device, for execution by or control of a data processing device, such as a programmable processor, computer, or multiple computers.

[0079] Computer programs can be written in any programming language, including compiled or interpreted languages, and can be deployed in any form, including as standalone programs or as modules, components, subroutines, or other units suitable for use in a computing environment. Computer programs can be deployed to execute on a single computer, multiple computers at a single site, or multiple sites interconnected via a communication network. Furthermore, programmers in the art related to the techniques described herein can readily interpret the functional programs, code, and code segments used to implement the techniques described herein as being within the scope of the claims. Computer programs, code, or instructions can be executed by one or more programmable processors to perform functions (e.g., by manipulating input data and / or generating output) to perform method steps associated with the illustrative embodiments. For example, the method steps can also be executed by special-purpose logic circuits such as field-programmable gate arrays (FPGAs) or application-specific integrated circuits (ASICs), and means for performing the method can be implemented as special-purpose logic circuits such as field-programmable gate arrays (FPGAs) or application-specific integrated circuits (ASICs).

[0080] The various illustrative logic blocks, modules, and circuits described in conjunction with the embodiments disclosed herein can be implemented using a general-purpose processor, a digital signal processor (DSP), an ASIC, an FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. The general-purpose processor can be a microprocessor, and optionally, it can also be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented using a combination of computing devices, such as a digital signal processor and a microprocessor, multiple microprocessors, one or more microprocessors coupled to a digital signal processor core, or any other similar configuration.

[0081] Processors suitable for executing computer programs include, for example, general-purpose and special-purpose microprocessors, and any one or more processors of any type of digital computer. Typically, the processor receives instructions and data from read-only memory or random access memory, or both. The necessary components of a computer are a processor for executing instructions and one or more storage devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or operatively coupled to receive data from and / or transfer data to one or more mass storage devices such as magnetic disks, magneto-optical disks, or optical disks. Information carriers suitable for embodying computer program instructions and data include all forms of non-volatile memory, including, for example, semiconductor storage devices, such as electrically programmable read-only memory or ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory devices, and data storage disks (e.g., magnetic disks, internal hard disks or removable disks, magneto-optical disks, CD-ROMs, and DVD-ROMs). The processor and memory may be supplemented by or integrated into dedicated logic circuitry.

[0082] Those skilled in the art will understand that information and signals can be represented using any of the different techniques and methods. For example, the data, instructions, commands, information, signals, bits, symbols, and chips mentioned above can be represented by voltage, current, electromagnetic waves, magnetic fields or particles, light fields or particles, or any combination thereof.

[0083] As used herein, “machine-readable medium” means a device capable of temporarily or permanently storing instructions and data, and may include, but is not limited to, random access memory (RAM), read-only memory (ROM), buffer memory, flash memory, optical media, magnetic media, cache memory, other types of memory (e.g., erasable programmable read-only memory (EEPROM)) and / or any suitable combination thereof. The term “machine-readable medium” should be considered to include a single medium or multiple media capable of storing processor instructions (e.g., a centralized or distributed database, or associated caches and servers). The term “machine-readable medium” should also be considered to include any medium or combination of media capable of storing instructions executable by one or more processors 602, such that when said instructions are executed by one or more processors 602, said one or more processors 602 perform any or more methods described herein. Therefore, “machine-readable medium” refers to a single storage device or apparatus, as well as a “cloud-based” storage system comprising multiple storage devices or apparatuses.

[0084] Those skilled in the art will understand that while example embodiments have been described in conjunction with methods, wherein the methods described in the example embodiments are used to provide diversified resource management when allocating applications to nodes (machines) in a hardware cluster, this disclosure is not limited thereto. For example, the techniques described herein can be used to allocate applications across virtual machines. The techniques described herein can also be used to allocate software to multiple nodes, thereby allowing multiple instances of an application to run on multiple threads on different machines. In these embodiments, the hardware resources can be used differently by different applications on different machines.

[0085] Furthermore, without departing from the scope of the invention, the technologies, systems, subsystems, and methods described and illustrated as discrete or separate in the various embodiments can be combined or integrated with other systems, modules, technologies, or methods. Other items shown or discussed as coupled or directly coupled or communicating with each other may also be indirectly coupled or communicating via an interface, device, or intermediate component in an electrical, mechanical, or other manner. Examples of other variations, substitutions, and modifications can be determined by those skilled in the art without departing from the spirit and scope of this document.

[0086] Although the invention has been described with reference to specific features and embodiments thereof, it will be apparent that various modifications and combinations thereof can be made without departing from the invention. The specification and drawings are to be regarded merely as an illustration of the invention as defined in the appended claims and any and all modifications, variations, combinations or equivalents falling within the scope of the invention are to be considered.

Claims

1. A computer-implemented method for distributed resource management in a distributed computing system, characterized in that, comprising: determining usage of an application program on a corresponding low-level hardware resource to generate a usage metric; determining hardware characteristics of a plurality of hardware resource nodes within a hardware resource cluster; generating a diversity score for the plurality of hardware resource nodes based on the hardware characteristics; the diversity score for the plurality of hardware resource nodes is indicative of low-level hardware resource diversity of the hardware resource nodes; selecting a least diverse hardware resource node from the plurality of hardware resource nodes based on the diversity score of the plurality of hardware resource nodes; allocating the application program to the least diverse hardware resource node to optimize diversity of usage of the low-level hardware resources in the hardware resource cluster and to improve utilization of low-level hardware resources by application programs running in the hardware resource cluster, wherein the diversity of usage of the low-level hardware resources in the hardware resource cluster is determined based on corresponding usage metrics of respective application programs running in the hardware resource cluster.

2. The method of claim 1, wherein, determining usage of the application program on the corresponding low-level hardware resource to generate the usage metric comprises determining usage of the application program on the corresponding low-level hardware resource with a hardware performance counter to generate the usage metric.

3. The method according to claim 1 or 2, characterized in that, determining usage of the application program on the corresponding low-level hardware resource to determine the usage metric is performed when the application program is running offline.

4. The method according to claim 1 or 2, characterized in that, determining usage of the application program on the corresponding low-level hardware resource to determine the usage metric is performed in real-time when the application program is executed by the low-level hardware resource.

5. The method according to claim 1 or 2, characterized in that, further comprising: characterizing the corresponding low-level hardware resource with micro-benchmarking to identify performance critical hardware resources.

6. The method of claim 1 or 2, wherein, allocating the application program to the least diverse hardware resource node to optimize diversity of usage of the low-level hardware resources in the hardware resource cluster comprises pushing hardware resources requested by the application program to the capabilities of each hardware resource and allocating the application program to a diversified pool of hardware resources based on the usage metric with a fair algorithm to maximize diversity of usage of the low-level hardware resources in the diversified pool.

7. The method of claim 1 or 2, wherein, allocating the application program to the least diverse hardware resource node to optimize diversity of usage of the low-level hardware resources in the hardware resource cluster comprises allocating the application program to a diversified pool of hardware resources, wherein the diversified pool of hardware resources is configured to minimize interference when application programs allocated to the diversified pool of hardware resources access the hardware resources in the diversified pool; and allocating application programs from different diversified pools to the least diverse hardware resource node.

8. The method of claim 1 or 2, wherein, allocating the application program to the least diverse hardware resource node to optimize diversity of usage of the low-level hardware resources in the hardware resource cluster comprises classifying the application program and other application programs into a diversified pool of hardware resources with a fair algorithm.

9. The method of claim 1 or 2, wherein, allocating the application to a hardware resource node with the least diversity of hardware resources to optimize the diversity of usage of the low-level hardware resources in the hardware resource cluster includes allocating the application from a corresponding diversification pool to the hardware resource node with the least diversity of hardware resources for processing.

10. The method of claim 1 or 2, wherein, Also included are: determining the performance of a module of a hardware resource by determining at least one of cache coherency, memory bandwidth, multi-level cache performance, processor bus performance, processor pipeline performance, cache bandwidth, cache hit rate, and non-uniform memory access latency.

11. An apparatus for providing distributed resource management in a distributed computing system, characterized in that, Included are: at least one hardware performance counter to generate a usage measure for a corresponding low-level hardware resource used by an application; a resource manager to determine hardware characteristics of a plurality of hardware resource nodes within a hardware resource cluster, generate a diversity score for the plurality of hardware resource nodes based on the hardware characteristics, select a hardware resource node with the least diversity of hardware resources from the plurality of hardware resource nodes according to the diversity score of the plurality of hardware resource nodes, and allocate the application to the hardware resource node with the least diversity of hardware resources according to the usage measure to optimize the diversity of usage of the low-level hardware resources in the hardware resource cluster and improve utilization of the low-level hardware resources by applications running in the hardware resource cluster, wherein the diversity score of the plurality of hardware resource nodes is to indicate low-level hardware resource diversity of a hardware resource node, and the diversity of usage of the low-level hardware resources in the hardware resource cluster is to be determined according to corresponding usage measures of corresponding applications running in the hardware resource cluster.

12. The apparatus of claim 11, wherein, determining the usage of the application by the corresponding low-level hardware resource includes utilizing a hardware performance counter to determine the usage of the application by the corresponding low-level hardware resource to generate the usage measure.

13. The apparatus of claim 12, wherein, the hardware performance counter is to determine the usage measure of the application when the application is running offline.

14. The apparatus of claim 12, wherein, the hardware performance counter is to determine the usage measure of the application in real-time when the application is being executed by the low-level hardware resource.

15. The apparatus of any one of claims 11 to 14, wherein, Also included are micro-benchmarking applications, wherein the micro-benchmarking applications are to run on nodes including the corresponding low-level hardware resources to identify performance-critical hardware resources of the nodes.

16. The apparatus of any one of claims 11 to 14, wherein, the resource manager is to allocate the application to the hardware resource node with the least diversity of hardware resources to optimize the diversity of usage of the low-level hardware resources in the hardware resource cluster by pushing low-level hardware resources requested by the application to the capabilities of each hardware resource and utilizing a fair algorithm to allocate the application to a diversification pool of hardware resources according to the usage measure to maximize the diversity of usage of the low-level hardware resources in the diversification pool.

17. The apparatus of any one of claims 11-14, wherein, The resource manager allocates the application to the least diverse hardware resource node to optimize the diversity of usage of the low-level hardware resources in the hardware resource cluster by allocating the application to a diversified pool of hardware resources, wherein the diversified pool of hardware resources is configured to minimize interference when applications allocated to the diversified pool of hardware resources access the hardware resources in the diversified pool of hardware resources, and allocating applications from different diversified pools to the least diverse hardware resource node.

18. The apparatus of any one of claims 11-14, wherein, The resource manager allocates the application to the least diverse hardware resource node to optimize the diversity of usage of the low-level hardware resources in the hardware resource cluster by classifying the application and other applications into diversified pools of hardware resources using a fair algorithm.

19. The apparatus of any one of claims 11-14, wherein, The resource manager allocates the application to the least diverse hardware resource node to optimize the diversity of usage of the low-level hardware resources in the hardware resource cluster by allocating applications from respective diversified pools to the least diverse hardware resource node for processing.

20. The apparatus of any one of claims 11-14, wherein, The hardware performance counter determines the performance of a module of a hardware resource by determining at least one of cache coherency, memory bandwidth, multi-level cache performance, processor bus performance, processor pipeline performance, cache bandwidth, cache hit rate, and non-coherent memory access latency.

21. A non-transitory computer readable medium, characterized in that, A computer program product for providing distributed resource management in a distributed computing system, the computer program product comprising computer instructions for causing one or more processors to perform the steps of: determining usage of an application on a respective low-level hardware resource to generate a usage metric; determining hardware characteristics of a plurality of hardware resource nodes within a hardware resource cluster; generating a diversity score for the plurality of hardware resource nodes based on the hardware characteristics; the diversity score for the plurality of hardware resource nodes is indicative of low-level hardware resource diversity of the hardware resource nodes; selecting a least diverse hardware resource node from the plurality of hardware resource nodes based on the diversity score of the plurality of hardware resource nodes; allocating the application to the least diverse hardware resource node based on the usage metric to optimize the diversity of usage of the low-level hardware resources in the hardware resource cluster and to improve utilization of low-level hardware resources by applications running in the hardware resource cluster, wherein the diversity of usage of the low-level hardware resources in the hardware resource cluster is determined based on the usage metric of respective applications running in the hardware resource cluster.

Images