Method for quantifying heterogeneous computing resource usage as single measurement unit

Abstract

To provide a method for quantifying resource usage.SOLUTION: A method for quantifying resource usage may include a step of measuring the amount of a plurality of physical computing resources used for a certain period of time. The method may also include a step of normalizing each measurement amount of each physical computing resource used. The method may also include a step of summing the normalized measurement amount of the physical computing resource used so as to produce a single usage amount representing the physical computing resource used over the period of time.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] This application claims the benefit of U.S. Provisional Patent Application No. 62 / 647,335, filed March 23, 2018, entitled "Quantifying Usage of Disparate Computing Resources as a Single Unit of Measure," and U.S. Provisional Patent Application No. 62 / 690,533, filed June 27, 2018, entitled "Quantifying Usage of Disparate Computing Resources as a Single Unit of Measure," both of which are incorporated herein by reference in their entireties. The present disclosure relates generally to computing, and more particularly to measuring usage of disparate computer resources and quantifying the usage as a single unit of measure. [Background technology]

[0002] Cloud computing allows users (e.g., individuals or organizations) to rent computing resources maintained and / or managed by a third party, thereby enabling users to rent computing resources tailored to their individual computing needs without having to make significant investments in physical hardware, data center real estate, electricity, data center personnel, and / or software licenses.

[0003] An example cloud computing system may include physical resources (e.g., processors and memory) and / or virtual resources (e.g., virtual processors and virtual memory). A user accessing a cloud computing system can rent an amount of resources capable of executing his or her workload. The amount of computing resources to be rented can be estimated based on a theoretical maximum workload. The theoretical maximum workload is assumed to use almost all (e.g., approximately 100%) of the available resources. In other words, a user rents the maximum amount of resources necessary to execute his or her workload to avoid system performance issues due to a sudden increase in resource requirements. However, usage spikes occur rarely, and the average required resources may be less than the theoretical maximum, leading to waste (e.g., renting unused computing resources).

[0004] Cloud computing providers may implement pricing models based on the amount of resources a user rents (at a given time), as opposed to the actual amount of resources used (or consumed). These resources may be packaged as servers with pre-configured central processing units (CPUs), memory (e.g., random access memory), storage, graphics processing units (GPUs), dedicated network capabilities (e.g., to provide a given network speed), field-programmable gate arrays (FPGAs), and / or any other computing resources. Packaged servers may generally be referred to as instances, which users can rent for a period of time (e.g., intervals of seconds, minutes, hours, days, weeks, months, years, and / or other time intervals). Thus, users may rent instances based on the maximum computing resources they estimate they will use (e.g., based on usage spikes). By renting resources in instances, users may only use a portion of the computing resources for most of the rental period. In other words, this results in users paying for resources they do not consume, and cloud computing providers may financially benefit from renting unused resources. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] U.S. Provisional Patent Application No. 62 / 647,335 [Patent Document 2] U.S. Provisional Patent Application No. 62 / 690,533 Summary of the Invention

[0006] These and other features and advantages will be better understood from the following detailed description taken in conjunction with the drawings. [Brief explanation of the drawings]

[0007] [Figure 1] 1 illustrates a schematic example of a computing network, according to an embodiment of the present disclosure. [Figure 2A] 2 illustrates a schematic example of a computing system usable in the computing network of FIG. 1, according to an embodiment of the present disclosure. [Figure 2B] 2B illustrates a schematic example of the computing system of FIG. 2A, according to an embodiment of the present disclosure. [Figure 3A] 3 illustrates a schematic example of an instrument for measuring usage of computing resources available in the computing system of FIG. 2, according to an embodiment of the present disclosure. [Figure 3B] 3B illustrates a schematic example of a metric / attribute collector usable with the instrument of FIG. 3A, according to an embodiment of the present disclosure. [Figure 3C] 3B shows a schematic example of an analytical platform usable with the instrument of FIG. 3A, according to an embodiment of the present disclosure. [Figure 3D] 1 is a plot (or graph) illustrating resource usage according to an embodiment of the present disclosure. [Figure 4] 4 illustrates a schematic example of a transform unit generator usable with the instrument of FIG. 3, according to an embodiment of the present disclosure. [Figure 5] 1 is an example flowchart of a method for quantifying usage of disparate computing resources as a single unit of measure according to an embodiment of the present disclosure. [Figure 6] 1 is an example flowchart of a method for quantifying the total amount of available heterogeneous computing resources (e.g., used and inactive resources) as a single unit of measure, according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0008] Generally, disclosed herein are systems, methods, and apparatus for measuring physical resource usage in computing and quantifying usage as a single unit of measure. For example, heterogeneous computing resources can be measured and normalized (e.g., converted) to a common unit of measure, and each of the normalized measurements can be summed so that total usage can be expressed as a single value. By expressing heterogeneous computing resources in a single unit, resource usage (or consumption) can be monitored in real time, and workloads can be optimized to improve utilization of available resources and / or reduce waste.

[0009] By measuring physical resources (as opposed to virtual resources), a more accurate measure of actual usage can be obtained. For example, due to the use of a technique known as ballooning, usage measurements based on virtual resources may overestimate the actual usage of physical resources. In other words, when measuring only virtual resources, the virtual resources may indicate that more resources are consumed (or used) by a workload than the workload actually consumes in the physical resources. In some cases, the amount of virtual resources indicated as consumed may exceed the available physical resources.

[0010] As used herein, a graphics processing unit (GPU) may generally refer to a computing resource that includes at least a graphics processor and / or graphics memory, although in some instances a GPU may be generally referred to as a graphics processing system (GPS).

[0011] 1 illustrates a schematic example of a computing network 100. As illustrated, computing network 100 includes devices 102 and computing systems 104, each communicatively coupled to a network 106 (e.g., the Internet). Computing systems 104 may include physical computing resources 108 configured to execute one or more workloads 110. One or more workloads 110 may include, for example, one or more of: one or more operating systems 112, one or more applications 114, and / or one or more idle processes 113. Idle processes 113 may be background processes associated with the operation of computing system 104 (e.g., as a result of operation of hardware in computing system 104) and may consume at least a portion of physical computing resources 108 even when no requests to execute applications 114 and / or operating systems 112 are received, for example, from devices 102. In other words, even when the computing system 104 is idle, the computing system 104 still consumes at least a portion of the physical computing resources 108 due to the operation of the physical hardware of the computing system 104 when it is powered on.

[0012] One or more of the workloads 110 may execute in response to a request generated by the device 102 (e.g., the operating system 112 and / or the applications 114). For example, the device 102 may request the execution of one or more applications 114 by communicating the request over the network 106 to the computing system 104. Upon receiving the request, the computing system 104 may allocate at least a portion of the physical computing resources 108 to the execution of the applications 114. In response to the applications 114 being executed, the computing system 104 may transmit data back to the device 102. For example, the data transmitted back to the device 102 may be used to generate a graphical user interface on the display 116 of the device 102. A user of the device 102 can then interact with the graphical user interface, which enables the device 102 to send further instructions to the computing system 104 that cause different portions of the applications 114 to execute on the computing system 104.

[0013] As shown, meters 118 may be provided. Meter 118 may include any combination of hardware, software, and / or firmware configured to measure usage of physical computing resources 108 used by workload 110 (e.g., idle process 113, operating system 112, and / or application 114) over a period of time (e.g., a predetermined or non-predetermined period of time). For example, instrument 118 may be implemented as software stored on one or more memories (e.g., any type of tangible, non-transitory storage medium, which may include any one or more of: magnetic recording media (e.g., hard disk drives), optical disks, semiconductor devices such as read-only memories (ROMs), random access memories (RAMs) such as dynamic and static RAMs, erasable programmable read-only memories (EPROMs), electrically erasable programmable read-only memories (EEPROMs), flash memory, magnetic or optical cards, or any type of storage medium for storing electronic instructions) and configured to be executed by one or more processors (e.g., processors commercially available from INTEL, ADVANCED MICRO DEVICES, IBM, ARM, ORACLE, and / or any other processor) to perform one or more operations. As a further example, instrument 118 may be implemented as circuitry (e.g., an application specific integrated circuit).

[0014] The usage of physical computing resources 108 may be measured using a host operating system (e.g., an operating system written in assembly language) and / or a hypervisor running on computing system 104. For example, meter 118 may include a plug-in database having one or more plug-ins configured to enable measurement of resource usage of a host operating system through a hypervisor running on computing system 104. As a further example, meter 118 may be configured to measure resource usage directly from physical computing resources 108 (e.g., with a host operating system). In some cases, meter 118 may measure physical computing resource usage across multiple different computing systems 104.

[0015] The meter 118 measures the usage of the physical computing resources 108, normalizes each of the physical computing resources 108 to a common unit of measurement, and combines the normalized computing resources so that the combined (e.g., aggregate) usage can be expressed as a single unit. In other words, the meter 118 may generally quantify the usage of multiple disparate computing resources as a single unit that represents the usage of the computing resources 108. For example, the meter 118 may be configured to measure the usage of multiple disparate computing resources over a period of time to generate a single usage value that represents the usage over that period of time.

[0016] To mitigate and / or avoid the effects of ballooning, the metering instruments 118 may, for example, measure physical resource usage as opposed to virtual resource usage. Thus, for example, the physical resource usage of a virtual machine may be more accurately measured. In some cases, the metering instruments 118 may also measure the maximum amount of available physical computing resources 108 (e.g., used and inactive resources), normalize the measured amount of each physical resource of the physical computing resources 108 to a common unit of measurement, and then sum the normalized resources to obtain the maximum amount of available physical resources expressed as a single unit. By measuring the maximum amount of available resources, a user may optimize the amount of physical computing resources 108 for executing the workload 110. Additionally or alternatively, the meter 118 measures the minimum amount of physical computing resources 108 required to run the computing system 104 (e.g., only the resources used by idle processes 113), normalizes the measured amount of each physical resource of the physical computing resources 108 to a common unit of measurement, and then sums the normalized resources to obtain a single unit representing the minimum amount of physical resources that need to be used (e.g., when the computing system 104 is running but is not running one or more operating systems 112 and / or applications 114).

[0017] In some cases, the meter 118 can be configured to measure the usage of a physical resource over a period of time (e.g., a resource usage billing period, a contract period, an hour, a day, a week, and / or any other period of time) and generate a single usage value that represents the usage of the physical resource over that period of time. The measured usage can then be utilized to determine the appropriate amount of resource to purchase over a period of time and / or used to generate billing based on actual usage. As a result, resource consuming users can reduce the amount of resource they purchase, and resource providers can reduce the total amount of resource they offer for sale and / or allocate unused resources to other users.

[0018] In some cases, the meter 118 can be configured to enable and / or disable access of the device 102 to the computing resources 108 of the computing system 104. In other words, the meter 118 can be configured to selectively communicatively couple and / or decouple the device 102 from the computing system 104. For example, the meter 118 can disable access by the device 102 to at least a portion of the computing resources 108 after a period of time (e.g., lapse of a contracted period), in response to a breach of a contractual provision (e.g., default on payment, misuse of computing resources, and / or the like), and / or in response to measured resource usage over a period of time exceeding or not exceeding a predetermined amount (or threshold). As a further example, the meter 118 can enable access to the computing resources 108 at a predetermined time (e.g., at the start of a contracted period).

[0019] In some cases, meter 118 may be configured to selectively enable and disable access to at least a portion of computing resource 108 based on availability. For example, device 102 may access at least a portion of computing resource 108 when other users are not using the computing resource 108. However, when other users request access to computing resource 108, a recall request may be sent to meter 118 that causes meter 118 to disable device 102's access to at least a portion of computing resource 108, allowing the other users to access computing resource 108. As a result, meter 118 may generally be configured to reduce the amount of computing resource 108 that is not being used to run, for example, one or more applications 114 or operating system 112.

[0020] As shown, the meter 118 may be communicatively coupled to the device 102, the computing system 104, and / or the network 106. In some cases, at least a portion of the meter 118 may be included in the device 102 and / or the computing system 104. Additionally or alternatively, at least a portion of the meter 118 may be included in a third-party device that is communicatively coupled (e.g., using the network 106) to one or more of the device 102 and / or the computing system 104.

[0021] The devices 102 may include any one or more of a personal computer, a tablet computer, a mobile phone, a smartphone, a smart watch, a smart TV, a fitness tracker, a smart scale (for weighing objects), a smart thermostat, a smart security monitoring system (e.g., indoor and / or outdoor camera systems, doorbells, alarm systems, locking systems, lighting systems, and / or any other smart security monitoring system), a smart display (e.g., a display configured to communicate with the network 106), a smart medical system (e.g., an ultrasound system, a molecular imaging system, a computed tomography system, an X-ray system, and / or any other smart medical system), a remote health management device, a gaming device (e.g., a gaming console, a handheld gaming device, a virtual reality headset, and / or any other gaming device), an Internet of Things device (e.g., an autonomous vehicle, a connected appliance, a drone, a robot, and / or any other Internet of Things device), a server, and / or any other device capable of communicating with the computing system 104.

[0022] Figure 2A illustrates a schematic embodiment of the computing system 104 of Figure 1. As illustrated, the physical computing resources 108 of the computing system 104 may include a central processing unit (CPU) 202, memory 204, storage 206, a network interface 208, and a graphics processing unit (GPU) 210. Each of the CPU 202, memory 204, storage 206, network interface 208, and GPU 210 has one or more metrics / attributes associated therewith.

[0023] Additionally or alternatively, computing resources 108 may include other resources 209. Other resources 209 may include, for example, any one or more of a field programmable gate array (FPGA), a tensor processing unit (TPU), an intelligence processing unit (IPU), a neural processing unit (NPU), a vision processing unit (VPU), a digital signal processor (DSP), an accelerated processing unit (APU), an application specific integrated circuit (ASIC), a system on a chip (SoC), a programmable SoC, an application specific standard product (ASSP), an adaptive computing acceleration platform (ACAP), a microcontroller, and / or any other computing resource.

[0024] 2B , CPU 202 may be associated with the following metrics / attributes: number of processing cores 212, processor clock speed 214, and / or processor load 216. Number of processing cores 212 indicates the number of independent processing units (or cores) available to CPU 202. Processor clock speed 214 indicates the default clock speed of each of CPU 202's cores. Processor load 216 indicates the amount of computational work performed by CPU 202. CPU 202 may be any computer processor, including, for example, single and / or multi-core processors capable of executing computer instructions. Examples of CPU 202 may include commercially available processors from INTEL, ADVANCED MICRO DEVICES, IBM, ARM, ORACLE, and / or any other processor.

[0025] The memory 204 may be associated with metrics / attributes of total available memory 218 and / or memory usage 220. The memory 204 may be tangible, non-transitory memory. The memory 204 may be volatile memory. For example, the memory 204 may be random access memory (RAM). Examples of RAM may include, for example, static RAM (SRAM), dynamic RAM (DRAM), fast page mode DRAM (FPM DRAM), extended data out DRAM (EDO DRAM), synchronous dynamic RAM (SDRAM), double data rate SDRAM (DDR SDRAM), RAMBUS dynamic RAM (RDRAM), and / or any other form of RAM.

[0026] Storage 206 may be associated with metrics / attributes of total available storage 222, total storage used 224, number of bytes written to storage 206 226, and / or number of bytes read from storage 206 228. Storage 206 may be non-volatile memory. For example, storage 206 may include any type of tangible, non-transitory storage medium, including any one or more of magnetic recording media (e.g., hard disk drives), optical disks, semiconductor devices such as read-only memory (ROM), random access memory (RAM) such as dynamic and static RAM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic or optical cards, or any type of storage medium for storing electronic instructions.

[0027] The network interface 208 may be associated with metrics / attributes for the number of bytes sent 230 and / or the number of bytes received 232. The network interface 208 may be configured to communicatively couple to the network 106 (FIG. 1). For example, the network interface 208 may be a network interface controller configured to communicatively couple to an Ethernet network, a wireless network, a Fibre Channel network, a Fiber Distributed Data Interface (FDDI) network, a Copper Distributed Data Interface (CDDI) network, and / or any other network.

[0028] GPU 210 may be associated with the following metrics / attributes: graphics processor default clock speed 234, graphics processor variable clock speed 236, graphics processor load 238, graphics memory default clock speed 240, variable graphics memory clock speed 242, graphics memory load 244, graphics memory bus 246, total available graphics memory 248, graphics memory usage 250, and / or shader quantity 252. Graphics processor default clock speed 234 indicates the default clock speed of the processor of GPU 210, and graphics processor variable clock speed 236 indicates the variable clock speed of the graphics processor. In some cases, graphics processor variable clock speed 236 may exceed the clock speed of graphics processor default clock speed 234. In other cases, graphics processor variable clock speed 236 does not exceed the graphics processor default clock speed 234. Similarly, graphics memory default clock speed 240 indicates the default clock speed of the graphics memory, and variable graphics memory clock speed 242 indicates the clock speed at which the graphics memory is actually running. In some cases, variable graphics memory clock speed 242 may be greater than default graphics memory clock speed 240. In other cases, variable graphics memory clock speed 242 may be less than (or equal to) default graphics memory clock speed 240. Graphics memory load 244 indicates the amount of read and / or write operations performed on GPU memory. Graphics memory bus 246 indicates the bus size of GPU 210.

[0029] GPU 210 may include an integrated and / or dedicated GPU. For example, the GPU may include any of the GPUs manufactured by INTEL, NVIDIA, ADVANCED MICRO DEVICES, and / or any other GPU. When GPU 210 includes an integrated GPU, the metrics / attributes may be based at least in part on shared resources of the integrated GPU.

[0030] Although CPU 202, memory 204, storage 206, network interface 208, and GPU 210 are shown in the singular, it should be understood that computing system 104 may include multiple physical CPUs, memories, storages, network interfaces, and GPUs, one or more of which may be configured to work together, for example, to improve the performance of computing system 104. Similarly, if computing system 104 includes one or more of other resources 209, there may be one or more of each of other resources 209.

[0031] 3A illustrates a schematic example of an instrumentation 118 of FIG. 1. As illustrated, the instrumentation 118 may include a metric / attribute collector 302, a calculator 305, and an analytics platform 312 communicatively coupled to the physical computing resources 108 (FIG. 2). Each of the metric / attribute collector 302, calculator 305, and / or analytics platform 312 may be distributed across different computing systems (or environments). In some cases, one or more of the metric / attribute collector 302, calculator 305, and analytics platform 312 may be located on the same computing system. For example, one or more of the metric / attribute collector 302, calculator 305, and / or analytics platform 312 may be located on the device 102. As a further example, one or more of the metric / attribute collector 302, calculator 305, and analytics platform 312 may be located on separate devices communicatively coupled to the network 106 and / or the computing system 104. As a further example, one or more of the metric / attribute collector 302 , the calculator 305 , and the analysis platform 312 may be located on the computing system 104 .

[0032] Metric / attribute collector 302 may request one or more metrics / attributes from one or more of CPU 202, memory 204, storage 206, network interface 208, GPU 210, and / or other resources 209. For example, metric / attribute collector 302 may request one or more metrics / attributes from a hypervisor (e.g., using a plug-in) running on one or more of CPU 202, memory 204, storage 206, network interface 208, GPU 210, and / or other resources 209. As a further example, metric / attribute collector 302 may request one or more metrics / attributes from a host operating system (e.g., an operating system written in assembly language) running on one or more of CPU 202, memory 204, storage 206, network interface 208, GPU 210, and / or other resources 209.

[0033] When metric / attribute collector 302 requests one or more metrics / attributes for CPU 202, the requested metrics / attributes may be input to CPU cycle utilization calculator 304. CPU cycle utilization calculator 304 may generate a metric / attribute corresponding to utilized CPU cycle utilization based at least in part on the requested metrics / attributes for CPU 202. For example, CPU cycle utilization calculator 304 may generate CPU cycle utilization based at least in part on number of processing cores 212, processor clock speed 214, and processor load 216. In these cases, CPU cycle utilization calculator 304 may generate utilized CPU cycles according to the following equation:

number

[0034] When metric / attribute collector 302 requests one or more metrics / attributes from GPU 210, the requested metrics / attributes may be input to GPU processor cycle utilization calculator 306 and / or GPU memory cycle utilization calculator 308. GPU processor cycle utilization calculator 306 and / or GPU memory cycle utilization calculator 308 may generate metrics / attributes corresponding to GPU processor cycle utilization and GPU memory cycle utilization, respectively, based at least in part on the metrics requested from GPU 210.

[0035] For example, GPU processor cycle utilization calculator 306 may generate the GPU processor cycle utilization based at least in part on graphics processor variable clock speed 236 and graphics processor load 238. In these cases, GPU processor cycle utilization calculator 306 may generate the utilized GPU processor cycles according to the following equation:

number

[0036] As a further example, GPU memory cycle utilization calculator 308 may generate GPU cycle utilization based at least in part on variable graphics memory clock speed 242 and graphics memory load 244. In these cases, GPU memory cycle utilization calculator 308 may generate consumed GPU memory clock cycles according to the following equation:

number

[0037] When metric / attribute collector 302 requests one or more metrics / attributes from storage 206, the metrics / attributes can be input to disk IO calculator 310. Disk IO calculator 310 can generate a metric / attribute corresponding to the total number of bits read from and written to storage 306 over a period of time (e.g., one second). Thus, in some cases, disk IO calculator 310 can add the number of bytes written 226 and the number of bytes read 228 over that period to generate a metric / attribute corresponding to the disk IO. If the period is greater than one second, then the number of bytes written 226 and the number of bytes read 228 can be divided by the total number of seconds. If the period is less than one second, then the number of bytes written 226 and the number of bytes read 228 can be divided by a fraction of a second.

[0038] When metric / attribute collector 302 requests one or more metrics / attributes from network interface 208, the metrics / attributes can be input to network IO calculator 311. Network IO calculator 311 can generate a metric / attribute corresponding to the total number of bits sent and received by network interface 208 over a period of time (e.g., one second). Thus, in some cases, network IO calculator 311 can add the number of bytes sent 230 and the number of bytes received 232 over that period to generate a metric / attribute corresponding to the network IO. If the period is greater than one second, then the number of bytes sent 230 and the number of bytes received 232 can be divided by the total number of seconds. If the period is less than one second, then the number of bytes sent 230 and the number of bytes received 232 can be divided by a fraction of a second.

[0039] As shown, the analytics platform 312 can receive one or more metrics / attributes related to the physical computing resources 108, including, for example, one or more metrics / attributes output from one or more of the calculators 305. In some cases, the analytics platform 312 can include one or more resource calculators 331 corresponding to one or more of the other resources 209, respectively. The one or more metrics / attributes output from one or more of the calculators 305 can include, for example, one or more of CPU cycle utilization (e.g., output from CPU cycle utilization calculator 394), GPU processor cycle utilization (e.g., output by GPU processor cycle calculator 306), GPU memory cycle utilization (e.g., output by GPU memory cycle utilization calculator 308), disk IO (e.g., output by disk IO calculator 301), and / or network IO (e.g., output by network IO calculator 311). The analytics platform 312 can transform the received metrics / attributes using one or more transformation units. In some cases, each metric / attribute has a corresponding translation unit. For example, the analytics platform 312 may utilize a CPU translation unit 314, a memory translation unit 316, a storage translation unit 318, a network translation unit 320, a GPU processor translation unit 322, a GPU memory translation unit 324, a GPU memory speed translation unit 326, a disk IO translation unit 328, and / or any other translation unit corresponding to a computing resource (e.g., a translation unit for one or more of other resources 209, such as FPGA resources, ASIC resources, SoC resources, DSP resources, microcontroller resources, ACAP resources, and / or the like).

[0040] The analysis platform 312 may convert the heterogeneous computing resources into normalized units by dividing the usage of each physical resource (e.g., CPU 202, memory 204, storage 206, network interface 208, and GPU 210) by the respective conversion unit corresponding to the physical resource (e.g., CPU conversion unit 314, memory conversion unit 316, storage conversion unit 318, network conversion unit 320, GPU processor conversion unit 322, GPU memory conversion unit 324, GPU memory speed conversion unit 326, and disk IO conversion unit 328) to obtain the normalized respective usage of each physical resource. In some cases, the analysis platform 312 may be configured to convert one or more of the other resources 209 into standard units. For example, the analysis platform 312 may include an other resource conversion unit 329.

[0041] The normalized usage amounts may then be summed to obtain a total usage value. Additionally or alternatively, as will be readily understood in light of the disclosure herein, in some cases, the normalized usage amounts may be summed according to computing resource type (e.g., one or more of CPU 202, memory 204, storage 206, network interface 208, GPU 210, and / or other resources 209) to obtain a normalized usage value for a particular resource. For example, the normalized usage amounts of the physical resources of GPU 210 (e.g., GPU processor and / or GPU memory) may be summed to generate a total GPU usage. Other examples of the normalized usage of a particular physical resource may include a total FPGA usage, a total ASIC usage, a total DSP usage, a total SoC usage, a total microcontroller usage, a total ACAP usage, a total TPU usage, and / or any other total usage amount corresponding to the respective resource.

[0042] 3B illustrates metric / attribute collector 303, which may be an embodiment of metric / attribute collector 302, configured to monitor resource usage for a number (e.g., a plurality) of different users and associate each user's usage with the respective user. In other words, metric / attribute collector 303 is configured to receive metrics corresponding to resource usage. Metric / attribute collector 303 may include any combination of hardware, software, and / or firmware.

[0043] The metric / attribute collector 303 can be configured for inclusion with any one or more of the computing system 104, the device 102, and / or any other device communicatively coupled to the network 106. For example, the device 102 can include the metric / attribute collector 303. When the device 102 includes the metric / attribute collector 303, a user of the metric / attribute collector 303 can utilize the instruments 118 with fewer modifications to the user's network security compared to when the metric / attribute collector 303 is not running on the device 102.

[0044] As shown, metric / attribute collector 303 is communicatively coupled to computing system 104 via communication link 330. Communication link 330 may transmit data (e.g., metrics / attributes related to computing resources 108) to metric / attribute collector 303. Data transmitted over communication link 330 may be encrypted to prevent unauthorized access to the data. As shown, metric / attribute collector 303 may be remote from computing system 104 (e.g., operating on a different server, in a different network, and / or on a different device). However, in some cases, metric / attribute collector 303 may be local to computing system 104 (e.g., operating on the same server, on the same network, and / or on the same device).

[0045] The metric / attribute collector 303 may also be configured to communicate with the analytics platform 312 via communication link 362. Data transmitted over communication link 362 may be encrypted to prevent unauthorized access to the data. In some cases, the metric / attribute collector 303 may include the analytics platform 312. Thus, the metric / attribute collector 303 may not transmit data over communication link 362.

[0046] The analytics platform 312 can generate a total resource usage value expressed as a single normalized unit. As shown, the analytics platform 312 can be remote from the metrics / attribute collector 303 (e.g., operating on a different server, in a different network, and / or on a different device). However, in some cases, the analytics platform 312 can be local to the metrics / attribute collector 303 (e.g., operating on the same server, on the same network, and / or on the same device).

[0047] As shown, metric / attribute collector 303 can include at least one collector core 332, at least one collector metric receiver 334, at least one collector synchronizer 336, at least one plug-in database 338, at least one collector automator 340, at least one machine manager 342, at least one database manager 344, at least one account manager 346, and at least one machine collection manager 348. Collector core 332 communicates with and / or manages each of collector metric receiver 334, collector synchronizer 336, plug-in database 338, collector automator 340, machine manager 342, collector database manager 344, account manager 346, and machine collection manager 348.

[0048] Collector metrics receiver 334 is configured to receive metrics / attributes representing one or more users and / or workloads' usage of computing resources 108. In some cases, collector metrics receiver 334 may include one or more of calculators 305.

[0049] At least a portion of the resource usage can be associated with one or more users. For example, at least a portion of the resource usage can be associated with one or more accounts 350 using the account manager 346. The accounts 350 can be associated with one or more resource-consuming users 352. As a result, resource usage for multiple accounts 350 and / or resource-consuming users 352 can be monitored simultaneously. For example, a business entity can have accounts 350 associated with its employees (e.g., resource-consuming users 352). As a result, when an employee accesses computing resources 108, usage is associated with the account 350 corresponding to the business entity. By associating usage with accounts 350, multiple accounts 350 can be monitored using a single metric / attribute collector 303, which can consume fewer resources compared to associating a metric / attribute collector 303 with each account 350 and / or resource-consuming user 352.

[0050] An account 350 may be associated with one or more machines 354 (e.g., virtual or physical) accessible to a resource consuming user 352 and operating using computing resources 108. A machine 354 may include, for example, one or more of an operating system 112 and / or applications 114. For example, a machine 354 may be configured to run a Linux-based operating system, a Windows-based operating system, a MAC-based operating system, and / or any other computing system in a virtual or physical environment. In some cases, a machine 354 may be configured to run one or more workloads 110 (e.g., applications 114) without running a guest operating system. For example, a machine 354 may be configured to run one or more applications 114 directly on a physical machine without any form of virtualization, or may be configured to run a container and execute one or more applications 114 within the container. Thus, a container may generally be an isolated environment configured to run on one or more kernels of a machine 354. A container may run on a physical or virtual machine.

[0051] The machine collection manager 348 can associate multiple machines 354 with at least one machine collection 356. The machine collection manager 348 can also associate multiple containers or uniquely identified (e.g., tagged) workloads with at least one collection. In some cases, multiple machine collections 356 can be associated with each account 350. For example, the owner (or authorized user) of each account 350 can add one or more machines 354 to each machine collection 356. Each machine collection 356 can include multiple machines 354, each of which can be configured to run, for example, a different operating system 112 and / or application 114 (e.g., the application 114 can be executed using a container).

[0052] The plug-in database 338 can be configured to be accessed by the metric / attribute collector 303 and can include one or more plug-ins 358 (e.g., plug-ins 358 can be drivers, listeners, publishers, or the like) that enable the metric / attribute collector 303 to communicate with one or more computing systems 104 to, for example, measure resource usage. Each computing system 104 can be configured to include one or more machines 354, each configured to run a different operating system 112 and / or application 114. In other words, the plug-ins 358 enable the metric / attribute collector 303 to simultaneously monitor resource usage across multiple different computing environments. For example, plugins 358 may include a LINUX plugin, a WINDOWS plugin, a MAC plugin, a SOLARIS plugin, a VMWARE plugin, a HYPER-V plugin, a CITRIX XEN plugin, an OPENSTACK plugin, an OPENSHIFT plugin, a KUBERNETES plugin, a DOCKER plugin, a PIVOTAL plugin, an AMAZON plugin, a MICROSOFT plugin, a GOOGLE plugin, a bare metal plugin (e.g., a plugin configured to communicate directly with non-virtual machines), and / or any other plugin that enables monitoring of usage across multiple operating systems / environments. Plugins 358 may also enable monitoring of one or more virtual and / or physical environments simultaneously. Plugins 358 may also enable monitoring of billing information, software program usage, rented software licenses, and / or any other additional user account information.

[0053] The plugins 358 can be configured to be activated (e.g., the metric / attribute collectors 303 can communicate with their respective computing systems 104 using their respective plugins 358) or deactivated (e.g., the metric / attribute collectors 303 cannot communicate with their respective computing systems 104 using their respective plugins 358).

[0054] Machine manager 342 may assign and / or associate machines 354 with respective machine collections 356. Machine manager 342 may analyze machines 354 in machine collections 356 to determine, for example, the specifications and / or operating systems of machines 354. The specifications may include, for example, information related to computing resources 108 (e.g., available memory, storage, and / or the like).

[0055] Machine manager 342 can also search for new machines to be monitored by metric / attribute collector 303. Machine manager 342 can detect new machines by searching for new IP addresses associated with particular accounts 350 and / or resource consuming users 352.

[0056] Collector database manager 344 manages data received and / or generated by metric / attribute collector 303. For example, collector database manager 344 may receive and / or store one or more records (e.g., data) related to machines 354, machine collections 356, accounts 350, and their associated resource usage (e.g., metrics / attributes). Collector database manager 344 may also purge data. For example, collector database manager 344 may purge records related to each machine 354, each machine collection 356, and / or each account 350. In some cases, collector database manager 344 may truncate records related to each machine 354, each machine collection 356, and / or each account 350. The purging and / or truncation may occur, for example, after a period of time and / or after deactivation / termination of each account 350.

[0057] Collector database manager 344 can communicate with collector database 364 via communication link 366 to store one or more records on collector database 364. As shown, collector database 364 can be remote from metric / attribute collector 303 (e.g., operating on a different server and / or different network). However, in some cases, collector database 364 can be local to metric / attribute collector 303 (e.g., operating on the same server, the same network, and / or the same device). Data transmitted over communication link 366 can be encrypted to prevent unauthorized access to the data during transmission. In some cases, records stored on collector database 364 can be encrypted to prevent unauthorized access to the records. For example, records stored on collector database 364 can be encrypted using transparent data encryption (TDE).

[0058] Collector synchronizer 336 may, for example, communicate with collector database manager 344 and may replicate and / or back up data received by collector database manager 344. For example, collector synchronizer 336 may replicate one or more records stored and / or to be stored on collector database 364 to collector backup 368. In some cases, records stored on collector backup 368 may be encrypted to prevent unauthorized access. When changes are made to one or more records, collector synchronizer 336 may replicate and / or back up only the changes to collector backup 368. As a result, collector synchronizer 336 does not replicate entire records to collector backup 368. As shown, collector synchronizer 336 may communicate with collector backup 368 via communication link 370. Data transmitted via communication link 370 may be encrypted to prevent unauthorized access to the data in transit. As shown, collector backup 368 may be remote (e.g., running on a different server, in a different network, and / or on a different device) from metric / attribute collector 303. However, in some cases, collector backup 368 may be local to metric / attribute collector 303 (e.g., running on the same server, on the same network, and / or on the same device).

[0059] In some cases, at least one backup collector 397 may be provided. The backup collector 397 performs substantially the same operations as the metric / attribute collector 303 and may be configured to run in parallel (or simultaneously) with the metric / attribute collector 303. In some cases, the backup collector 397 is configured to be remote from the metric / attribute collector 303 (e.g., operating on a different server, in a different network, and / or on a different device). As a result, in the event of a failure of the metric / attribute collector 303, the instrument 118 can continue to monitor resource usage. In other words, the backup collector 397 may generally provide redundancy. In some cases, there may be two or more backup collectors 397, e.g., to achieve n+2 redundancy. Thus, the backup collector 397 may include any one or more of the functionality discussed herein with respect to the metric / attribute collector 303. The backup collector 397 may communicate with the computing system 104 via a communication link 399. Data transmitted over communication link 399 may be encrypted to prevent unauthorized access to the data during transmission.

[0060] Backup collector 397 can also be configured to generate independent records that can be used, for example, during audit processes and / or in data recovery processes (e.g., if metric / attribute collector 303 experiences a failure). In some cases, collector backup 368 can include backup collector 397.

[0061] As shown, the metric / attribute collector 303 can also include a collector automator 340. The collector automator 340 can automate one or more functions of the metric / attribute collector 303. For example, the collector automator 340 can be configured to receive a trigger event that causes the metric / attribute collector 303 to perform a predetermined action (e.g., start or stop receiving metrics / attributes, start database cleanup, enable or disable access to the computing resources 108, activate or deactivate one or more plug-ins 358, and / or any other action). The trigger event can include, for example, the discovery of a new machine, the expiration of a contract term, the use of a predetermined amount of resources, the addition of a new machine to a respective machine collection 356, and / or the like. In some cases, the trigger event can be the expiration of a period of time. For example, every 1, 2, 3, 4, or 5 seconds (or any other suitable interval), metric / attribute collector 303 may look for new Internet Protocol (IP) addresses corresponding to new machines. In these cases, metric / attribute collector 303 can generally be described as self-updating.

[0062] As also shown, an administrator user interface 372 can be communicatively coupled to the collector core 332. The administrator user interface 372 can provide an interface for an operator of the metric / attribute collector 303 to control and / or modify the metric / attribute collector 303 and / or review data generated and / or received by the metric / attribute collector 303. For example, the administrator user interface 372 can be configured to display and / or generate usage summaries on an account 350 or resource consumption user 352 basis. The administrator user interface 372 can also be used, for example, to manually add one or more machines 354, adjust and / or delete one or more records, add or remove new accounts 350, add or remove plug-ins 358, adjust / create triggers for the collector automator 340, continue and / or discontinue monitoring resource usage at predetermined times, e.g., in response to a contract period, and / or any other administrative task.

[0063] As shown, administrator user interface 372 can communicate with collector core 332 via communication link 374. Data transmitted via communication link 374 can be encrypted to prevent unauthorized access to the data in transit. As shown, administrator user interface 372 can be remote from metric / attribute collector 303 (e.g., operating on a different server, in a different network, and / or on a different device). However, in some cases, administrator user interface 372 can be local to metric / attribute collector 303 (e.g., operating on the same server, on the same network, and / or on the same device).

[0064] In some cases, for example, administrator user interface 372 may be generated and displayed on a web browser. As a further example, in some cases, administrator user interface 372 may be an application running on a computer that communicates with metric / attribute collector 303 over a network connection (e.g., the Internet). As a further example, in some cases, administrator user interface 372 may be an application running locally (e.g., on the same device as metric / attribute collector 303).

[0065] FIG. 3C illustrates an example analytics platform 360, which may be an example of analytics platform 312 in FIG. 3A. Analytics platform 360 may generally be configured to normalize (e.g., convert) multiple metrics / attributes received over a period of time to normalized units and sum the normalized metrics / attributes to generate a total resource usage value expressed as a single normalized unit for the period of time. In other words, analytics platform 360 may be configured to generate a single usage value representing the total resource usage over a period of time. Analytics platform 360 may include any combination of hardware, software, and / or firmware. In some cases, analytics platform 360 may run on the same server and / or the same network as metric / attribute collector 302. In other cases, analytics platform 360 may run on a different server and / or a different network (e.g., on a user's network and / or server) than metric / attribute collector 302.

[0066] The analytics platform 360 may be configured to receive one or more metrics / attributes indicative of usage of the computing resources 108 from one or more of the metric / attribute collector 302 and / or one or more backup collectors 397. In some cases, the analytics platform 360 may receive the one or more metrics / attributes from the computing system 104. In these cases, the instrumentation 118 may not include a metric / attribute collector 302, and the analytics platform 360 may include a plug-in database 338. The analytics platform 360 may receive the metrics / attributes in the form of, for example, a JavaScript Object Notation (JSON) file, a Comma Separated Values ​​(CSV) file, and / or any other file type.

[0067] The analytics platform 360 can be configured for inclusion with any one or more of the computing system 104, the device 102, and / or any other device communicatively coupled to the network 106. For example, the device 102 can include the analytics platform 360. When the device 102 includes the analytics platform 360, users of the analytics platform 360 can utilize the meters 118 with fewer modifications to the user's network security compared to when the analytics platform 360 is not running on the device 102. In some cases, when the device 102 includes the analytics platform 360, the analytics platform 360 can be configured to analyze resource usage of one or more users, e.g., on a local network, each having their own device. Additionally or alternatively, when the device 102 includes both the analytics platform 360 and the metric / attribute collector 302, only metrics / attributes indicative of usage of the computing resources 108 need be transmitted over a public network (e.g., the Internet).

[0068] Analytics platform 360 may include at least one analytics core 376, at least one analytics metric receiver 378, at least one analytics synchronizer 380, at least one analytics database manager 382, ​​at least one analytics automator 384, at least one usage analyzer 386, at least one notification generator 388, at least one subscription manager 390, and at least one exchange / contract manager 393. Analytics core 376 may generally be in communication with and / or manage each of analytics metric receiver 378, analytics synchronizer 380, analytics database manager 382, ​​analytics automator 384, usage analyzer 386, notification generator 388, and subscription manager 390.

[0069] As shown, the analytics core 376 is communicatively coupled to an analytics metric receiver 378. The analytics metric receiver 378 receives metrics / attributes representing usage of the computing resources 108 from the metrics / attributes collector 302. The analytics metric receiver 378 may include one or more of the calculators 305. In some cases, the analytics metric receiver 378 may be configured to generate a total resource usage value expressed as a single normalized unit. In these cases, the analytics core 376 may be configured to receive the total resource usage value expressed as a single normalized unit. In other words, the analytics core 376 may be configured to generate a total resource usage value expressed as a single normalized unit.

[0070] The analytics core 376 and / or analytics metrics receiver 378 can be in communication with a usage analyzer 386. The usage analyzer 386 monitors usage associated with each account 350 and / or resource consuming user 352 to develop (or determine) typical usage, peak usage, and / or floor usage. In other words, the typical usage, peak usage, and / or floor usage can be based at least in part on the received metrics / attributes corresponding to each of the computing resources 108. In some cases, for example, the peak usage can represent a maximum usage over a period of time (e.g., an hour, a day, a week, a month, and / or any other suitable period), the floor usage can represent a minimum usage over a period of time (e.g., an hour, a day, a week, a month, and / or any other suitable period), and the typical usage can represent an average usage over a period of time (e.g., an hour, a day, a week, a month, and / or any other suitable period). In some cases, peak usage may approximate maximum usage over a period of time (e.g., usage values ​​within 5%, 10%, or 15% of actual maximum usage), and trough usage may approximate minimum usage over a period of time (e.g., usage values ​​within 5%, 10%, or 15% of actual minimum usage).

[0071] The typical usage, peak usage, and / or floor usage may correspond to an individual computing resource, a plurality of computing resources, and / or one or more sub-resources forming a computing resource (e.g., one or more of the graphics processor and graphics memory of GPU 210). For example, a typical usage, peak usage, and / or floor usage may be generated for each of a plurality of computing resources (e.g., each of one or more of CPU 202, memory 204, storage 206, network interface 208, GPU 210, and / or other resources 209). In these cases, the typical usage, peak usage, and / or floor usage of each resource may be in normalized units and / or in non-normalized units (e.g., units corresponding to the individual resource). As a further example, typical usage, peak usage, and / or floor usage may be generated corresponding to multiple computing resources (e.g., two or more of one or more of CPU 202, memory 204, storage 206, network interface 208, GPU 210, and / or other resources 209). In these cases, the typical usage, peak usage, and / or floor usage may be expressed as a single normalized unit.

[0072] In some cases, the usage analyzer 386 can be configured to generate an aggregate peak usage average. The aggregate peak usage average can generally be an average of two or more peak usage values ​​corresponding to two or more respective predetermined time periods. For example, peak usage can be determined for each day of a week and averaged to determine an aggregate peak usage average for the week. Additionally or alternatively, the usage analyzer 386 can be configured to generate an aggregate trough usage average. The aggregate trough usage average can generally be an average of two or more trough usage values ​​corresponding to two or more respective predetermined time periods. For example, trough usage can be determined for each day of a week and averaged to determine an aggregate trough usage average for the week.

[0073] The aggregate peak usage average and / or aggregate trough usage average may correspond to an individual computing resource and / or multiple computing resources. For example, an aggregate peak usage average and / or aggregate trough usage average may be generated for each of multiple computing resources (e.g., each of one or more of CPU 202, memory 204, storage 206, network interface 208, GPU 210, and / or other resources 209). In these cases, the aggregate peak usage average and / or aggregate trough usage average for each resource may be expressed in normalized units and / or non-normalized units (e.g., units corresponding to the individual resource). As a further example, an aggregate peak usage average and / or aggregate trough usage average may be generated for multiple computing resources (e.g., two or more of CPU 202, memory 204, storage 206, network interface 208, GPU 210, and / or other resources 209). In these cases, the aggregate peak usage average and / or aggregate trough usage average can be expressed as a single normalized unit.

[0074] A typical average usage may generally be an average resource usage calculated by taking multiple measurements of actual resource usage over a period of time during a repeating time interval and averaging the measurements. For example, actual resource usage may be measured at 11:00 AM every day for five days, and an average usage at 11:00 AM may be calculated based on these measurements. A peak average usage may generally be an average resource usage calculated by taking multiple measurements of actual resource usage over a period of time during a repeating time interval, selecting a portion of the measurements having the highest usage (e.g., the highest 5%, 10%, 15%, 20%, 25%, 35%, 40%, 50%, or 60% of resource usage), and calculating an average of the selected portion. For example, actual resource usage may be measured at 11:00 AM every day for five days, and an average peak usage at 11:00 AM may be calculated based on the three highest measurements. The rock-bottom average usage may generally be an average resource usage calculated by taking multiple measurements of actual resource usage over a period of time during a repeating time interval, selecting the portion of the measurements having the lowest usage (e.g., the lowest 5%, 10%, 15%, 20%, 25%, 35%, 40%, 50%, or 60% of resource usage), and calculating the average of the selected portion. For example, actual resource usage may be measured at 11:00 AM each day for five days, and the rock-bottom average usage at 11:00 AM may be calculated based on the three lowest measurements.

[0075] The typical average usage, peak average usage, and / or trough average usage may correspond to an individual computing resource and / or multiple computing resources. For example, a typical average usage, peak average usage, and / or trough average usage may be generated for each of multiple computing resources (e.g., each of one or more of CPU 202, memory 204, storage 206, network interface 208, GPU 210, and / or other resources 209). In these cases, the typical average usage, peak average usage, and / or trough average usage of each resource may be expressed in normalized units and / or non-normalized units (e.g., units corresponding to the individual resource). As a further example, a typical average usage, peak average usage, and / or trough average usage may be generated for multiple computing resources (e.g., two or more of CPU 202, memory 204, storage 206, network interface 208, GPU 210, and / or other resources 209). In these cases, the typical average usage, the peak average usage, and / or the trough average usage may be expressed as a single normalized unit.

[0076] The usage analyzer 386 can generate a typical average usage pattern 392, a peak average usage pattern 394, and a trough average usage pattern 395 based on the typical average usage, the peak average usage, and the trough average usage, respectively (shown in FIG. 3C for clarity). The typical average usage pattern 392 can be a data set including multiple typical average usage values ​​corresponding to different predetermined times (or predetermined periods). The peak average usage pattern 394 can be a data set including multiple peak average usage values ​​corresponding to different predetermined times (or predetermined periods). The trough average usage pattern 395 can be a data set including multiple trough average usage values ​​corresponding to different predetermined times (or predetermined periods).

[0077] A data set corresponding to the typical average usage pattern 392 may be compared to a data set corresponding to the peak average usage pattern 394 to determine a threshold usage value or range for one or more predetermined times (or predetermined time periods). For example, a threshold usage value for a predetermined time may be obtained by averaging the typical average usage value and the peak average usage value for each predetermined time period. As a further example, a threshold usage range may be expressed as a range spanning the typical average usage value to the peak average usage value for each predetermined time period.

[0078] Additionally or alternatively, a data set corresponding to trough average usage pattern 395 can be compared to a data set corresponding to peak average usage pattern 394 to determine a threshold usage value or range for one or more predetermined times (or predetermined time periods). For example, a threshold usage value for a predetermined time period can be obtained by averaging the trough average usage value and the peak average usage value for each predetermined time period. As a further example, a threshold usage range can be expressed as a range spanning from the trough average usage value to the peak average usage value for each predetermined time period.

[0079] Additionally or alternatively, a data set corresponding to the typical average usage pattern 392 can be compared to a data set corresponding to the trough average usage pattern 395 to determine a threshold usage value or range for one or more predetermined times (or predetermined time periods). For example, a threshold usage value for a predetermined time period can be obtained by averaging the typical average usage value and the trough average usage value for each predetermined time period. As a further example, a threshold usage range can be expressed as a range spanning from the trough average usage value to the typical average usage value for each predetermined time period.

[0080] Additionally or alternatively, the threshold usage value may be a trough average usage value, a peak average usage value, a trough usage value (i.e., the minimum usage over a predetermined time period) or an overall trough usage average value, and / or a peak usage value (i.e., the maximum usage over a predetermined time period) or an overall peak usage average value.

[0081] The notification generator 388 can be configured to generate a notification, for example, in response to one or more of resource usage meeting, exceeding, and / or not exceeding a threshold usage value at a predetermined time (or predetermined time interval) and / or resource usage outside a threshold usage range. In other words, the notification can be generated based at least in part on a comparison of resource usage to a threshold or threshold usage range. For example, detecting usage above a threshold usage value may indicate use of the resource by an unauthorized third party, while detecting usage not exceeding a rock-bottom average usage value may indicate a hardware failure (e.g., at least a portion of the computing resources 108 may be malfunctioning).

[0082] In some cases, the usage analyzer 386 can be configured to identify outliers (e.g., resource usage outside a threshold usage range). For example, the usage analyzer 386 can utilize machine learning to identify periods of high and low usage. During periods of high and / or low usage, the usage analyzer 386 can identify usage as an outlier if the usage does not exceed a peak average usage value and / or does not exceed a trough average usage value for at least a certain period of time (e.g., 10 minutes, 30 minutes, 60 minutes, 2 hours, and / or any other time period). As a result, the notification generator 388 can be configured to not generate a notification in response to detecting an outlier. Furthermore, once an outlier is identified, the usage analyzer 386 can exclude the outlier from the calculation of the normal average usage pattern 392, the peak average usage pattern 394, and / or the trough average usage pattern 395, for example, unless the outlier is detected multiple times. Thus, each of the typical average usage pattern 392, the peak average usage pattern 394, and / or the trough average usage pattern 395 may generally be generated using machine learning.

[0083] In some cases, the usage analyzer 386 may analyze the usage of other users of the meter 118 to determine whether other users are experiencing similar usage patterns. For example, if multiple users are experiencing resource usage that does not exceed the rock-bottom average usage value, this may indicate a hardware fault (e.g., at least a portion of the computing resource 108 may be malfunctioning). In this case, the notification generator 388 may generate a notification.

[0084] The generated notification may be in the form of an alert sent to one or more owners / users of account 350, for example, via a phone call, an SMS text message, an email, a fax, a tactile sensation (e.g., via a smart watch, fitness tracker, remote health monitoring device, and / or any other wearable device), an alert generated within an application or software to manage analytics platform 360 and / or metric / attribute collector 302 (e.g., an in-application message), and / or any other type of alert.

[0085] The usage analyzer 386 may determine, using, for example, machine learning, periods of high and low resource usage that occur periodically over a period of time. Once the periods of high and low resource usage are identified, the threshold usage for generating an alert may be adjusted, for example. For example, the threshold usage for generating an alert may be adjusted to be higher during periods known to be high resource usage than during periods known to be low resource usage.

[0086] The usage analyzer 386 can also be configured to generate a benchmark (e.g., an average cost index) that represents the cost per unit of consumed resources for one or more users and / or accounts. The usage analyzer 386 can receive total resource usage values ​​associated with one or more accounts 350, resource consuming users 352, and / or machine collections 356, expressed as a single normalized unit. In some cases, the usage analyzer 386 may receive total resource usage values ​​corresponding to a workload type (e.g., SPARK cluster workload, ELASTICSEARCH workload, data storage workload, disaster recovery workload, developer workload, quality assurance workload, manufacturing workload, and / or the like), a service provider (e.g., AMAZON, MICROSOFT, GOOGLE, and / or any other service provider), a hardware provider (e.g., DELL, SUPERMICRO, HP, and / or any other hardware provider), a hardware configuration, a hypervisor (e.g., VMWARE, CITRIX, OPENSTRACK, and / or any other hypervisor), and / or a group (e.g., a business unit within an organization, such as an engineering department, a sales department, or a marketing department).

[0087] The usage analyzer 386 may also receive cost data associated with the consumed resources. The cost data may be received from the computing system 104 (e.g., a service provider), from user input, and / or from any other source. The cost data may represent the cost of total resource usage associated with each account 350, resource consuming user 352, machine collection 356, workload type, and / or group. The cost data associated with the consumed resources may include one or more of colocation costs, hardware costs, networking costs, cloud hosting costs, labor costs, software license costs, compliance costs, managed services costs, consultant costs, support costs, developer / operations training costs, cybersecurity costs, migration opportunity costs, and / or any other associated costs for the consumed resources. The usage analyzer 386 may then divide the cost of the total resource usage by the total resource usage value to obtain a cost per unit consumption value. The usage analyzer 386 may use the cost per unit consumption to generate a benchmark (e.g., an average cost index).

[0088] The cost per unit consumption value may then be compared to, for example, a market benchmark (e.g., the average cost per unit across all users of one or more instruments 118), a group or industry benchmark (e.g., the average cost per unit across all users within a given industry or group), a workload benchmark (e.g., the average cost per unit across all users executing a given workload), a service provider benchmark (e.g., AMAZON, MICROSOFT, GOOGLE, and / or any other service provider), and / or the like. Group or industry benchmarks may be based, for example, on company size (e.g., market capitalization, revenue, and / or the like), business units or departments (e.g., sales department, engineering department, and / or the like) within a particular company or across multiple companies, and / or any other grouping.

[0089] In some cases, usage analyzer 386 may also be configured to generate market benchmarks, group or industry benchmarks, service provider benchmarks, and / or workload benchmarks. For example, usage analyzer 386 may be configured to access data related to resource usage and resource costs for one or more accounts 350 corresponding to one or more different users.

[0090] In some cases, the usage analyzer 386 can be separate from the analytics platform 360. For example, the usage analyzer 386 can be configured to communicate with multiple analytics platforms 360 and / or metrics / attribute collectors 302. In some cases, there may be a market usage analyzer remote from the analytics platform 360 that is configured to determine the average usage and average cost for a particular group or industry, a particular account, a particular workload, and the like across multiple different users. In these cases, there may also be a local usage analyzer configured to determine the average usage and average cost for a particular analytics platform 360.

[0091] In some cases, the usage analyzer 386 may also be configured to analyze the consumed computing resources 108 to determine whether the workload 110 executing on the computing resources 108 is using the computing resources 108 efficiently. For example, the usage analyzer 386 may analyze the specifications of the computing resources 108 (e.g., processor speed and / or size, memory speed and / or size, GPU speed and / or size, storage speed and / or size, and / or the like) and compare them to the requirements of the workload 110, thereby enabling the usage analyzer 386 to generate an optimized computing resource configuration (e.g., a best guess configuration). In this case, the usage analyzer 386 may be configured to generate a notification to be sent to a user, the notification may include a description of the optimized computing resource configuration. In some cases, the usage analyzer 386 may be configured to automatically migrate the workload 110 to the optimized computing resource configuration. As a further example, a computing resource 108 may include a dedicated graphics processing unit, but the applications 114 and / or operating system 112 running on the computing resource 108 may not require the dedicated graphics processing unit to operate efficiently. In this case, the usage analyzer 386 may be configured to generate a notification to be sent to a user. In some cases, the usage analyzer 386 may be configured to automatically move the workload 110 to another computing system 104 that does not include a dedicated graphics processing unit. As a result, the cost per unit of computing resource consumed may be reduced.

[0092] In some cases, the usage analyzer 386 may be further configured to generate computing resource configuration recommendations based on the workloads 110 to be executed. The computing resource configuration recommendations may be associated with a cost per unit of computing resource consumption values ​​and / or an estimated performance rating of the respective workloads 110. As a result, a user may compare resource costs and / or performance among multiple resource providers and / or multiple computer resource options available from a particular resource provider.

[0093] In some cases, the usage analyzer 386 can be configured to analyze a user's resource usage when the user pre-purchases a predetermined amount of resource. In these cases, the usage analyzer 386 can convert the pre-purchased amount of resource to a normalized value that represents the maximum amount of resource available and compare the normalized value of the pre-purchased resource to the peak usage value. In other words, the usage analyzer 386 can be configured to determine whether the supply of resource will meet the user's peak resource demand when the user pre-purchases a predetermined amount of resource. Accordingly, based on this comparison, the usage analyzer 386 can be configured to provide a recommendation regarding the amount of resource to pre-purchase.

[0094] The subscription manager 390 manages and maintains subscription information associated with users of the meters 118. For example, the subscriptions may determine how many resource consuming users 352 can be assigned to each account 350, determine how many accounts can be assigned to each user (e.g., business entity), determine how many machines 354 each user can access, determine how many machines 354 can be assigned to machine collections 356, determine how many machine collections 356 each user can have, associate login information with each user, determine the validity of the login information (e.g., whether the login information has expired), and / or any other tasks related to subscription management.

[0095] The exchange / contract manager 393 is configured to coordinate contracts between two parties (e.g., a buyer and a seller). The exchange / contract manager 393 can be configured to disable meters 118 according to the contract terms. When a meter 118 is disabled, a user's access to the computing resource 108 can be disabled. The exchange / contract manager 393 can also be configured to audit resource usage. Auditing resource usage can include verifying that resources shown as in use were actually used and / or that usage complied with the contract. The audit process can include comparing data collected by the metric / attribute collector 302 with data generated by a second collector (e.g., a backup collector).

[0096] The analytics automator 384 can automate one or more functions of the analytics platform 360. For example, the analytics automator 384 can be configured to receive a trigger event that causes the analytics platform 360 to perform a predetermined action. The predetermined action can include, for example, one or more of: enabling or disabling access to the computing resources 108; activating or deactivating at least a portion of the metric / attribute collector 302 (e.g., the metric receiver 334); activating or deactivating one or more plug-ins 358 (e.g., one or more plug-ins 358 configured to enable communication with the computing system 104); initiating database cleanup; and / or any other action. The trigger event can include, for example, receiving a new contract, receiving a new subscription, user input from the administrator user interface 372, a predetermined time (e.g., the start of a new contract / subscription and / or any other predetermined time), a recall request, and / or the like. In some cases, the trigger event can be the expiration of a period of time (e.g., the expiration of a contract term). For example, analytics platform 360 may perform a predetermined action in response to the expiration of a contract period.

[0097] As shown, analytical database manager 382 communicates with analytics core 376 and analytics synchronizer 380. Analytic database manager 382 manages data received and / or generated by analytics platform 360. For example, analytical database manager may manage and / or store one or more records (e.g., data) related to subscriptions, contracts, metrics / attributes, usage patterns (e.g., typical average usage pattern 392, peak average usage pattern 394, or trough average usage pattern 395), notification thresholds, trigger events, and / or any other data received by analytics platform 360. Analytic database manager 382 may also purge and / or truncate data. For example, the analytics database manager 382 may purge and / or truncate records related to subscriptions, contracts, metrics / attributes, usage patterns (e.g., typical average usage pattern 392, peak average usage pattern 394, or trough average usage pattern 395), notification thresholds, trigger events, and / or any other data received or collected by the analytics platform 360. Purge and / or pruning may occur, for example, after a period of time and / or after, for example, deactivation / termination of a subscription or contract.

[0098] The analytical database manager 382 can communicate with the analytical database 396 via a communication link 398 and can store one or more records on the analytical database 396. As shown, the analytical database 396 can be remote from the analytical platform 360 (e.g., running on a different server, in a different network, and / or on a different device). However, in some cases, the analytical database 396 can be local to the analytical platform 360 (e.g., running on the same server, on the same network, and / or on the same device). Data transmitted via the communication link 398 can be encrypted to prevent unauthorized access to the data during transmission. In some cases, records stored on the analytical database 396 can be encrypted to prevent unauthorized access to the records. For example, records stored on the analytical database 396 can be encrypted using transparent data encryption (TDE).

[0099] Analytics synchronizer 380 may communicate with analytics metric receiver 378 and / or analytics database manager 382 to replicate and / or back up data received by analytics platform 360 (e.g., data received by analytics database manager 382). For example, analytics synchronizer 380 may replicate and / or back up one or more of the records stored and / or to be stored on analytics database 396 to analytics backup 381. In some cases, records stored on analytics backup 381 may be encrypted to prevent unauthorized access. When changes are made to one or more of the records, analytics synchronizer 380 may back up only those changes to analytics backup 381. As shown, analytics synchronizer 380 may communicate with analytics backup 381 via communications link 383. Data transmitted via communications link 383 may be encrypted to prevent unauthorized access to the data in transit. As shown, analytics backup 381 may be remote from analytics platform 360 (e.g., running on a different server, in a different network, and / or on a different device). However, in some cases, analytics backup 381 may be local to analytics platform 360 (e.g., running on the same server, on the same network, and / or on the same device).

[0100] In some cases, a backup analytics platform 355 may be provided that performs substantially the same operations as analytics platform 360. As a result, an independent record may be maintained. The independent record may be used, for example, in audit processes and / or data recovery processes (e.g., if analytics platform 360 experiences a failure). Backup analytics platform 355 may be communicatively coupled to metrics / attribute collector 302 using communications link 353.

[0101] In some cases, analytics backup 381 may also include a backup analytics platform that operates substantially the same as analytics platform 360. In other words, analytics backup 381 may include a standalone analytics platform. As a result, an independent record may be maintained. The independent record may be used, for example, in audit processes and / or data recovery processes (e.g., if analytics platform 360 experiences a failure).

[0102] As shown, usage predictor 385 can be communicatively coupled to analytics core 376 via communications link 387. Usage predictor 385 can estimate future usage for each user (e.g., owning or controlling one or more accounts 350), each account 350, and / or each resource-consuming user 352. Usage predictor 385 can predict future usage using, for example, typical average usage patterns 392, peak average usage patterns 394, and / or trough average usage patterns 395. Additionally or alternatively, usage predictor 385 can utilize machine learning to analyze historical resource usage to predict future usage. For example, usage predictor 385 can analyze usage for patterns and / or spikes (e.g., increases in resource usage). Once future usage is predicted, usage predictor 385 can also estimate a predicted economic cost corresponding to the predicted usage.

[0103] As shown, analytics user interface 389 can communicate with analytics core 376 via communications link 391. Analytics user interface 389 can be configured to generate displays presenting information related to analytics platform 360 and / or metrics / attributes collector 302. For example, analytics user interface 389 can be configured to display usage forecasts, typical average usage patterns 392 (e.g., as a graph or plot), peak average usage patterns 394 (e.g., as a graph or plot), trough average usage patterns 395 (e.g., as a graph or plot), current contract period, current subscriptions, active and / or inactive notifications / alerts, and / or any other data related to analytics platform 360. In some cases, analytics user interface 389 can be configured to allow a user to modify various functions of analytics platform 360. For example, analytics user interface 389 may be configured to allow a user to modify and / or cancel one or more subscriptions, modify and / or cancel one or more contracts, modify, add, and / or remove notification thresholds and / or triggers for analytics automator 384, and / or modify any other functionality of analytics platform 360.

[0104] As shown, the analytics user interface 389 can be remote from the analytics platform 360 (e.g., operating on a different server, in a different network, and / or on a different device). However, in some cases, the analytics user interface 389 can be local to the analytics platform 360 (e.g., operating on the same server, on the same network, and / or on the same device). Data transmitted over the communications link 391 can be encrypted to prevent unauthorized access to the data.

[0105] For example, in some cases, the analytics user interface 389 may be generated and displayed on a web server. As a further example, in some cases, the analytics user interface 389 may be an application running on a computer that communicates with the analytics platform 360 over a network connection (e.g., the Internet). As a further example, in some cases, the analytics user interface 389 may be an application running locally (e.g., on the same hardware as the analytics platform 360).

[0106] Communications links 330, 353, 362, 366, 370, 374, 383, 387, 391, 398, and / or 399 may utilize the Secure Sockets Layer (SSL) security protocol when establishing communications. As discussed herein, communications links 330, 353, 362, 366, 370, 374, 383, 387, 391, 398, and / or 399 may each carry encrypted data. Thus, one or more of communication links 330, 353, 362, 366, 370, 374, 383, 387, 391, 398, and / or 399 may generally be secure communication links (e.g., communication links 330, 353, 362, 366, 370, 374, 383, 387, 391, 398, and / or 399 use at least one of the SSL security protocols and / or transmitted data is encrypted).

[0107] Metric / attribute collector 302 and analytics platform 360 can be implemented in software, firmware, hardware, and / or combinations thereof. For example, metric / attribute collector 302 and analytics platform 360 can be implemented as software stored on one or more memories (e.g., any type of tangible, non-transitory storage medium, which may include any one or more of: magnetic recording media (e.g., hard disk drives), optical disks, semiconductor devices such as read-only memory (ROM), random access memory (RAM) such as dynamic and static RAM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic or optical cards, or any type of storage medium for storing electronic instructions) and configured to be executed by one or more processors (e.g., commercially available processors from INTEL, ADVANCED MICRO DEVICES, IBM, ARM, ORACLE, and / or any other processor). As a further example, metric / attribute collector 302 and analytics platform 360 may be implemented as a circuit (e.g., an application specific integrated circuit). Metric / attribute collector 302 and analytics platform 360 may be implemented on the same or different machines, servers, and / or networks.

[0108] FIG. 4 illustrates a schematic example of a transform unit generator 400 configured to generate the transform units discussed in connection with FIG. 3A . The transform unit generator 400 may be separate from the computing system 104, the device 102, and the network 106. For example, the transform unit generator 400 may be part of a third-party system such that the transform units can be generated and input as fixed values ​​into the analysis platform 312. As illustrated, the transform unit generator 400 may include at least a first resource transform unit generator 402 and a second resource transform unit generator 404. The first resource transform unit generator 402 may, for example, generate one or more of the CPU transform unit 314, the memory transform unit 316, the storage transform unit 318, the network transform unit 320, and / or the disk I / O transform unit 328. The second resource transform unit generator 404 may, for example, generate one or more of the GPU processor transform unit 322, the GPU memory transform unit 324, and / or the GPU memory speed transform unit 326.

[0109] As shown, the first resource conversion unit generator 402 includes a CPU conversion unit generator 406 , a memory conversion unit generator 408 , a storage conversion unit generator 410 , a network conversion unit generator 412 , and a disk IO conversion unit generator 414 .

[0110] The CPU transform unit generator 406 may generate the CPU transform unit 314 based at least in part on the CPU allocation 416 and the CPU computation portion 418. For example, the CPU transform unit 314 may be generated by dividing the CPU allocation 416 by the CPU computation portion 418.

[0111] CPU allocation 416 corresponds to the clock speed of CPU 202 accessible to a user. For example, only a portion of a physical processor may be allocated to one or more user workloads. In other words, the allocated available clock speed may be less than the default clock speed of the physical processor. CPU computation portion 418 may generally correspond to a weight representing the CPU resources used by one or more of workloads 110. In some cases, CPU computation portion 418 may be an empirically derived value based on analysis of multiple user workloads. Thus, in some cases, CPU computation portion 418 may generally be a dynamic weight that can be adjusted to account for changes in workload over time.

[0112] The memory translation unit generator 408 may generate the memory translation unit 316 based at least in part on the memory allocation 420 and the memory calculation portion 422. For example, the memory translation unit 316 may be generated by dividing the memory allocation 420 by the memory calculation portion 422.

[0113] The memory allocation 420 corresponds to the amount of memory 204 accessible to a user. For example, only a portion of the physical memory may be allocated to one or more user workloads. In other words, the allocated memory may be less than the total available physical memory. The memory calculation portion 422 may generally correspond to a weight that represents the memory resources used by one or more of the workloads 110. In some cases, the memory calculation portion 422 may be an empirically derived value based on an analysis of multiple user workloads. Thus, in some cases, the memory calculation portion 422 may generally be a dynamic weight that can be adjusted to account for changes in workload over time.

[0114] The storage conversion unit generator 410 may generate the storage conversion unit 318 based at least in part on the storage allocation 424 and the storage computation portion 426. For example, the storage conversion unit 318 may be generated by dividing the storage allocation 424 by the storage computation portion 426.

[0115] The storage allocation 424 corresponds to the amount of storage 206 accessible to a user. For example, only a portion of the physical storage may be allocated to one or more user workloads. In other words, the allocated storage may be less than the total physical storage available. The storage computation portion 426 may generally correspond to a weight that represents the storage resources used by one or more of the workloads 110. In some cases, the storage computation portion 426 may be an empirically derived value based on an analysis of multiple user workloads. Thus, in some cases, the storage computation portion 426 may generally be a dynamic weight that can be adjusted to account for changes in workload over time.

[0116] The network transformation unit generator 412 may generate the network transformation unit 320 based at least in part on the network allocation 428 and the network calculation portion 430. For example, the network transformation unit 320 may be generated by dividing the network allocation 428 by the network calculation portion 430.

[0117] Network allocation 428 corresponds to the network bandwidth accessible to a user. For example, only a portion of the total physical network bandwidth may be allocated to one or more user workloads. In other words, the allocated network bandwidth may be less than the available network bandwidth. Network computation portion 430 may generally correspond to weights that represent the network resources (e.g., bandwidth) used by one or more of workloads 110. In some cases, network computation portion 430 may be empirically derived values ​​based on analysis of multiple user workloads. Thus, in some cases, network computation portion 430 may generally be dynamic weights that can be adjusted to account for changes in workload over time.

[0118] Disk IO translation unit generator 414 may generate disk IO translation units 328 based at least in part on disk IO allocation 432 and disk IO calculation portion 434. For example, disk IO translation units 328 may be generated by dividing disk IO allocation 432 by disk IO calculation portion 434.

[0119] The disk IO allocation 432 corresponds to the disk bandwidth accessible to a user (e.g., reads and writes to storage 206). For example, only a portion of the physical disk bandwidth may be allocated to one or more user workloads. In other words, the allocated disk bandwidth may be less than the total available physical disk bandwidth. The disk IO computation portion 434 may generally correspond to a weight that represents the disk IO resources used by one or more of the workloads 110. In some cases, the disk computation portion 434 may be an empirically derived value based on an analysis of multiple user workloads. Thus, in some cases, the disk computation portion 434 may generally be a dynamic weight that can be adjusted to account for changes in workload over time.

[0120] The sum of the CPU computation portion 418, the memory computation portion 422, the storage computation portion 426, the network computation portion 430, and the disk IO computation portion 434 may be 100. In other words, each computation portion may be expressed as a percentage of the total computing resource assessed by the first resource conversion unit generator 402 by dividing the computation portion by 100.

[0121] As shown, the second resource conversion unit generator 404 includes a GPU processor conversion unit generator 436 , a GPU memory conversion unit generator 438 , and a GPU memory speed conversion unit generator 440 .

[0122] The GPU processor transform unit generator 436 may generate the GPU processor transform unit 322 based at least in part on the GPU processor allocation 442 and the GPU processor computation portion 444. For example, the GPU processor transform unit 322 may be generated by dividing the GPU processor allocation 442 by the GPU processor computation portion 444.

[0123] GPU processor allocation 442 corresponds to the clock speed of GPU 210 accessible to a user. For example, only a portion of a physical GPU processor may be allocated to one or more user workloads. In other words, the available clock speed of the allocated GPU processor may be less than the default clock speed of the physical GPU processor. GPU processor computation portion 444 may generally correspond to a weight that represents the GPU processor resources used by one or more of workloads 110. In some cases, GPU processor computation portion 444 may be an empirically derived value based on an analysis of multiple user workloads. Thus, in some cases, GPU processor computation portion 444 may generally be a dynamic weight that can be adjusted to account for changes in workload over time.

[0124] The GPU memory conversion unit generator 438 may generate the GPU memory conversion unit 324 based at least in part on the GPU memory allocation 446 and the GPU memory computation portion 448. For example, the GPU memory conversion unit 324 may be generated by dividing the GPU memory allocation 446 by the GPU memory computation portion 448.

[0125] GPU memory allocation 446 corresponds to the amount of GPU 210 memory accessible to a user. For example, only a portion of GPU 210's physical memory may be allocated to one or more user's workloads. In other words, the allocated GPU 210 memory may be less than the total GPU 210's physical memory available. GPU memory calculation portion 448 may generally correspond to a weight that represents the memory resources used by one or more of workloads 110. In some cases, GPU memory calculation portion 448 may be an empirically derived value based on an analysis of multiple user's workloads. Thus, in some cases, GPU memory calculation portion 448 may generally be a dynamic weight that can be adjusted to account for changes in workload over time.

[0126] The GPU memory speed conversion unit generator 440 may generate the GPU memory speed conversion unit 326 based at least in part on the GPU memory speed allocation 450 and the GPU memory speed calculation portion 452. For example, the GPU memory speed conversion unit 326 may be generated by dividing the GPU memory speed allocation 450 by the GPU memory speed calculation portion 452.

[0127] GPU memory speed allocation 450 corresponds to the clock speed of GPU memory accessible to a user. For example, only a portion of the clock speed of the physical GPU memory may be allocated to one or more user workloads. In other words, the available clock speed of the allocated GPU memory may be less than the default clock speed of the physical GPU memory. GPU memory speed calculation portion 452 may generally correspond to a weight representing the GPU memory speed resource used by one or more of workloads 110. In some cases, GPU memory calculation portion 452 may be an empirically derived value based on an analysis of multiple user workloads. Thus, in some cases, GPU memory speed calculation portion 452 may generally be a dynamic weight that can be adjusted to account for changes in workload over time.

[0128] The sum of the GPU processor computation portion 444, the GPU memory computation portion 448, and the GPU memory speed computation portion 452 may be 100. In other words, each computation portion may be expressed as a percentage of the total computing resource assessed by the second resource conversion unit generator 404 by dividing the computation portion by 100.

[0129] As mentioned above, the physical computing resources 108 may additionally or alternatively include other resources 209. As discussed, the other resources 209 may include any one or more of a field programmable gate array (FPGA), a tensor processing unit (TPU), an intelligence processing unit (IPU), a neural processing unit (NPU), a vision processing unit (VPU), a digital signal processor (DSP), an accelerated processing unit (APU), an application-specific integrated circuit (ASIC), a system-on-chip (SoC), a programmable SoC, an application-specific standard product (ASSP), an adaptive computing acceleration platform (ACAP), a microcontroller, and / or any other computing resource. In these cases, the resource conversion units may each be determined using an independent resource conversion unit generator corresponding to each of the other computing resources 209. For example, the conversion unit generator 400 may include an other resource conversion unit generator 454. The other resource transform unit generator 454 may be configured to generate one or more transform units corresponding to each of the other resources 209 .

[0130] Figure 5 illustrates an example flowchart of a method 500 for quantifying usage of disparate computing resources as a single unit of measure, which may be used, for example, in the system of Figure 1. Method 500 may be embodied as one or more instructions in one or more non-transitory computer-readable media (e.g., storage 206) configured for execution by one or more processors (e.g., CPU 202).

[0131] As shown, method 500 may include step 502. Step 502 may include causing a workload (e.g., workload 110) to execute on a computing system (e.g., computing system 104). In some cases, the workload may be executed in response to receiving a request from device 102, for example. Additionally or alternatively, the workload may include one or more idle processes resulting from operation of the computing system.

[0132] Method 500 may also include step 504. Step 504 may include measuring the amount of physical resources used (e.g., consumed) by the execution of the workload over a period of time (e.g., 1 second, 10 seconds, 1 minute, 1 hour, 1 day, 1 week, 1 month, 1 year, and / or any other suitable period of time). The amount of physical resources (e.g., as opposed to virtual resources) may be measured to avoid or otherwise mitigate inaccurate readings used by ballooning techniques available to hypervisors that measure virtual resource usage. Furthermore, by directly measuring physical resources, the method may be performed on multiple forms of hardware, including hardware not running virtual machines. Thus, the method may be implemented in systems that utilize packaged software (e.g., containers).

[0133] Method 500 may include step 506. Step 506 may include normalizing measurements of the physical resources used. Normalizing the measurements may include converting each of the disparate resources (e.g., CPU resources, memory resources, storage resources, network interface resources, GPU resources, and / or any other computing resources) into a common unit (e.g., as described in connection with FIGS. 3 and 4 ).

[0134] Method 500 may include step 508. Step 508 may include summing the normalized measurements. Method 500 may also include step 510. Step 510 may include generating a single value that represents the total normalized resources used. For example, the generated single value may be the summation result. In other cases, the generated value may represent an average of the normalized resources summed over a time window (e.g., a weekly average where the individual normalized values ​​summed correspond to days of the week).

[0135] Figure 6 illustrates an example flowchart of a method 600 for quantifying maximum or minimum values ​​of heterogeneous computing resources as a single unit of measure, such as may be used in the system of Figure 1. The method may be embodied as one or more instructions in one or more non-transitory computer-readable media (e.g., storage 206) configured for execution by one or more processors (e.g., CPU 202).

[0136] The method may include step 602. Step 602 may include measuring, over a period of time, a maximum amount of physical resources available and / or measuring, over a period of time, a minimum amount of physical resources for running an idle process (e.g., idle process 113).

[0137] The method may include step 604. Step 604 may include normalizing measurements of available heterogeneous physical resources (e.g., used and inactive resources). Normalizing the measurements may include converting each of the heterogeneous resources (e.g., CPU resources, memory resources, storage resources, network interface resources, GPU resources, and / or any other computing resources) into a common unit (e.g., as described in connection with FIGS. 3A-C and 4).

[0138] The method may also include step 606. Step 606 may include summing the normalized measurements of the available physical resources. The method may include step 608. Step 608 may include generating a single value representing the total normalized available resources. For example, the generated single value may be the summation result. As a further example, the generated single value may be an adjustment value based on the summation result.

[0139] In some cases, the maximum and / or minimum amount of physical computing resources available over a period of time can be calculated based at least in part on specifications corresponding to individual physical computing resources. This allows a user to estimate the maximum and / or minimum amount of physical computing resources available without having access to the physical computing resources. In some cases, the calculated values ​​can be compared to measurements from method 600. For example, the calculated values ​​can be compared to measurements to adjust the specifications and / or calculation method corresponding to individual computing resources to improve the estimation of the maximum and / or minimum amount of physical computing resources available.

[0140] According to one aspect of the present disclosure, there is provided a computing network. The computing network may include physical computing resources configured to execute at least one workload. The computing network may also include a meter configured to measure an amount of the physical computing resources used over a period of time. The meter may generate a single usage value representative of the amount of the physical resources used over the period of time.

[0141] According to another aspect of the present disclosure, at least one computer-readable storage medium having stored thereon instructions that, when executed by one or more processors, cause the one or more processors to perform one or more operations. The one or more operations may include measuring amounts of one or more physical computing resources used over a period of time. The one or more operations may also include normalizing each measured amount of each physical computing resource used. The one or more operations may further include summing the normalized measured amounts of the physical computing resources used. The one or more operations may also include generating a single usage value representative of the physical computing resources used over the period of time based on the sum of the normalized physical computing resources.

[0142] According to yet another aspect of the present disclosure, there is provided a method. The method may include measuring an amount of one or more physical computing resources used over a period of time. The method may also include normalizing each measurement of each physical computing resource used. The method may further include summing the normalized measurements of the physical computing resources used. The method may also include generating a single usage value representative of the physical computing resources used over the period of time based on the sum of the normalized physical computing resources.

[0143] According to yet another aspect of the present disclosure, a computing network is provided. The computing network may include a computing system configured to communicatively couple to a device. The computing system may be configured to execute at least one workload. The computing system may include physical computing resources configured to execute the at least one workload. The computing system may also include a meter configured to measure an amount of the physical computing resource used over a period of time. The meter may generate a single usage value representative of the amount of the physical resource used over the period of time.

[0144] According to yet another aspect of the present disclosure, there is provided a system for metering heterogeneous computer resources. The metering instrument may include at least one collector. The collector may include at least one metric receiver configured to receive one or more metrics from at least one computing system over a period of time, the metrics corresponding to physical resource usage. The metering instrument may also include at least one analytics platform configured to normalize each of the received metrics and sum the normalized metrics to generate a single usage value representing physical resource usage over the period of time.

[0145] According to yet another aspect of the present disclosure, a computing network is provided. The computing network can include a computing system having a plurality of physical computing resources configured to execute one or more workloads and an instrumentation communicatively coupled to the computing system. The instrumentation can be configured to measure an amount of the physical computing resources used over a period of time. The instrumentation can include at least one collector including at least one metric receiver configured to receive one or more metrics from the computing system, the metrics corresponding to the amount of physical resources used. The instrumentation can also include at least one analytics platform in communication with the collector and configured to normalize each of the received metrics and sum the normalized metrics to generate a single usage value representing the physical resource usage over the period of time.

[0146] According to yet another aspect of the present disclosure, there is provided a computing network. The computing network may include a computing system configured to communicatively couple to a device. The computing system may be configured to execute at least one workload. The computing system may include physical computing resources configured to execute the at least one workload. The computing system may be configured to communicate with a meter. The meter may be configured to measure an amount of the physical computing resource used, the meter being further configured to generate a single usage value representative of the amount of physical resource used.

[0147] According to yet another aspect of the present disclosure, a computing network is provided. The computing network may include computing systems having a plurality of physical computing resources configured to execute one or more workloads. The computing network may also include at least one collector having at least one metric receiver configured to receive one or more metrics from the computing systems, the one or more metrics corresponding to usage of the physical computing resources. The computing network may further include at least one backup collector having a backup receiver configured to receive one or more metrics from the computing systems. The computing network may also include at least one analytics platform in communication with at least one of the metric collectors or the backup collectors, the analytics platform being configured to normalize each of the received metrics and sum the normalized metrics to generate a single usage value.

[0148] According to yet another aspect of the present disclosure, there is provided a system for measuring heterogeneous computer resources. The system may include at least one collector including at least one metric receiver and an account manager. The metric receiver may be configured to receive one or more metrics from at least one computing system, the metrics corresponding to physical resource usage. The account manager may be configured to manage a plurality of accounts. Each of the plurality of accounts may be associated with at least one resource-consuming user, and physical resource usage of each resource-consuming user is associated with a respective account. The system may also include a usage analyzer configured to generate a benchmark for the one or more accounts representing a cost per unit of physical resource usage.

[0149] According to yet another aspect of the present disclosure, a computing network is provided. The computing network may include a computing system configured to communicatively couple to a device. The computing system may also be configured to execute at least one workload. The computing system may include physical computing resources configured to execute the at least one workload. The computing network may also include a meter configured to measure an amount of the physical computing resource used over a period of time to generate a single usage value representative of the amount of the physical resource used over the period of time. The meter may be further configured to selectively disable access by the device to at least a portion of the computing system.

[0150] According to yet another aspect of the present disclosure, there is provided a method that may include measuring amounts of a plurality of physical computing resources used over a period of time. The method may also include normalizing each measurement of each physical computing resource used. The method may also include summing the normalized measurements of the physical computing resources used to generate a single usage value that represents the physical computing resources used over the period of time.

[0151] In some cases, the plurality of physical computing resources may include a plurality of heterogeneous physical computing resources. In some cases, the plurality of physical computing resources may include one or more of a graphics processing system, a field programmable gate array, an application specific integrated circuit, a system on a chip, a digital signal processor, a microcontroller, or an adaptive computing acceleration platform. In some cases, the plurality of physical computing resources may include a graphics processing system, which may include a graphics processor and a graphics memory. In some cases, the method may further include measuring a maximum amount of the plurality of physical computing resources available over a period of time, normalizing each measured maximum amount of each available physical computing resource, and summing the normalized maximum measured amounts of the available physical computing resources to generate a single available resource value representing the maximum amount of the physical computing resource available over the period of time.

[0152] According to yet another aspect of the present disclosure, a computing network is provided. The computing network may include one or more computing systems having a plurality of physical computing resources configured to execute one or more workloads. The computing network may also include one or more metric collectors, each having at least one metric receiver configured to receive a plurality of metrics from the one or more computing systems. The plurality of metrics may correspond to usage of the physical computing resources. The computing network may also include one or more analytics platforms configured to normalize each of the received metrics.

[0153] In some cases, the one or more analytics platforms may be further configured to sum the normalized metrics to generate a single usage value. In some cases, at least a portion of the physical computing resource usage may be associated with one or more accounts. In some cases, the one or more accounts may include multiple accounts, where at least one account may be associated with a portion of the physical computing resource usage and at least one other account may be associated with a different portion of the physical computing resource usage. In some cases, at least one of the metric collectors may further include a plugin database having one or more plugins, where the one or more plugins may be configured to enable at least one of the metric collectors to communicate with one or more computing systems. In some cases, the analytics platform may be configured to generate a notification based at least in part on a comparison of the physical computing resource usage to a threshold value. In some cases, the one or more analytics platforms may include multiple analytics platforms, where at least one of the analytics platforms may be a backup analytics platform. In some cases, the one or more analytics platforms may be configured to activate or deactivate at least one of the metric collectors at a predetermined time. In some cases, at least one of the metric collectors can include at least one of the one or more analytics platforms. In some cases, the one or more metric collectors can include multiple metric collectors, and at least one of the multiple metric collectors can be a backup metric collector. In some cases, the analytics platform can be further configured to determine, based at least in part on the received metrics, one or more of a peak usage (e.g., maximum usage), a trough usage (e.g., minimum usage), or an average usage corresponding to each physical computing resource over a period of time.

[0154] According to yet another aspect of the present disclosure, at least one computer-readable storage medium having stored thereon one or more instructions that, when executed by one or more processors, cause one or more operations to be performed. The operations may include measuring amounts of a plurality of physical computing resources used over a period of time. The operations may also include normalizing each measurement of each physical computing resource used. The operations may also include summing the normalized measurements of the physical computing resources used to generate a single usage value representative of the physical computing resources used over the period of time.

[0155] In some cases, the plurality of physical computing resources may include a plurality of heterogeneous physical computing resources. In some cases, the plurality of physical computing resources may include one or more of a graphics processing system, a field programmable gate array, an application-specific integrated circuit, a system-on-chip, a digital signal processor, a microcontroller, or an adaptive computing acceleration platform. In some cases, the plurality of physical computing resources may include a graphics processing system, which may include a graphics processor and a graphics memory. In some cases, the operations may further include measuring a maximum amount of the plurality of physical computing resources available over a period of time, normalizing each measured maximum amount of each available physical computing resource, and summing the normalized maximum measured amounts of each available physical computing resource to generate a single available resource value representing the maximum amount of the physical computing resource available over the period of time. In some cases, the operations may also include accessing a plug-in database, the plug-in database including one or more plug-ins configured to enable measurement of the plurality of physical computing resources. In some cases, the operations may further include determining one or more of a peak usage (e.g., maximum usage), a trough usage (e.g., minimum usage), or an average usage corresponding to each physical computing resource over a period of time.

[0156] According to yet another aspect of the present disclosure, there is provided a method. The method may include measuring an amount of graphics processing system resources used over a period of time, where the graphics processing system resources may include a graphics processor and a graphics memory. The method may also include normalizing the measurements corresponding to the graphics processor and the graphics memory. The method may also include summing the normalized measurements to generate a single usage value representative of the graphics processing system resources used over the period of time.

[0157] While several embodiments of the present disclosure have been described and illustrated herein, those skilled in the art will readily conceive of various other means and / or structures for performing the functions and / or obtaining one or more of the results and / or advantages described herein, and each of such variations and / or modifications is deemed to be within the scope of the present disclosure. More generally, those skilled in the art will readily understand that all parameters, dimensions, materials, and configurations described herein are meant to be exemplary, and that the actual parameters, dimensions, materials, and / or configurations will depend on the particular application or applications in which the teachings of the present disclosure are used.

[0158] Those skilled in the art will understand or be able to ascertain, using no more than routine experimentation, many equivalents to the specific embodiments described herein. Accordingly, it is to be understood that the foregoing embodiments are presented by way of example only, and that, within the scope of the appended claims and their equivalents, the disclosure may be practiced otherwise than as specifically described and claimed. The present disclosure is directed to each individual feature, system, article, material, kit, and / or method described herein. In addition, any combination of two or more of such features, systems, articles, materials, kits, and / or methods, where such features, systems, articles, materials, kits, and / or methods are not mutually inconsistent, is also within the scope of the present disclosure.

[0159] The indefinite articles "a" and "an," as used in the specification and claims, unless expressly indicated otherwise, should be understood to mean "at least one."

[0160] As used herein, the terms "couple" and "coupled" include both direct and indirect coupling, unless clearly indicated otherwise.

[0161] The term "and / or," as used in this specification and in the claims, should be understood to mean "either or both" of the elements so conjoined; that is, the elements should be understood to be present contiguously in some cases and separately in other cases. Other elements, whether related or unrelated to those elements specifically identified, other than the elements specifically identified by the "and / or" clause may optionally be present, unless clearly indicated otherwise. [Explanation of symbols]

[0162] 100 Computing Networks 102 devices 104 Computing Systems 106 Network 108 Computing Resources 110 workloads 112 Operating Systems 113 Idle Processes 114 Applications 116 Display 118 Measuring Instruments

Claims

1. measuring the amount of a plurality of physical computing resources used over a period of time; normalizing each measurement of each physical computing resource used; summing the normalized measures of the physical computing resource used to generate a single usage value representative of the physical computing resource used over the period of time; A method comprising:

2. The method of claim 1 , wherein the plurality of physical computing resources comprises a plurality of heterogeneous physical computing resources.

3. the plurality of physical computing resources include one or more of a graphics processing system, a field programmable gate array, an application specific integrated circuit, a system on a chip, a digital signal processor, a microcontroller, or an adaptive computing acceleration platform; The method of claim 1.

4. the plurality of physical computing resources includes the graphics processing system, the graphics processing system including a graphics processor and a graphics memory; The method of claim 3.

5. measuring a maximum amount of a plurality of physical computing resources available over said time period; normalizing each measured maximum amount of each available physical computing resource; summing the normalized maximum measures of the available physical computing resource to generate a single available resource value representing the maximum amount of physical computing resource available over the period of time; Further comprising: The method of claim 1.

6. one or more computing systems having a plurality of physical computing resources configured to execute one or more workloads; one or more metric collectors, each having at least one metric receiver configured to receive a plurality of metrics from the one or more computing systems, the plurality of metrics corresponding to usage of the physical computing resources; one or more analytics platforms configured to normalize each of the received metrics; A computing network comprising:

7. the one or more analytics platforms are further configured to sum the normalized metrics to generate a single usage value. The computing network of claim 6.

8. at least a portion of the physical computing resource usage is associated with one or more accounts; The computing network of claim 6.

9. the one or more accounts include a plurality of accounts, at least one account associated with a portion of the physical computing resource usage and at least one other account associated with a different portion of the physical computing resource usage; The computing network of claim 8.

10. At least one of the metric collectors further includes a plug-in database having one or more plug-ins configured to enable at least one of the metric collectors to communicate with one or more computing systems. The computing network of claim 6.

11. the analytics platform is configured to generate a notification based at least in part on a comparison of the physical computing resource usage to a threshold value. The computing network of claim 6.

12. the one or more analytics platforms include a plurality of analytics platforms, and at least one of the analytics platforms is a backup analytics platform; The computing network of claim 6.

13. the one or more analytics platforms are configured to activate or deactivate at least one of the metric collectors at a given time; The computing network of claim 6.

14. at least one of the metric collectors includes at least one of the one or more analytics platforms; The computing network of claim 6.

15. the one or more metric collectors include a plurality of metric collectors, at least one of the plurality of metric collectors being a backup metric collector; The computing network of claim 6.

16. the analytics platform is further configured to determine, based at least in part on the received metrics, one or more of a peak usage, a trough usage, or an average usage corresponding to each physical computing resource over the time period. The computing network of claim 6.

17. At least one computer-readable storage medium having stored thereon one or more instructions that, when executed by one or more processors, cause the processor to perform one or more operations, the operations including: measuring the amount of a plurality of physical computing resources used over a period of time; normalizing each measurement of each physical computing resource used; summing the normalized measures of the physical computing resource used to generate a single usage value representative of the physical computing resource used over the period of time; Including, 1. A computer-readable storage medium comprising:

18. the plurality of physical computing resources includes a plurality of heterogeneous physical computing resources; 20. The computer-readable storage medium of claim 17.

19. the plurality of physical computing resources include one or more of a graphics processing system, a field programmable gate array, an application specific integrated circuit, a system on a chip, a digital signal processor, a microcontroller, or an adaptive computing acceleration platform; 20. The computer-readable storage medium of claim 17.

20. the plurality of physical computing resources includes the graphics processing system, the graphics processing system including a graphics processor and a graphics memory; 20. The computer-readable storage medium of claim 19.

21. The operation further comprises: measuring a maximum amount of a plurality of physical computing resources available over said time period; normalizing each of the measured maximum amounts of each of the available physical computing resources; summing the normalized maximum measures of the available physical computing resource to generate a single available resource value representing the maximum amount of physical computing resource available over the period of time; 20. The computer-readable storage medium of claim 17, comprising:

22. The operations further include accessing a plug-in database, the plug-in database including one or more plug-ins configured to enable the measurement of the plurality of physical computing resources of one or more computing systems.

20. The computer-readable storage medium of claim 17.

23. the operations further include determining one or more of a peak usage, a trough usage, or an average usage corresponding to each physical computing resource over the time period; 20. The computer-readable storage medium of claim 17.

Images