A method for quantifying the usage amount of heterogeneous computing resources as a single measurement unit

Abstract

To provide a method for quantifying resource usage.SOLUTION: A method for quantifying resource usage may include a step of measuring the amount of a plurality of physical computing resources used for a certain period of time. The method may also include a step of normalizing each measurement amount of each physical computing resource used. The method may also include a step of summing the normalized measurement amount of the physical computing resource used so as to produce a single usage amount representing the physical computing resource used over the period of time.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] This application claims the benefit of U.S. Provisional Patent Application No. 62 / 647,335, filed on Mar. 23, 2018, entitled "Quantifying Usage of Disparate Computing Resources as a Single Unit of Measure", and U.S. Provisional Patent Application No. 62 / 690,533, filed on Jun. 27, 2018, entitled "Quantifying Usage of Disparate Computing Resources as a Single Unit of Measure", the entire contents of both of which are incorporated herein by reference. The present disclosure generally relates to computing, and more specifically, to measuring the usage of disparate computer resources and quantifying the usage as a single unit of measure.

Background Art

[0002] With cloud computing, a user (e.g., an individual or an organization) can lease computing resources maintained and / or managed by a third party. Thereby, the user does not need to make a huge investment in physical hardware, real estate of a data center, electricity, personnel of a data center, and / or software licenses, and can lease computing resources suitable for the individual computing needs of the user.

[0003] Examples of cloud computing systems can include physical resources (e.g., processors and memory), and / or virtual resources (e.g., virtual processors and virtual memory). A user accessing a cloud computing system can lease an amount of resources on which to execute their workload. The amount of computing resources leased can be estimated based on a theoretical maximum workload. The theoretical maximum workload assumes using nearly all (e.g., approximately 100%) of the available resources. In other words, the user will lease the maximum amount of resources required to execute their workload to avoid system performance issues due to a sudden increase in resource requirements. However, sudden increases in usage are rare, and the average required resources can potentially be less than the theoretical maximum, which can lead to waste (e.g., leasing unused computing resources).

[0004] A cloud computing provider can implement a pricing model based on the amount of resources leased (over a given period), as opposed to the actual amount of resources used (or consumed) by a user. These resources can be packaged as servers with preconfigured central processing units (CPUs), memory (e.g., random access memory), storage, graphics processing units (GPUs), dedicated network functions (e.g., for providing a given network speed), field programmable gate arrays (FPGAs), and / or any other computing resources. The packaged servers can generally be referred to as instances that a user can lease for a period (e.g., intervals of seconds, minutes, hours, days, weeks, months, years, and / or other time intervals). Thus, a user can lease an instance based on the maximum computing resources that the user estimates to use (e.g., can estimate based on a spike in usage). By leasing resources in an instance, a user may use only a portion of the computing resources for most of the lease period. In other words, this results in the user paying for resources that are not consumed, and the cloud computing provider can benefit financially from leasing the unused resources.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Summary of the Invention

[0006] These and other features and advantages will be better understood upon reading the following detailed description with reference to the drawings.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2A

Figure 2B

Figure 3A

Figure 3B

Figure 3C

Figure 3D

Figure 4

Figure 5

Figure 6

DETAILED DESCRIPTION OF THE INVENTION

[0008] This specification generally discloses systems, methods, and apparatuses for measuring the usage of physical resources in computing and quantifying the usage as a single measurement unit. For example, heterogeneous computing resources can be measured and normalized (e.g., converted) to a common measurement unit, and each of the normalized measurements can be summed so that the total usage can be represented as a single value. By representing heterogeneous computing resources in a single unit, the resource usage (or consumption) can be monitored in real time, the workload can be optimized to improve the usage of available resources, and / or waste can be reduced.

[0009] Measuring physical resources (as opposed to virtual resources) can provide a more accurate measure of actual usage. For example, the use of a technique known as ballooning can cause measurements of usage based on virtual resources to overestimate the actual usage of physical resources. In other words, when only virtual resources are measured, the virtual resources may indicate that a workload is consuming (or using) more resources than the workload actually consumes within the physical resources. In some cases, the amount of virtual resources shown as being consumed may exceed the available physical resources.

[0010] As used herein, a graphics processing unit (GPU) generally can refer to computing resources that include at least a graphics processor and / or graphics memory, but in some cases, a GPU may generally be referred to as a graphics processing system (GPS).

[0011] FIG. 1 shows a schematic example of a computing network 100. As shown, the computing network 100 includes a device 102 and a computing system 104, each communicatively coupled to a network 106 (e.g., the Internet). The computing system 104 can include physical computing resources 108 configured to execute one or more workloads 110. The one or more workloads 110 can include, for example, one or more of one or more operating systems 112, one or more applications 114, and / or one or more idle processes 113. The idle process 113 can be a background process related to the operation of the computing system 104 (e.g., as a result of the operation of the hardware of the computing system 104), and can consume at least a portion of the physical computing resources 108 even if, for example, the computing system 104 does not receive a request to execute the application 114 and / or the operating system 112 from the device 102. In other words, even when the computing system 104 is in an idle state, the computing system 104 still consumes at least a portion of the physical computing resources 108 for the operation of the physical hardware of the computing system 104 when powered on.

[0012] One or more of workload 110 can be executed in response to requests generated by device 102 (e.g., operating system 112 and / or application 114). For example, device 102 can request the execution of one or more applications 114 by transmitting a request to computing system 104 through network 106. When the request is received, computing system 104 can allocate at least a portion of physical computing resources 108 for the execution of application 114. In response to the execution of application 114, computing system 104 can return data to device 102. For example, using the data returned to device 102, a graphical user interface can be generated on display 116 of device 102. Thereafter, the user of device 102 can interact with the graphical user interface, enabling device 102 to send further instructions to computing system 104 to execute different portions of application 114 on computing system 104.

[0013] As shown in the figure, a measuring instrument 118 can be provided. The measuring instrument 118 can include any combination of hardware, software, and / or firmware configured to measure the amount of physical computing resources 108 used by a workload 110 (e.g., an idle process 113, an operating system 112, and / or an application 114) over a period of time (e.g., a predetermined or non-predetermined period). For example, the measuring instrument 118 can be stored on one or more memories (e.g., any type of tangible non-transitory storage medium, including any one or more of a magnetic recording medium (e.g., a hard disk drive), an optical disk, a semiconductor device such as a read-only memory (ROM), a random access memory (RAM) such as dynamic and static RAM, an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic or optical card, or any other type of storage medium for storing electronic instructions) to execute one or more operations, and can be implemented as software configured to be executed by one or more processors (e.g., processors commercially available from INTEL, ADVANCED MICRO DEVICES, IBM, ARM, ORACLE and / or any other processor). As a further example, the measuring instrument 118 can be implemented as a circuit (e.g., an application-specific integrated circuit).

[0014] The usage of the physical computing resources 108 can be measured using the host operating system (e.g., an operating system described in assembly language) and / or hypervisor running on the computing system 104. For example, the meter 118 can include a plugin database having one or more plugins configured to enable measurement of the resource usage of the host operating system via the hypervisor running on the computing system 104. As a further example, the meter 118 can be configured to measure the resource usage directly from the physical computing resources 108 (e.g., using the host operating system). In some cases, the meter 118 can measure the usage of the physical computing resources across multiple different computing systems 104.

[0015] The meter 118 measures the usage of the physical computing resources 108, normalizes each of the physical computing resources 108 to a common unit of measurement, and combines the normalized computing resources so as to be able to represent the combined (e.g., total) usage as a single unit. In other words, the meter 118 can generally be considered to quantify the usage of multiple heterogeneous computing resources as a single unit representing the usage of the computing resources 108. For example, the meter 118 can be configured to measure the usage of multiple heterogeneous computing resources over a period of time and generate a single usage value representing the usage over that period.

[0016] To mitigate and / or avoid the impact of ballooning, the meter 118 measures, for example, the physical resource usage as opposed to the virtual resource usage. Thus, for example, the physical resource usage of a virtual machine can be measured more accurately. In some cases, the meter 118 also measures the maximum amount of available physical computing resources 108 (e.g., used and inactive resources), normalizes the measured amount of each physical resource of the physical computing resources 108 to a common measurement unit, and then sums the normalized resources to obtain the maximum amount of available physical resources represented as a single unit. By measuring the maximum amount of available resources, the user can optimize the amount of physical computing resources 108 for executing the workload 110. Additionally or alternatively, the meter 118 measures the minimum amount of physical computing resources 108 required to execute the computing system 104 (e.g., only the resources used by the idle process 113), normalizes the measured amount of each physical resource of the physical computing resources 108 to a common measurement unit, and then sums the normalized resources to obtain a single unit representing the minimum amount of physical resources that need to be used (e.g., when the computing system 104 is operating but not executing one or more operating systems 112 and / or applications 114).

[0017] In some cases, the meter 118 can be configured to measure the usage of physical resources over a period of time (e.g., a billing period for resource usage, a contract period, one hour, one day, one week, and / or any other period) and generate a single usage value representing the usage of physical resources over that period. The measured usage can then be utilized to determine the appropriate amount of resources to purchase over a period of time and / or to create a bill based on the actual usage. As a result, the resource-consuming user can reduce the amount of resources to purchase, and the resource provider can reduce the total amount of resources offered for sale and / or allocate unused resources to other users.

[0018] In some cases, the meter 118 can be configured to enable and / or disable access of the device 102 to the computing resources 108 of the computing system 104. In other words, the meter 118 can be configured to selectively communicatively couple and / or decouple the device 102 to / from the computing system 104. For example, the meter 118 can disable access by the device 102 to at least a portion of the computing resources 108 in response to expiration of a period (e.g., expiration of a contract period), non-compliance with contract terms (e.g., non-payment, misuse of computing resources, and / or the like), and / or the resource usage measured over a period exceeding or not exceeding a predetermined amount (or threshold). As a further example, the meter 118 can enable access to the computing resources 108 at a predetermined time (e.g., at the start of a contract period).

[0019] In some cases, the meter 118 can be configured to selectively enable and disable access to at least a portion of the computing resources 108 based on availability. For example, the device 102 can access at least a portion of the computing resources 108 when other users are not using the computing resources 108. However, when other users request access to the computing resources 108, a recall request is sent to the meter 118 to disable access by the device 102 to at least a portion of the computing resources 108 by the meter 118 to allow other users to access the computing resources 108. As a result, the meter 118 can generally be configured to reduce the amount of computing resources 108 not being used, for example, to run one or more applications 114 or the operating system 112.

[0020] As shown, the measuring instrument 118 can be communicatively coupled to the device 102, the computing system 104, and / or the network 106. In some instances, at least a portion of the measuring instrument 118 can be included in the device 102 and / or the computing system 104. Additionally or alternatively, at least a portion of the measuring instrument 118 can be included in a third-party device communicatively coupled to one or more of the device 102 and / or the computing system 104 (e.g., using the network 106).

[0021] The device 102 can include any one or more of a personal computer, a tablet computer, a mobile phone, a smartphone, a smartwatch, a smart TV, a fitness tracker, a smart scale (for weighing an object), a smart thermostat, a smart security monitoring system (e.g., an indoor and / or outdoor camera system, a doorbell, an alarm system, a locking system, a lighting system, and / or any other smart security monitoring system), a smart display (e.g., a display configured to communicate with the network 106), a smart medical system (e.g., an ultrasonic system, a molecular imaging system, a computed tomography system, an X-ray system, and / or any other smart medical system), a remote health management device, a gaming device (e.g., a gaming console, a handheld gaming device, a virtual reality headset, and / or any other gaming device), an Internet of Things device (e.g., a self-driving vehicle, a connected household appliance, a drone, a robot, and / or any other Internet of Things device), a server, and / or any other device communicable with the computing system 104.

[0022] Figure 2A shows a schematic example of the computing system 104 of FIG. 1. As shown, the physical computing resources 108 of the computing system 104 can include a central processing unit (CPU) 202, a memory 204, a storage 206, a network interface 208, and a graphics processing unit (GPU) 210. Each of the CPU 202, the memory 204, the storage 206, the network interface 208, and the GPU 210 has one or more metrics / attributes associated therewith.

[0023] Additionally or alternatively, the computing resources 108 can include other resources 209. The other resources 209 can include, for example, any one or more of a field programmable gate array (FPGA), a tensor processing unit (TPU), an intelligence processing unit (IPU), a neural processing unit (NPU), a vision processing unit (VPU), a digital signal processor (DSP), an accelerated processing unit (APU), an application specific integrated circuit (ASIC), a system on chip (SoC), a programmable SoC, an application specific standard product (ASSP), an adaptive computing acceleration platform (ACAP), a microcontroller, and / or any other computing resource.

[0024] As shown in FIG. 2B, the CPU 202 can be associated with metrics / attributes of the number of processing cores 212, the processor clock speed 214, and / or the processor load 216. The number of processing cores 212 indicates the number of independent processing units (or cores) available to the CPU 202. The processor clock speed 214 indicates the default clock speed of each of the cores of the CPU 202. The processor load 216 indicates the amount of computational work executed by the CPU 202. The CPU 202 can be any computer processor, including, for example, a single and / or multi-core processor capable of executing computer instructions. Examples of the CPU 202 can include processors commercially available from INTEL, ADVANCED MICRO DEVICES, IBM, ARM, ORACLE, and / or any other processor.

[0025] The memory 204 can be associated with metrics / attributes of the total available memory amount 218 and / or the memory usage 220. The memory 204 can be tangible non-transitory memory. The memory 204 can be volatile memory. For example, the memory 204 can be random access memory (RAM). Examples of RAM can include, for example, static RAM (SRAM), dynamic RAM (DRAM), fast page mode DRAM (FPM DRAM), extended data out DRAM (EDO DRAM), synchronous dynamic RAM (SDRAM), double data rate SDRAM (DDR SDRAM), RAMBUS dynamic RAM (RDRAM), and / or any other form of RAM.

[0026] Storage 206 can be associated with metrics / attributes such as the total available storage amount 222, the total storage usage amount 224, the number of bytes written to storage 206, and / or the number of bytes read from storage 206. Storage 206 can be non-volatile memory. For example, storage 206 can include any type of tangible non-transitory storage medium, including a magnetic recording medium (e.g., a hard disk drive), an optical disk, a semiconductor device such as a read-only memory (ROM), a random access memory (RAM) such as dynamic and static RAM, an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), flash memory, a magnetic or optical card, or any one or more of any type of storage medium for storing electronic instructions.

[0027] Network interface 208 can be associated with metrics / attributes such as the number of bytes transmitted 230 and / or the number of bytes received 232. Network interface 208 can be configured to communicatively couple to network 106 (FIG. 1). For example, network interface 208 can be a network interface controller configured to communicatively couple to an Ethernet network, a wireless network, a fiber channel network, a fiber distributed data interface (FDDI) network, a copper distributed data interface (CDDI) network, and / or any other network.

[0028] The GPU 210 can be associated with metrics / attributes such as the default clock speed 234 of the graphics processor, the variable clock speed 236 of the graphics processor, the graphics processor load 238, the default clock speed 240 of the graphics memory, the variable graphics memory clock speed 242, the graphics memory load 244, the graphics memory bus 246, the total amount of available graphics memory 248, the amount of graphics memory used 250, and / or the shader amount 252. The default clock speed 234 of the graphics processor indicates the default clock speed of the GPU 210's processor, and the variable clock speed 236 of the graphics processor indicates the variable clock speed of the graphics processor. In some cases, the variable clock speed 236 of the graphics processor may exceed the clock speed of the default clock speed 234 of the graphics processor. In other cases, the variable clock speed 236 of the graphics processor does not exceed the default clock speed 234 of the graphics processor. Similarly, the default clock speed 240 of the graphics memory indicates the default clock speed of the graphics memory, and the variable graphics memory clock speed 242 indicates the clock speed at which the graphics memory is actually operating. In some cases, the variable graphics memory clock speed 242 may be greater than the default clock speed 240 of the graphics memory. In other cases, the variable graphics memory clock speed 242 may be less than (or equal to) the default clock speed 240 of the graphics memory. The graphics memory load 244 indicates the amount of read and / or write operations being performed on the GPU memory. The graphics memory bus 246 indicates the bus size of the GPU 210.

[0029] The GPU 210 can include an integrated and / or dedicated GPU. For example, the GPU can include, for example, any of the GPUs made by INTEL, NVIDIA, ADVANCED MICRO DEVICES, and / or any other GPU. When the GPU 210 includes an integrated GPU, the metric / attribute can be at least partially based on the shared resources of the integrated GPU.

[0030] Although the CPU 202, memory 204, storage 206, network interface 208, and GPU 210 are shown in singular, it should be understood that the computing system 104 can include multiple physical CPUs, memories, storages, network interfaces, and GPUs, and one or more of these can be configured to cooperate, for example, to improve the performance of the computing system 104. Similarly, if the computing system 104 includes one or more of the other resources 209, one or more of each of the other resources 209 can exist.

[0031] Figure 3A shows a schematic example of the measuring instrument 118 of FIG. 1. As shown, the measuring instrument 118 can include a metric / attribute collector 302 communicatively coupled to the physical computing resource 108 (FIG. 2), a computer 305, and an analysis platform 312. Each of the metric / attribute collector 302, the computer 305, and / or the analysis platform 312 can be distributed across different computing systems (or environments). In some cases, one or more of the metric / attribute collector 302, the computer 305, and the analysis platform 312 can be disposed in the same computing system. For example, one or more of the metric / attribute collector 302, the computer 305, and / or the analysis platform 312 can be disposed on the device 102. As a further example, one or more of the metric / attribute collector 302, the computer 305, and the analysis platform 312 can be disposed on an individual device communicatively coupled to the network 106 and / or the computing system 104. As a further example, one or more of the metric / attribute collector 302, the computer 305, and the analysis platform 312 can be disposed on the computing system 104.

[0032] The metric / attribute collector 302 requests one or more metrics / attributes for one or more of the CPU 202, memory 204, storage 206, network interface 208, GPU 210, and / or other resources 209. For example, the metric / attribute collector 302 can request one or more metrics / attributes for one or more of the CPU 202, memory 204, storage 206, network interface 208, GPU 210, and / or other resources 209 for a hypervisor running on one or more of them (e.g., using a plugin). As a further example, the metric / attribute collector 302 can request one or more metrics / attributes for one or more of the CPU 202, memory 204, storage 206, network interface 208, GPU 210, and / or other resources 209 for a host operating system running on one or more of them (e.g., an operating system written in assembly language).

[0033] When the metric / attribute collector 302 requests one or more metrics / attributes for the CPU 202, the requested metrics / attributes can be input to the CPU cycle utilization computer 304. The CPU cycle utilization computer 304 can generate a metric / attribute corresponding to the CPU cycle utilization used, at least in part, based on the metrics / attributes requested for the CPU 202. For example, the CPU cycle utilization computer 304 can generate the CPU cycle utilization, at least in part, based on the number of processing cores 212, the processor clock speed 214, and the processor load 216. In these cases, the CPU cycle utilization computer 304 can generate the CPU cycles used according to the following equation.

Equation

[0034] When the metric / attribute collector 302 requests one or more metrics / attributes for the GPU 210, the requested metrics / attributes can be input to the GPU processor cycle utilization computer 306 and / or the GPU memory cycle utilization computer 308. The GPU processor cycle utilization computer 306 and / or the GPU memory cycle utilization computer 308 can generate metrics / attributes corresponding to the GPU processor cycle utilization and the GPU memory cycle utilization respectively, at least partially based on the metrics requested for the GPU 210.

[0035] For example, the GPU processor cycle utilization computer 306 can generate the GPU processor cycle utilization, at least partially based on the variable clock speed 236 of the graphics processor and the graphics processor load 238. In these cases, the GPU processor cycle utilization computer 306 can generate the utilized GPU processor cycles according to the following equation.

Number

[0036] As a further example, the GPU memory cycle utilization computer 308 can generate the GPU cycle utilization, at least partially based on the variable graphics memory clock speed 242 and the graphics memory load 244. In these cases, the GPU memory cycle utilization computer 308 can generate the clock cycles of the consumed GPU memory according to the following equation.

Number

[0037] When the metric / attribute collector 302 requests one or more metrics / attributes for the storage 206, the metrics / attributes can be input to the disk I / O computer 310. The disk I / O computer 310 can generate a metric / attribute corresponding to the total number of bits read and written to the storage 306 over a certain period (e.g., 1 second). Thus, in some cases, the disk I / O computer 310 can add the number of write bytes 226 and the number of read bytes 228 over that period to generate a metric / attribute corresponding to the disk I / O. If this period exceeds 1 second, the number of write bytes 226 and the number of read bytes 228 can be divided by the total number of seconds. If this period is less than 1 second, the number of write bytes 226 and the number of read bytes 228 can be divided by a fraction of 1 second.

[0038] When the metric / attribute collector 302 requests one or more metrics / attributes for the network interface 208, the metrics / attributes can be input to the network I / O computer 311. The network I / O computer 311 can generate a metric / attribute corresponding to the total number of bits transmitted and received by the network interface 208 over a certain period (e.g., 1 second). Thus, in some cases, the network I / O computer 311 can add the number of transmit bytes 230 and the number of receive bytes 232 over that period to generate a metric / attribute corresponding to the network I / O. If this period exceeds 1 second, the number of transmit bytes 230 and the number of receive bytes 232 can be divided by the total number of seconds. If this period is less than 1 second, the number of transmit bytes 230 and the number of receive bytes 232 can be divided by a fraction of 1 second.

[0039] As shown in the illustration, the analysis platform 312 can receive one or more metrics / attributes related to the physical computing resources 108, including, for example, one or more metrics / attributes output from one or more of the computers 305. In some cases, the analysis platform 312 can include one or more resource computers 331 corresponding to each of one or more of the other resources 209. One or more metrics / attributes output from one or more of the computers 305 can include, for example, CPU cycle utilization (e.g., output from the CPU cycle utilization computer 394), GPU processor cycle utilization (e.g., output by the GPU processor cycle computer 306), GPU memory cycle utilization (e.g., output by the GPU memory cycle utilization computer 308), disk I / O (output by the disk I / O computer 301), and / or network I / O (e.g., output by the network I / O computer 311). The analysis platform 312 can convert the received metrics / attributes using one or more conversion units. In some cases, each metric / attribute has a corresponding conversion unit. For example, the analysis platform 312 can utilize a CPU conversion unit 314, a memory conversion unit 316, a storage conversion unit 318, a network conversion unit 320, a GPU processor conversion unit 322, a GPU memory conversion unit 324, a GPU memory speed conversion unit 326, a disk I / O conversion unit 328, and / or any other conversion unit corresponding to the computing resources (e.g., a conversion unit for one or more of the other resources 209 such as FPGA resources, ASIC resources, SoC resources, DSP resources, microcontroller resources, ACAP resources, and / or the like).

[0040] The analysis platform 312 can convert heterogeneous computing resources into normalized units by dividing the usage of each physical resource (e.g., CPU 202, memory 204, storage 206, network interface 208, and GPU 210) by its respective conversion unit corresponding to the physical resource (e.g., CPU conversion unit 314, memory conversion unit 316, storage conversion unit 318, network conversion unit 320, GPU processor conversion unit 322, GPU memory conversion unit 324, GPU memory speed conversion unit 326, and disk I / O conversion unit 328), so as to obtain the normalized usage of each physical resource. In some cases, the analysis platform 312 can be configured to convert one or more of the other resources 209 into standard units. For example, the analysis platform 312 can include another resource conversion unit 329.

[0041] Next, the normalized usages can be summed to obtain a total usage value. Additionally or alternatively, as will be readily understood in light of the disclosure herein, in some cases, the normalized usages can be summed according to computing resource types (e.g., one or more of CPU 202, memory 204, storage 206, network interface 208, GPU 210, and / or other resources 209) to obtain the normalized usage value of a particular resource. For example, the normalized usages of the physical resources of the GPU 210 (e.g., GPU processor and / or GPU memory) can be summed to generate the total GPU usage. Other examples of the normalized usage of a particular physical resource can include the total FPGA usage, total ASIC usage, total DSP usage, total SoC usage, total microcontroller usage, total ACAP usage, total TPU usage, and / or any other total usage corresponding to each resource.

[0042] FIG. 3B shows a metric / attribute collector 303 that can be an embodiment of a metric / attribute collector 302 configured to monitor resource usage of a number (e.g., multiple) of different users and associate the usage of each user with the respective user. In other words, the metric / attribute collector 303 is configured to receive metrics corresponding to resource usage. The metric / attribute collector 303 can include any combination of hardware, software, and / or firmware.

[0043] The metric / attribute collector 303 can be configured to be included with any one or more of the computing system 104, the device 102, and / or any other device communicatively coupled to the network 106. For example, the device 102 can include the metric / attribute collector 303. When the device 102 includes the metric / attribute collector 303, the user of the metric / attribute collector 303 can utilize the meter 118 with a slight modification to the user's network security as compared to when the metric / attribute collector 303 is not executed on the device 102.

[0044] As shown, the metric / attribute collector 303 is communicatively coupled to the computing system 104 via a communication link 330. The communication link 330 can transmit data (e.g., metrics / attributes related to the computing resource 108) to the metric / attribute collector 303. The data transmitted over the communication link 330 can be encrypted to prevent unauthorized access to the data. As shown, the metric / attribute collector 303 can be remote from the computing system 104 (e.g., operate on a different server, within a different network, and / or on a different device). However, in some cases, the metric / attribute collector 303 can be local to the computing system 104 (e.g., operate on the same server, on the same network, and / or on the same device).

[0045] The metric / attribute collector 303 can also be configured to communicate with the analysis platform 312 via the communication link 362. The data transmitted on the communication link 362 can be encrypted to prevent unauthorized access to the data. In some cases, the metric / attribute collector 303 can include the analysis platform 312. Accordingly, the metric / attribute collector 303 cannot transmit data on the communication link 362.

[0046] The analysis platform 312 can generate a total resource usage value represented as a single normalization unit. As shown, the analysis platform 312 can be remote from the metric / attribute collector 303 (e.g., operating on a different server, within a different network, and / or on a different device). However, in some cases, the analysis platform 312 can be local to the metric / attribute collector 303 (e.g., operating on the same server, on the same network, and / or on the same device).

[0047] As shown, the metric / attribute collector 303 can include at least one collector core 332, at least one collector metric receiver 334, at least one collector synchronizer 336, at least one plugin database 338, at least one collector automator 340, at least one machine manager 342, at least one database manager 344, at least one account manager 346, and at least one machine collection manager 348. The collector core 332 communicates with and / or manages each of the collector metric receiver 334, collector synchronizer 336, plugin database 338, collector automator 340, machine manager 342, collector database manager 344, account manager 346, and machine collection manager 348.

[0048] The collector metric receiver 334 is configured to receive metrics / attributes representing the usage of computing resources 108 of one or more users and / or workloads. In some cases, the collector metric receiver 334 can include one or more of the computers 305.

[0049] At least a portion of the resource usage can be associated with one or more users. For example, at least a portion of the resource usage can be associated with one or more accounts 350 using the account manager 346. The account 350 can be associated with one or more resource-consuming users 352. As a result, the resource usage of multiple accounts 350 and / or resource-consuming users 352 can be monitored simultaneously. For example, an enterprise can have accounts 350 associated with its employees (e.g., resource-consuming users 352). As a result, when an employee accesses the computing resources 108, the usage is associated with the account 350 corresponding to the enterprise. By associating the usage with the account 350, multiple accounts 350 can be monitored using a single metric / attribute collector 303, which can reduce the resources consumed compared to the case where the metric / attribute collector 303 is associated with each account 350 and / or resource-consuming user 352.

[0050] Account 350 can be associated with one or more machines 354 (e.g., virtual or physical) that are accessible to resource-consuming user 352 and operate using computing resources 108. Machine 354 can include, for example, one or more of operating system 112 and / or application 114. For example, machine 354 can be configured to execute a LINUX-based operating system, a WINDOWS-based operating system, a MAC-based operating system, and / or any other computing system in a virtual or physical environment. In some cases, machine 354 can be configured to execute one or more workloads 110 (e.g., application 114) without executing a guest operating system. For example, machine 354 can be configured to directly execute one or more applications 114 on a physical machine without any form of virtualization, or to execute containers and configure one or more applications 114 to execute within the containers. Thus, a container can generally be an isolated environment configured to execute on one or more kernels of machine 354. Containers can execute on physical or virtual machines.

[0051] The machine collection manager 348 can associate a plurality of machines 354 with at least one machine collection 356. The machine collection manager 348 can also associate a plurality of containers or uniquely identified (e.g., tagged) workloads with at least one collection. In some cases, the plurality of machine collections 356 can be associated with respective accounts 350. For example, the owner (or authorized user) of each account 350 can add one or more machines 354 to each machine collection 356. Each machine collection 356 can include a plurality of machines 354, each of which is configured to execute, for example, different operating systems 112 and / or applications 114 (e.g., applications 114 can be executed using containers).

[0052] The plugin database 338 can be configured to be accessed by the metric / attribute collector 303, and the metric / attribute collector 303 can communicate with one or more computing systems 104 to enable, for example, measuring resource usage. One or more plugins 358 (for example, the plugin 358 can be a driver, a listener, a publisher, or the like) can be included. Each computing system 104 can be configured to include one or more machines 354 each configured to execute a different operating system 112 and / or application 114. In other words, through the plugin 358, the metric / attribute collector 303 can simultaneously monitor resource usage across multiple different computing environments. For example, the plugin 358 can include a LINUX plugin, a WINDOWS plugin, a MAC plugin, a SOLARIS plugin, a VMWARE plugin, a HYPER-V plugin, a CITRIX XEN plugin, an OPENSTACK plugin, an OPENSHIFT plugin, a KUBERNETES plugin, a DOCKER plugin, a PIVOTAL plugin, an AMAZON plugin, a MICROSOFT plugin, a GOOGLE plugin, a bare metal plugin (for example, a plugin configured to communicate directly with a non-virtual machine), and / or any other plugin that enables monitoring usage across multiple operating systems / environments. The plugin 358 can also enable simultaneous monitoring of one or more virtual and / or physical environments. The plugin 358 can also enable monitoring of billing information, software program utilization, leased software licenses, and / or any other additional user account information.

[0053] The plugin 358 can be configured to be activated (e.g., the metric / attribute collector 303 can communicate with each computing system 104 using each plugin 358), or deactivated (e.g., the metric / attribute collector 303 cannot communicate with each computing system 104 using each plugin 358).

[0054] The machine manager 342 can assign and / or associate the machines 354 to respective machine collections 356. The machine manager 342 can analyze the machines 354 within the machine collection 356 to determine, for example, the specifications and / or operating system of the machines 354. The specifications can include information related to computing resources 108 (e.g., available memory, storage, and / or the like).

[0055] The machine manager 342 can also perform a search for new machines to be monitored by the metric / attribute collector 303. The machine manager 342 can detect new machines by searching for new IP addresses associated with a particular account 350 and / or resource-consuming user 352.

[0056] The Collector Database Manager 344 manages data received and / or generated by the Metric / Attribute Collector 303. For example, the Collector Database Manager 344 can receive and / or store one or more records (e.g., data) related to the machine 354, the machine collection 356, the account 350, and the resource usage (e.g., metric / attribute) associated therewith. The Collector Database Manager 344 can also delete data. For example, the Collector Database Manager 344 can delete records related to each machine 354, each machine collection 356, and / or each account 350. In some cases, the Collector Database Manager 344 can truncate records related to each machine 354, each machine collection 356, and / or each account 350. Deletion and / or truncation can be performed, for example, after a certain period of time has elapsed and / or after each account 350 has been deactivated / ended.

[0057] The Collector Database Manager 344 can communicate with the Collector Database 364 via the communication link 366 to store one or more records on the Collector Database 364. As shown, the Collector Database 364 can be remote from the Metric / Attribute Collector 303 (e.g., operating on a different server and / or a different network). However, in some cases, the Collector Database 364 can be local to the Metric / Attribute Collector 303 (e.g., operating on the same server, on the same network, and / or on the same device). The data transmitted via the communication link 366 can be encrypted to prevent unauthorized access to the data being transmitted. In some cases, the records stored on the Collector Database 364 can be encrypted to prevent unauthorized access to the records. For example, the records stored on the Collector Database 364 can be encrypted using Transparent Data Encryption (TDE).

[0058] The collector synchronizer 336 can communicate, for example, with the collector database manager 344, and the collector database manager 344 can replicate and / or back up the data it receives. For example, the collector synchronizer 336 can replicate one or more records stored on and / or to be stored on the collector database 364 to the collector backup 368. In some cases, the records stored on the collector backup 368 can be encrypted to prevent unauthorized access. When one or more records are modified, the collector synchronizer 336 can replicate and / or back up only that modification to the collector backup 368. As a result, the collector synchronizer 336 does not necessarily replicate the entire record to the collector backup 368. As shown, the collector synchronizer 336 can communicate with the collector backup 368 via a communication link 370. The data transmitted via the communication link 370 can be encrypted to prevent unauthorized access to the data in transit. As shown, the collector backup 368 can be remote from the metric / attribute collector 303 (e.g., operating on a different server, within a different network, and / or on a different device). However, in some cases, the collector backup 368 can be local to the metric / attribute collector 303 (e.g., operating on the same server, on the same network, and / or on the same device).

[0059] In some cases, at least one backup collector 397 can be provided. The backup collector 397 can perform substantially the same operations as the metric / attribute collector 303 and can be configured to run in parallel with (or simultaneously with) the metric / attribute collector 303. In some cases, the backup collector 397 is configured to be remote from the metric / attribute collector 303 (e.g., operate on a different server, within a different network, and / or on a different device). As a result, in the event of a failure of the metric / attribute collector 303, the gauge 118 can continue to monitor resource usage. In other words, the backup collector 397 can generally provide redundancy. In some cases, for example, there may be two or more backup collectors 397 so that n + 2 redundancy can be achieved. Accordingly, the backup collector 397 can include any one or more of the functions discussed herein with respect to the metric / attribute collector 303. The backup collector 397 can communicate with the computing system 104 via the communication link 399. The data transmitted via the communication link 399 can be encrypted to prevent unauthorized access to the data in transit.

[0060] The backup collector 397 can also be configured to generate independent records. The independent records can be used, for example, during an audit process and / or in a data recovery process (e.g., if the metric / attribute collector 303 experiences a failure). In some cases, the collector backup 368 can include the backup collector 397.

[0061] As shown, the metric / attribute collector 303 can also include a collector automator 340. The collector automator 340 can automate one or more functions of the metric / attribute collector 303. For example, the collector automator 340 can be configured to receive a trigger event that causes the metric / attribute collector 303 to perform a predetermined operation (e.g., start or end receiving metrics / attributes, start database cleanup, enable or disable access to computing resource 108, activate or deactivate one or more plugins 358, and / or any other operation). The trigger event can include, for example, discovery of a new machine, expiration of a contract period, use of a predetermined amount of resources, addition of a new machine to each machine collection 356, and / or the like. In some cases, the trigger event can be the expiration of a period. For example, every 1 second, 2 seconds, 3 seconds, 4 seconds, or 5 seconds (or at any other appropriate interval), the metric / attribute collector 303 can search for a new Internet Protocol (IP) address corresponding to a new machine. In these cases, the metric / attribute collector 303 can generally be described as self-updating.

[0062] As also shown in the figure, the administrator user interface 372 can be communicatively coupled to the collector core 332. The administrator user interface 372 can provide an interface for an operator of the metric / attribute collector 303 to control and / or modify the metric / attribute collector 303 and / or review data generated and / or received by the metric / attribute collector 303. For example, the administrator user interface 372 can be configured to display and / or generate usage summaries on an account 350 or resource consumption user 352 basis. The administrator user interface 372 can also be used, for example, to manually add one or more machines 354, adjust and / or delete one or more records, add or delete a new account 350, add or delete a plugin 358, adjust / create a trigger for the collector automator 340, for example, to continue and / or interrupt monitoring of resource usage at a predetermined time in response to a contract period, and / or perform any other administrative task.

[0063] As shown in the figure, the administrator user interface 372 can communicate with the collector core 332 via a communication link 374. The data transmitted via the communication link 374 can be encrypted to prevent unauthorized access to the data being transmitted. As shown in the figure, the administrator user interface 372 can be remote from the metric / attribute collector 303 (e.g., operating on a different server, within a different network, and / or on a different device). However, in some cases, the administrator user interface 372 can be local to the metric / attribute collector 303 (e.g., operating on the same server, on the same network, and / or on the same device).

[0064] In some cases, for example, the administrator user interface 372 can be generated and displayed on a web browser. As a further example, in some cases, the administrator user interface 372 can be an application that runs on a computer that communicates with the metric / attribute collector 303 through a network connection (e.g., the Internet). As a further example, in some cases, the administrator user interface 372 can be an application that runs locally (e.g., on the same device as the metric / attribute collector 303).

[0065] FIG. 3C shows an example of an analysis platform 360 that can be an example of the analysis platform 312 of FIG. 3A. The analysis platform 360 is generally configured to normalize (e.g., convert) a plurality of metrics / attributes received over a period of time into normalized units, sum the normalized metrics / attributes, and generate a total resource usage value represented as a single normalized unit for that period. In other words, the analysis platform 360 can be configured to generate a single usage value representing the total resource usage over a period of time. The analysis platform 360 can include any combination of hardware, software, and / or firmware. In some cases, the analysis platform 360 can run on the same server and / or the same network as the metric / attribute collector 302. In other cases, the analysis platform 360 can run on a server and / or network different from the metric / attribute collector 302 (e.g., on the user's network and / or server).

[0066] The analytics platform 360 can be configured to receive one or more metrics / attributes representing the usage of computing resources 108 from one or more of the metric / attribute collector 302 and / or one or more backup collectors 397. In some cases, the analytics platform 360 can receive one or more metrics / attributes from the computing system 104. In these cases, the gauge 118 does not include the metric / attribute collector 302, and the analytics platform 360 can include the plug-in database 338. The analytics platform 360 can receive the metrics / attributes in the form of, for example, JavaScript Object Notation (JSON) files, Comma Separated Values (CSV) files, and / or any other file type.

[0067] The analytics platform 360 can be configured to be included with any one or more of the computing system 104, the device 102, and / or any other device communicatively coupled to the network 106. For example, the device 102 can include the analytics platform 360. When the device 102 includes the analytics platform 360, a user of the analytics platform 360 can utilize the gauge 118 with minor modifications to the user's network security as compared to when the analytics platform 360 is not executed on the device 102. In some cases, when the device 102 includes the analytics platform 360, the analytics platform 360 can be configured to analyze, for example, the resource usage of one or more users on a local network, each having their respective device. Additionally or alternatively, when the device 102 includes both the analytics platform 360 and the metric / attribute collector 302, only the metrics / attributes representing the usage of the computing resources 108 need to be transmitted over a public network (e.g., the Internet).

[0068] The analysis platform 360 can include at least one analysis core 376, at least one analysis metric receiver 378, at least one analysis synchronizer 380, at least one analysis database manager 382, at least one analysis automator 384, at least one usage analyzer 386, at least one notification generator 388, at least one subscription manager 390, and at least one exchange / contract manager 393. The analysis core 376 can generally communicate with and / or manage each of the analysis metric receiver 378, the analysis synchronizer 380, the analysis database manager 382, the analysis automator 384, the usage analyzer 386, the notification generator 388, and the subscription manager 390.

[0069] As shown, the analysis core 376 is communicatively coupled to the analysis metric receiver 378. The analysis metric receiver 378 receives metrics / attributes representing the usage of computing resources 108 from the metric / attribute collector 302. The analysis metric receiver 378 can include one or more of the computers 305. In some cases, the analysis metric receiver 378 can be configured to generate a total resource usage value represented as a single normalization unit. In these cases, the analysis core 376 can be configured to receive the total resource usage value represented as a single normalization unit. In other words, the analysis core 376 can be configured to generate the total resource usage value represented as a single normalization unit.

[0070] The analysis core 376 and / or the analysis metric receiver 378 can communicate with the usage analyzer 386. The usage analyzer 386 monitors the usage associated with each account 350 and / or resource consumption user 352 to create (or determine) a standard usage, peak usage, and / or low value usage. In other words, the standard usage, peak usage, and / or low value usage can be at least partially based on the received metrics / attributes corresponding to each of the computing resources 108. In some cases, for example, the peak usage can represent the maximum usage over a period (e.g., 1 hour, 1 day, 1 week, 1 month, and / or any other suitable period), the low value usage can represent the minimum usage over a period (e.g., 1 hour, 1 day, 1 week, 1 month, and / or any other suitable period), and the standard usage can represent the average usage over a period (e.g., 1 hour, 1 day, 1 week, 1 month, and / or any other suitable period). In some cases, the peak usage can approximate the maximum usage over a period (e.g., a usage value within 5%, 10%, or 15% of the actual maximum usage), and the low value usage can approximate the minimum usage over a period (e.g., a usage value within 5%, 10%, or 15% of the actual minimum usage).

[0071] The standard usage, peak usage, and / or bottom usage can correspond to individual computing resources, multiple computing resources, and / or one or more sub-resources (e.g., one or more of the graphics processor and graphics memory of GPU 210) that form the computing resources. For example, the standard usage, peak usage, and / or bottom usage can be generated for each of multiple computing resources (e.g., one or more of CPU 202, memory 204, storage 206, network interface 208, GPU 210, and / or other resource 209). In these cases, the standard usage, peak usage, and / or bottom usage of each resource can be in normalized units and / or non-normalized units (e.g., units corresponding to individual resources). As a further example, the standard usage, peak usage, and / or bottom usage corresponding to multiple computing resources (e.g., two or more of one or more of CPU 202, memory 204, storage 206, network interface 208, GPU 210, and / or other resource 209) can be generated. In these cases, the standard usage, peak usage, and / or bottom usage can be represented as a single normalized unit.

[0072] In some cases, the usage analyzer 386 can be configured to generate an overall peak usage average. The overall peak usage average can generally be an average of two or more peak usage values corresponding to two or more respective predetermined periods. For example, the overall peak usage average for a week can be determined by determining and averaging the peak usage for each day of the week. Additionally or alternatively, the usage analyzer 386 can be configured to generate an overall bottom usage average. The overall bottom usage average can generally be an average of two or more bottom usage values corresponding to two or more respective predetermined periods. For example, the overall bottom usage average for a week can be determined by determining and averaging the bottom usage for each day of the week.

[0073] The overall peak usage average and / or the overall trough usage average can correspond to individual computing resources and / or a plurality of computing resources. For example, the overall peak usage average and / or the overall trough usage average can be generated for each of a plurality of computing resources (e.g., each of one or more of CPU 202, memory 204, storage 206, network interface 208, GPU 210, and / or other resource 209). In these cases, the overall peak usage average and / or the overall trough usage average for each resource can be expressed as a normalized unit and / or as a non-normalized unit (e.g., a unit corresponding to an individual resource). As a further example, an overall peak usage average and / or an overall trough usage average corresponding to a plurality of computing resources (e.g., two or more of one or more of CPU 202, memory 204, storage 206, network interface 208, GPU 210, and / or other resource 209) can be generated. In these cases, the overall peak usage average and / or the overall trough usage average can be expressed as a single normalized unit.

[0074] The standard average usage can generally be the average resource usage calculated by taking multiple measurements of the actual resource usage over a period within the repetition time interval and averaging the measurements. For example, the actual resource usage can be measured at 11:00 am every day for 5 days, and based on these measurements, the average usage at 11:00 am can be calculated. The peak average usage can generally be the average resource usage calculated by taking multiple measurements of the actual resource usage over a period within the repetition time interval, selecting the measurement portion with the highest usage (e.g., the highest 5%, 10%, 15%, 20%, 25%, 35%, 40%, 50%, or 60% of the resource usage), and calculating the average of the selected portion. For example, the actual resource usage can be measured at 11:00 am every day for 5 days, and based on the three highest measurements, the peak average usage at 11:00 am can be calculated. The bottom average usage can generally be the average resource usage calculated by taking multiple measurements of the actual resource usage over a period within the repetition time interval, selecting the measurement portion with the lowest usage (e.g., the lowest 5%, 10%, 15%, 20%, 25%, 35%, 40%, 50%, or 60% of the resource usage), and calculating the average of the selected portion. For example, the actual resource usage can be measured at 11:00 am every day for 5 days, and based on the three lowest measurements, the bottom average usage at 11:00 am can be calculated.

[0075] The standard average usage, peak average usage, and / or bottom average usage can correspond to individual computing resources and / or a plurality of computing resources. For example, the standard average usage, peak average usage, and / or bottom average usage can be generated for each of a plurality of computing resources (e.g., each of one or more of CPU 202, memory 204, storage 206, network interface 208, GPU 210, and / or other resource 209). In these cases, the standard average usage, peak average usage, and / or bottom average usage of each resource can be expressed as a normalized unit and / or as a non-normalized unit (e.g., a unit corresponding to an individual resource). As a further example, a standard average usage, peak average usage, and / or bottom average usage corresponding to a plurality of computing resources (e.g., two or more of one or more of CPU 202, memory 204, storage 206, network interface 208, GPU 210, and / or other resource 209) can be generated. In these cases, the standard average usage, peak average usage, and / or bottom average usage can be expressed as a single normalized unit.

[0076] Usage analyzer 386 can generate a standard average usage pattern 392, a peak average usage pattern 394, and a bottom average usage pattern 395 based on each of the standard average usage, peak average usage, and bottom average usage (shown in FIG. 3? for clarity). The standard average usage pattern 392 can be a data set including a plurality of standard average usage values corresponding to different predetermined times (or predetermined periods). The peak average usage pattern 394 can be a data set including a plurality of peak average usage values corresponding to different predetermined times (or predetermined periods). The bottom average usage pattern 395 can be a data set including a plurality of bottom average usage values corresponding to different predetermined times (or predetermined periods).

[0077] A dataset corresponding to the standard average usage pattern 392 can be compared with a dataset corresponding to the peak average usage pattern 394 to determine a threshold usage value or range for one or more predetermined times (or predetermined periods). For example, the threshold usage value for a predetermined time can be obtained by taking the average of the standard average usage value and the peak average usage value at each predetermined time. As a further example, the threshold usage range can be expressed as the range from the standard average usage value to the peak average usage value at each predetermined time.

[0078] Additionally or alternatively, a dataset corresponding to the bottom average usage pattern 395 can be compared with a dataset corresponding to the peak average usage pattern 394 to determine a threshold usage value or range for one or more predetermined times (or predetermined periods). For example, the threshold usage value for a predetermined time can be obtained by taking the average of the bottom average usage value and the peak average usage value at each predetermined time. As a further example, the threshold usage range can be expressed as the range from the bottom average usage value to the peak average usage value at each predetermined time.

[0079] Additionally or alternatively, a dataset corresponding to the standard average usage pattern 392 can be compared with a dataset corresponding to the bottom average usage pattern 395 to determine a threshold usage value or range for one or more predetermined times (or predetermined periods). For example, the threshold usage value for a predetermined time can be obtained by taking the average of the standard average usage value and the bottom average usage value at each predetermined time. As a further example, the threshold usage range can be expressed as the range from the bottom average usage value to the standard average usage value at each predetermined time.

[0080] Additionally or alternatively, the threshold usage value can be the bottom average usage value, the peak average usage value, the bottom usage value (i.e., the minimum usage over a predetermined time) or the overall bottom usage average value, and / or the peak usage value (i.e., the maximum usage over a predetermined time) or the overall peak usage average value.

[0081] The notification generator 388 can be configured to generate a notification, for example, in response to one or more of a resource usage amount that meets, exceeds, and / or does not exceed a threshold usage amount value, and / or a resource usage amount outside a threshold usage amount range, at a predetermined time (or a predetermined time interval). In other words, the notification can be generated at least partially based on a comparison with a threshold or a threshold usage amount range of the resource usage amount. For example, detecting a usage amount that exceeds the threshold usage amount value may indicate the use of the resource by an unauthorized third party, and detecting a usage amount that does not exceed the bottom average usage amount value may indicate a hardware failure (for example, at least a part of the computing resource 108 may be malfunctioning).

[0082] In some cases, the usage analyzer 386 can be configured to identify outliers (for example, resource usage amounts outside a threshold usage amount range). For example, the usage analyzer 386 can utilize machine learning to identify high-usage and low-usage periods. During high-usage and / or low-usage periods, the usage analyzer 386 can identify a usage amount as an outlier if, for at least a certain period (for example, 10 minutes, 30 minutes, 60 minutes, 2 hours, and / or any other time period), the usage amount does not exceed the peak average usage amount value and / or does not exceed the bottom average usage amount value. As a result, the notification generator 388 can be configured not to generate a notification in response to the detection of an outlier. Further, when an outlier is identified, the usage analyzer 386 can exclude this outlier from the calculation of the standard average usage pattern 392, the peak average usage pattern 394, and / or the bottom average usage pattern 395, for example, unless the outlier is detected multiple times. Accordingly, each of the standard average usage pattern 392, the peak average usage pattern 394, and / or the bottom average usage pattern 395 can generally be generated using machine learning.

[0083] In some cases, the usage analyzer 386 can analyze the usage of other users of the meter 118 to determine whether other users are experiencing similar usage patterns. For example, if multiple users are experiencing resource usage that does not exceed the low average usage value, it may indicate a hardware failure (e.g., at least a portion of the computing resource 108 may be malfunctioning). In this case, the notification generator 388 can generate a notification.

[0084] The generated notification can be in the form of an alert sent to one or more owners / users of the account 350 via, for example, a phone call, an SMS text message, an email, a fax, a tactile sensation (e.g., via a smartwatch, a fitness tracker, a remote health monitoring device, and / or any other wearable device), an alert generated within an application or software to manage the analytics platform 360 and / or the metric / attribute collector 302 (e.g., an in-application message), and / or any other type of alert.

[0085] The usage analyzer 386 can determine, for example, using machine learning, periods of high resource usage and periods of low resource usage that occur regularly over a period of time. Once the periods of high and low resource usage are identified, for example, the threshold usage for generating an alert can be adjusted. For example, during a period known to be of high resource usage, the threshold usage for generating an alert can be adjusted to be higher than during a period known to be of low resource usage.

[0086] The usage analyzer 386 can also be configured to generate a benchmark (e.g., an average cost metric) representing the cost per unit of consumed resources for one or more users and / or accounts. The usage analyzer 386 can receive a total resource usage value represented as a single normalized unit, related to one or more accounts 350, resource-consuming users 352, and / or machine collections 356. In some cases, the usage analyzer 386 can receive total resource usage values corresponding to a workload type (e.g., a SPARK cluster workload, an ELASTICSEARCH workload, a data storage workload, a disaster recovery workload, a developer workload, a quality assurance workload, a manufacturing workload, and / or the like), a service provider (e.g., AMAZON, MICROSOFT, GOOGLE, and / or any other service provider), a hardware provider (e.g., DELL, SUPERMICRO, HP, and / or any other hardware provider), a hardware configuration, a hypervisor (e.g., VMWARE, CITRIX, OPENSTRACK, and / or any other hypervisor), and / or a group (e.g., a business unit within an organization such as a technical department, a sales department, or a marketing department).

[0087] The usage analyzer 386 can also receive cost data related to the resources consumed. The cost data can be received from the computing system 104 (e.g., a service provider), from user input, and / or from any other source. The cost data can represent the cost of the total resource usage related to each account 350, resource-consuming user 352, machine collection 356, workload type, and / or group. The cost data related to the resources consumed can include one or more of a plurality of collocation costs, hardware costs, networking costs, cloud hosting costs, labor costs, software license costs, compliance costs, managed service costs, consultant costs, support costs, developer / operation education costs, cybersecurity costs, migration opportunity costs, and / or any other related costs for the resources consumed. Next, the usage analyzer 386 can divide the cost of the total resource usage by the total resource usage value to obtain the cost per unit consumption value. The usage analyzer 386 can utilize the cost per unit consumption to generate a benchmark (e.g., an average cost metric).

[0088] Next, the cost per unit consumption value can be compared to, for example, a market benchmark (e.g., the average cost per unit across all users of one or more gauges 118), a group or industry benchmark (e.g., the average cost per unit across all users within a given industry or group), a workload benchmark (e.g., the average cost per unit across all users executing a given workload), a service provider benchmark (e.g., AMAZON, MICROSOFT, GOOGLE, and / or any other service provider), and / or the like. The group or industry benchmark can be based on, for example, the size of the company (e.g., market capitalization, revenue, and / or the like), business units or departments within a particular company or across multiple companies (e.g., sales, technology, and / or the like) and / or any other grouping.

[0089] In some cases, the usage analyzer 386 can also be configured to generate market benchmarks, group or industry benchmarks, service provider benchmarks, and / or workload benchmarks. For example, the usage analyzer 386 can be configured to access data related to the resource usage and resource costs of one or more accounts 350 corresponding to one or more different users.

[0090] In some cases, the usage analyzer 386 can be separated from the analysis platform 360. For example, the usage analyzer 386 can be configured to communicate with multiple analysis platforms 360 and / or metric / attribute collectors 302. In some cases, there may be a market usage analyzer configured to determine the average usage and average cost remotely from the analysis platform 360 for a particular group or industry, a particular account, a particular workload, and the like across multiple different users. In these cases, there can also be a local usage analyzer configured to determine the average usage and average cost for a particular analysis platform 360.

[0091] In some cases, the usage analyzer 386 can also be configured to analyze the consumed computing resources 108 to determine whether the workload 110 running on the computing resources 108 is using the computing resources 108 efficiently. For example, the usage analyzer 386 can analyze the specifications of the computing resources 108 (e.g., processor speed and / or size, memory speed and / or size, GPU speed and / or size, storage speed and / or size, and / or the like) and compare them with the requirements of the workload 110 so that the usage analyzer 386 can generate an optimized computing resource configuration (e.g., the best estimated configuration). In this case, the usage analyzer 386 can be configured to generate a notification to be sent to the user, and the notification can include a description of the optimized computing resource configuration. In some cases, the usage analyzer 386 can be configured to automatically migrate the workload 110 to the optimized computing resource configuration. As a further example, the computing resources 108 can include a dedicated graphics processing unit, but the applications 114 and / or the operating system 112 running on the computing resources 108 may not require the dedicated graphics processing unit to operate efficiently. In this case, the usage analyzer 386 can be configured to generate a notification to be sent to the user. In some cases, the usage analyzer 386 can be configured to automatically move the workload 110 to another computing system 104 that does not include a dedicated graphics processing unit. As a result, the cost per unit of the consumed computing resources can be reduced.

[0092] In some cases, the usage analyzer 386 can be further configured to generate computing resource configuration recommendations based on the workload 110 to be executed. The computing resource configuration recommendations can be associated with the cost per unit of the computing resource consumption value of each workload 110, and / or an estimated performance rating. As a result, the user can compare the resource costs and / or performance among multiple resource providers and / or multiple computer resource options available from a specific resource provider.

[0093] In some cases, the usage analyzer 386 can be configured to analyze the user's resource usage when the user pre-purchases a predetermined amount of resources. In these cases, the usage analyzer 386 can convert the pre-purchased resource amount into a normalized value representing the maximum available resource amount, and compare the normalized value of the pre-purchased resources with the peak usage value. In other words, the usage analyzer 386 can be configured to determine whether the resource supply meets the user's peak resource demand when the user pre-purchases a predetermined amount of resources. Therefore, based on this comparison, the usage analyzer 386 can be configured to provide recommendations regarding the amount of resources to pre-purchase.

[0094] The subscription manager 390 manages and holds subscription information related to the users of the meter 118. For example, the subscription can determine how many resource-consuming users 352 can be assigned to each account 350, how many accounts can be assigned to each user (e.g., an enterprise entity), how many machines 354 each user can access, how many machines 354 can be assigned to the machine collection 356, how many machine collections 356 each user can have, associate login information with each user, and determine the validity of the login information (e.g., whether the login information has expired), and / or any other task related to subscription management.

[0095] The exchange / contract manager 393 is configured to coordinate a contract between two parties (e.g., a purchaser and a seller). The exchange / contract manager 393 can be configured to disable the meter 118 according to the contract period. When the meter 118 is disabled, the user's access to the computing resource 108 can be disabled. The exchange / contract manager 393 can also be configured to audit the resource usage. Auditing the resource usage can include verifying that the resources shown as being in use are actually being used and / or that the usage is in accordance with the contract. The auditing process can include comparing the data collected by the metric / attribute collector 302 with the data generated by a second collector (e.g., a backup collector).

[0096] The analysis automator 384 can automate one or more functions of the analysis platform 360. For example, the analysis automator 384 can be configured to receive a trigger event that causes the analysis platform 360 to perform a predetermined operation. The predetermined operation can include, for example, enabling or disabling access to the computing resource 108, activating or deactivating at least a portion of the metric / attribute collector 302 (e.g., the metric receiver 334), activating or deactivating one or more plugins 358 (e.g., one or more plugins 358 configured to enable communication with the computing system 104), initiating a database cleanup, and / or one or more of any other operations. The trigger event can include, for example, receiving a new contract, receiving a new subscription, user input from the administrator user interface 372, a predetermined time (e.g., the start of a new contract / subscription and / or any other predetermined time), a recall request, and / or the like. In some cases, the trigger event can be the expiration of a period (e.g., the expiration of the contract period). For example, the analysis platform 360 can perform a predetermined operation in response to the expiration of the contract period.

[0097] As shown, the analysis database manager 382 communicates with the analysis core 376 and the analysis synchronizer 380. The analysis database manager 382 manages the data received and / or generated by the analysis platform 360. For example, the analysis database manager can manage and / or store subscriptions, contracts, metrics / attributes, usage patterns (e.g., standard average usage pattern 392, peak average usage pattern 394, or trough average usage pattern 395), notification thresholds, one or more records (e.g., data) related to trigger events, and / or any other data received by the analysis platform 360. The analysis database manager 382 can also erase and / or discard data. For example, the analysis database manager 382 can erase and / or discard records related to subscriptions, contracts, metrics / attributes, usage patterns (e.g., standard average usage pattern 392, peak average usage pattern 394, or trough average usage pattern 395), notification thresholds, trigger events, and / or any other data received or collected by the analysis platform 360. The erasure and / or discard can be performed, for example, after the passage of a certain period and / or after the inactivation / termination of a subscription or contract, for example.

[0098] The analysis database manager 382 can communicate with the analysis database 396 via the communication link 398 and can store one or more records on the analysis database 396. As shown, the analysis database 396 can be remote from the analysis platform 360 (e.g., operating on a different server, within a different network, and / or on a different device). However, in some cases, the analysis database 396 can be local to the analysis platform 360 (e.g., operating on the same server, on the same network, and / or on the same device). Data transmitted via the communication link 398 can be encrypted to prevent unauthorized access to the data in transit. In some cases, records stored on the analysis database 396 can be encrypted to prevent unauthorized access to the records. For example, records stored on the analysis database 396 can be encrypted using transparent data encryption (TDE).

[0099] The analysis synchronizer 380 can communicate with the analysis metric receiver 378 and / or the analysis database manager 382 to replicate and / or back up data received by the analysis platform 360 (e.g., data received by the analysis database manager 382). For example, the analysis synchronizer 380 can replicate and / or back up one or more records stored on and / or to be stored on the analysis database 396 to the analysis backup 381. In some cases, the records stored on the analysis backup 381 can be encrypted to prevent unauthorized access. When one or more records are changed, the analysis synchronizer 380 can back up only that change to the analysis backup 381. As shown, the analysis synchronizer 380 can communicate with the analysis backup 381 via a communication link 383. The data transmitted via the communication link 383 can be encrypted to prevent unauthorized access to the data in transit. As shown, the analysis backup 381 can be remote from the analysis platform 360 (e.g., operating on a different server, within a different network, and / or on a different device). However, in some cases, the analysis backup 381 can be local to the analysis platform 360 (e.g., operating on the same server, on the same network, and / or on the same device).

[0100] In some cases, a backup analysis platform 355 that performs substantially the same operations as the analysis platform 360 can be provided. As a result, independent records can be maintained. The independent records can be used, for example, in an audit process and / or a data recovery process (e.g., when the analysis platform 360 experiences a failure). The backup analysis platform 355 can be communicatively coupled to the metric / attribute collector 302 using a communication link 353.

[0101] In some cases, the analytics backup 381 can also include a backup analytics platform that performs substantially the same operations as the analytics platform 360. In other words, the analytics backup 381 can include a stand-alone analytics platform. As a result, independent records can be maintained. The independent records can be used, for example, in an audit process and / or a data recovery process (e.g., if the analytics platform 360 experiences a failure).

[0102] As shown, the usage predictor 385 can be communicatively coupled to the analytics core 376 via a communication link 387. The usage predictor 385 can estimate the future usage of each user (e.g., who owns or controls one or two or more accounts 350), each account 350, and / or each resource-consuming user 352. The usage predictor 385 can predict future usage, for example, using a standard average usage pattern 392, a peak average usage pattern 394, and / or a trough average usage pattern 395. Additionally or alternatively, the usage predictor 385 can utilize machine learning to analyze the history of resource usage to predict future usage. For example, the usage predictor 385 can analyze usage with respect to patterns and / or spikes (e.g., increases in resource usage). Once future usage is predicted, the usage predictor 385 can also estimate the predicted economic cost corresponding to the predicted usage.

[0103] As shown in the illustration, the analytics user interface 389 can communicate with the analytics core 376 via a communication link 391. The analytics user interface 389 can be configured to generate a display that presents information related to the analytics platform 360 and / or the metric / attribute collector 302. For example, the analytics user interface 389 can display usage predictions, standard average usage patterns 392 (e.g., as a graph or plot), peak average usage patterns 394 (e.g., as a graph or plot), bottom average usage patterns 395 (e.g., as a graph or plot), the current contract period, the current subscription, active and / or inactive notifications / alerts, and / or any other data related to the analytics platform 360. In some cases, the analytics user interface 389 can be configured to enable a user to modify various functions of the analytics platform 360. For example, the analytics user interface 389 can be configured to enable a user to change and / or cancel one or more subscriptions, change and / or cancel one or more contracts, change, add, and / or delete notification thresholds and / or triggers for the analytics automator 384, and / or change any other function of the analytics platform 360.

[0104] As shown in the illustration, the analytics user interface 389 can be remote from the analytics platform 360 (e.g., operating on a different server, within a different network, and / or on a different device). However, in some cases, the analytics user interface 389 can be local to the analytics platform 360. (e.g., operating on the same server, on the same network, and / or on the same device). The data transmitted via the communication link 391 can be encrypted to prevent unauthorized access to the data.

[0105] For example, in some cases, the analysis user interface 389 can be generated and displayed on a web server. As a further example, in some cases, the analysis user interface 389 can be an application that runs on a computer that communicates with the analysis platform 360 through a network connection (e.g., the Internet). As a further example, in some cases, the analysis user interface 389 can be an application that runs locally (e.g., on the same hardware as the analysis platform 360).

[0106] Communication links 330, 353, 362, 366, 370, 374, 383, 387, 391, 398, and / or 399 can utilize the Secure Sockets Layer (SSL) security protocol when establishing communication. As discussed herein, communication links 330, 353, 362, 366, 370, 374, 383, 387, 391, 398, and / or 399 can each carry encrypted data. Accordingly, one or more of communication links 330, 353, 362, 366, 370, 374, 383, 387, 391, 398, and / or 399 can generally be secure communication links (e.g., communication links 330, 353, 362, 366, 370, 374, 383, 387, 391, 398, and / or 399 use at least one of the SSL security protocols and / or the data being transmitted is encrypted).

[0107] The metric / attribute collector 302 and the analysis platform 360 can be implemented in software, firmware, hardware, and / or combinations thereof. For example, the metric / attribute collector 302 and the analysis platform 360 can be stored on one or more memories (e.g., any type of tangible non-transitory storage medium, including any one or more of a magnetic recording medium (e.g., a hard disk drive), an optical disk, a semiconductor device such as a read-only memory (ROM), a random access memory (RAM) such as dynamic and static RAM, an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic or optical card, or any type of storage medium for storing electronic instructions) and implemented as software configured to be executed by one or more processors (e.g., processors commercially available from INTEL, ADVANCED MICRO DEVICES, IBM, ARM, ORACLE and / or any other processors). As a further example, the metric / attribute collector 302 and the analysis platform 360 can be implemented as a circuit (e.g., an application specific integrated circuit). The metric / attribute collector 302 and the analysis platform 360 can be implemented on the same or different machines, servers, and / or networks.

[0108] FIG. 4 shows a schematic embodiment of a conversion unit generator 400 configured to generate a conversion unit discussed in connection with FIG. 3A. The conversion unit generator 400 can be separated from the computing system 104, the device 102, and the network 106. For example, the conversion unit generator 400 can be part of a third-party system so that it can generate a conversion unit and input it as a fixed value within the analysis platform 312. As shown, the conversion unit generator 400 can include at least a first resource conversion unit generator 402 and a second resource conversion unit generator 404. The first resource conversion unit generator 402 can generate, for example, one or more of a CPU conversion unit 314, a memory conversion unit 316, a storage conversion unit 318, a network conversion unit 320, and / or a disk I / O conversion unit 328. The second resource conversion unit generator 404 can generate, for example, one or more of a GPU processor conversion unit 322, a GPU memory conversion unit 324, and / or a GPU memory speed conversion unit 326.

[0109] As shown, the first resource conversion unit generator 402 includes a CPU conversion unit generator 406, a memory conversion unit generator 408, a storage conversion unit generator 410, a network conversion unit generator 412, and a disk I / O conversion unit generator 414.

[0110] The CPU conversion unit generator 406 can generate the CPU conversion unit 314 based at least in part on the CPU allocation 416 and the CPU calculation part 418. For example, the CPU conversion unit 314 can be generated by dividing the CPU allocation 416 by the CPU calculation part 418.

[0111] The CPU allocation 416 corresponds to the clock speed of the CPU 202 accessible to the user. For example, only a portion of the physical processor can be allocated to the workloads of one or more users. In other words, the available clock speed that can be allocated can be lower than the default clock speed of the physical processor. The CPU calculation portion 418 can generally correspond to the weight representing the CPU resources used by one or more of the workloads 110. In some cases, the CPU calculation portion 418 can be an experimentally derived value based on the analysis of the workloads of multiple users. Therefore, in some cases, the CPU calculation portion 418 can generally be an adjustable dynamic weight to account for changes in the workload at a given time.

[0112] The memory translation unit generator 408 can generate the memory translation unit 316 based at least in part on the memory allocation 420 and the memory calculation portion 422. For example, the memory translation unit 316 can be generated by dividing the memory allocation 420 by the memory calculation portion 422.

[0113] The memory allocation 420 corresponds to the amount of memory 204 accessible to the user. For example, only a portion of the physical memory can be allocated to the workloads of one or more users. In other words, the allocated memory can be less than the total available physical memory. The memory calculation portion 422 can generally correspond to the weight representing the memory resources used by one or more of the workloads 110. In some cases, the memory calculation portion 422 can be an experimentally derived value based on the analysis of the workloads of multiple users. Therefore, in some cases, the memory calculation portion 422 can generally be an adjustable dynamic weight to account for changes in the workload at a given time.

[0114] The storage conversion unit generator 410 can generate the storage conversion unit 318 based at least in part on the storage allocation 424 and the storage calculation part 426. For example, the storage conversion unit 318 can be generated by dividing the storage allocation 424 by the storage calculation part 426.

[0115] The storage allocation 424 corresponds to the amount of storage 206 accessible to the user. For example, only a portion of the physical storage can be allocated to the workloads of one or more users. In other words, the allocated storage can be less than the total available physical storage. The storage calculation part 426 can generally correspond to the weights representing the storage resources used by one or more of the workloads 110. In some cases, the storage calculation part 426 can be an experimentally derived value based on the analysis of the workloads of multiple users. Therefore, in some cases, the storage calculation part 426 can generally be a dynamic weight that can be adjusted to take into account the changes in the workload at a given time.

[0116] The network conversion unit generator 412 can generate the network conversion unit 320 based at least in part on the network allocation 428 and the network calculation part 430. For example, the network conversion unit 320 can be generated by dividing the network allocation 428 by the network calculation part 430.

[0117] Network allocation 428 corresponds to the network bandwidth accessible to the user. For example, only a portion of the total physical network bandwidth can be allocated to the workloads of one or more users. In other words, the allocated network bandwidth can be less than the available network bandwidth. The network calculation part 430 can generally correspond to weights representing the network resources (e.g., bandwidth) used by one or more of the workloads 110. In some cases, the network calculation part 430 can be an experimentally derived value based on the analysis of the workloads of multiple users. Therefore, in some cases, the network calculation part 430 can generally be an adjustable dynamic weight to account for changes in the workload at a given time.

[0118] The disk I / O conversion unit generator 414 can generate the disk I / O conversion unit 328 based at least in part on the disk I / O allocation 432 and the disk I / O calculation part 434. For example, the disk I / O conversion unit 328 can be generated by dividing the disk I / O allocation 432 by the disk I / O calculation part 434.

[0119] The disk I / O allocation 432 corresponds to the disk bandwidth accessible to the user (e.g., reading and writing to the storage 206). For example, only a portion of the physical disk bandwidth can be allocated to the workloads of one or more users. In other words, the allocated disk bandwidth can be less than the available total physical disk bandwidth. The disk I / O calculation part 434 can generally correspond to weights representing the disk I / O resources used by one or more of the workloads 110. In some cases, the disk calculation part 434 can be an experimentally derived value based on the analysis of the workloads of multiple users. Therefore, in some cases, the disk calculation part 434 can generally be an adjustable dynamic weight to account for changes in the workload at a given time.

[0120] The sum of the CPU computing part 418, the memory computing part 422, the storage computing part 426, the network computing part 430, and the disk I / O computing part 434 can be 100. In other words, each computing part can be represented as a percentage of the total computing resources evaluated by the first resource conversion unit generator 402 by dividing the computing part by 100.

[0121] As shown in the figure, the second resource conversion unit generator 404 includes a GPU processor conversion unit generator 436, a GPU memory conversion unit generator 438, and a GPU memory speed conversion unit generator 440.

[0122] The GPU processor conversion unit generator 436 can generate a GPU processor conversion unit 322 based at least in part on the GPU processor allocation 442 and the GPU processor computing part 444. For example, the GPU processor conversion unit 322 can be generated by dividing the GPU processor allocation 442 by the GPU processor computing part 444.

[0123] The GPU processor allocation 442 corresponds to the clock speed of the GPU 210 accessible to the user. For example, only a portion of the physical GPU processor can be allocated to the workloads of one or more users. In other words, the available clock speed of the allocated GPU processor can be lower than the default clock speed of the physical GPU processor. The GPU processor computing part 444 can generally correspond to the weight representing the GPU processor resources used by one or more of the workloads 110. In some cases, the GPU processor computing part 444 can be an experimentally derived value based on the analysis of the workloads of multiple users. Therefore, in some cases, the GPU processor computing part 444 can generally be an adjustable dynamic weight to account for changes in the workload over a given time.

[0124] The GPU memory conversion unit generator 438 can generate the GPU memory conversion unit 324 based at least in part on the GPU memory allocation 446 and the GPU memory calculation part 448. For example, the GPU memory conversion unit 324 can be generated by dividing the GPU memory allocation 446 by the GPU memory calculation part 448.

[0125] The GPU memory allocation 446 corresponds to the amount of memory of the GPU 210 accessible to the user. For example, only a portion of the physical memory of the GPU 210 can be allocated to the workloads of one or more users. In other words, the memory of the GPU 210 to be allocated can be less than the total physical memory of the available GPU 210. The GPU memory calculation part 448 can generally correspond to the weight representing the memory resources used by one or more of the workloads 110. In some cases, the GPU memory calculation part 448 can be an experimentally derived value based on the analysis of the workloads of multiple users. Therefore, in some cases, the GPU memory calculation part 448 can generally be an adjustable dynamic weight to account for changes in the workload at a given time.

[0126] The GPU memory speed conversion unit generator 440 can generate the GPU memory speed conversion unit 326 based at least in part on the GPU memory speed allocation 450 and the GPU memory speed calculation part 452. For example, the GPU memory speed conversion unit 326 can be generated by dividing the GPU memory speed allocation 450 by the GPU memory speed calculation part 452.

[0127] The GPU memory speed allocation 450 corresponds to the clock speed of the GPU memory accessible to the user. For example, only a portion of the clock speed of the physical GPU memory can be allocated to the workloads of one or more users. In other words, the available clock speed of the allocated GPU memory can be lower than the default clock speed of the physical GPU memory. The GPU memory speed calculation portion 452 can generally correspond to the weights representing the GPU memory speed resources used by one or more of the workloads 110. In some cases, the GPU memory calculation portion 452 can be an experimentally derived value based on the analysis of the workloads of multiple users. Thus, in some cases, the GPU memory speed calculation portion 452 can generally be an adjustable dynamic weight to account for changes in the workload over a given time.

[0128] The sum of the GPU processor calculation portion 444, the GPU memory calculation portion 448, and the GPU memory speed calculation portion 452 can be 100. In other words, each calculation portion can be expressed as a percentage of the total computing resources evaluated by the second resource conversion unit generator 404 by dividing the calculation portion by 100.

[0129] As described above, the physical computing resources 108 can additionally or alternatively include other resources 209. As will be discussed, the other resources 209 can include any one or more of a field programmable gate array (FPGA), a tensor processing unit (TPU), an intelligence processing unit (IPU), a neural processing unit (NPU), a vision processing unit (VPU), a digital signal processor (DSP), an accelerated processing unit (APU), an application specific integrated circuit (ASIC), a system on chip (SoC), a programmable SoC, an application specific standard product (ASSP), an adaptive computing acceleration platform (ACAP), a microcontroller, and / or any other computing resources. In these cases, each of the resource conversion units can be determined using an independent resource conversion unit generator corresponding to each of the other computing resources 209. For example, the conversion unit generator 400 can include a conversion unit generator 454 for other resources. The conversion unit generator 454 for other resources can be configured to generate one or more conversion units corresponding to each of the other resources 209.

[0130] FIG. 5 shows, for example, a flowchart example of a method 500 for quantifying the usage amount of heterogeneous computing resources usable in the system of FIG. 1 as a single measurement unit. The method 500 can be embodied as one or more instructions in one or more non-transitory computer-readable media (e.g., storage 206) configured to be executed by one or more processors (e.g., CPU 202).

[0131] As shown, method 500 can include step 502. Step 502 can include causing a workload (e.g., workload 110) to execute on a computing system (e.g., computing system 104). In some cases, the workload can be caused to execute in response to receiving a request from, for example, device 102. Additionally or alternatively, the workload can include one or more idle processes resulting from the operation of the computing system.

[0132] Method 500 can also include step 504. Step 504 can include measuring the amount of physical resources used (e.g., consumed) by the execution of the workload over a period of time (e.g., 1 second, 10 seconds, 1 minute, 1 hour, 1 day, 1 week, 1 month, 1 year, and / or any other suitable period). The amount of physical resources can be measured to avoid or otherwise mitigate inaccurate readings used by ballooning techniques available to a hypervisor that measures virtual resource usage. Further, by directly measuring physical resources, the method can be performed in multiple forms of hardware including hardware on which no virtual machines are running. Thus, this method can be implemented in systems that utilize packaged software (e.g., containers).

[0133] Method 500 can include step 506. Step 506 can include normalizing the measured values of the physical resources used. Normalizing the measurements can include converting each of the heterogeneous resources (e.g., CPU resources, memory resources, storage resources, network interface resources, GPU resources, and / or any other computing resources) to a common unit (e.g., as described in connection with FIGS. 3 and 4).

[0134] Method 500 can include step 508. Step 508 can include the step of summing the normalized measurement values. Method 500 can also include step 510. Step 510 can include the step of generating a single value representing the sum of the normalization resources used. For example, the generated single value can be the sum result. In other cases, the generated value can represent the average of the normalization resources summed over a certain time window (e.g., the weekly average where the individual normalized values summed correspond to days of the week).

[0135] FIG. 6 shows, for example, a flowchart example of method 600 for quantifying the maximum or minimum value of heterogeneous computing resources that can be used in the system of FIG. 1 as a single measurement unit. The method can be embodied as one or more instructions in one or more non-transitory computer-readable media (e.g., storage 206) configured to be executed by one or more processors (e.g., CPU 202).

[0136] The method can include step 602. Step 602 can include the step of measuring the maximum amount of available physical resources over a certain period and / or the step of measuring the minimum amount of physical resources for executing an idle process (e.g., idle process 113) over a certain period.

[0137] The method can include step 604. Step 604 can include the step of normalizing the measurement values of the available heterogeneous physical resources (e.g., used and inactive resources). The normalization of the measurement values can include the step of converting each of the heterogeneous resources (e.g., CPU resources, memory resources, storage resources, network interface resources, GPU resources, and / or any other computing resources) into a common unit (e.g., as described in connection with FIGS. 3A-C and 4).

[0138] The method can also include step 606. Step 606 can include the step of summing the normalized measurements of the available physical resources. The method can include step 608. Step 608 can include the step of generating a single value representing the sum of the available normalized resources. For example, the generated single value can be the sum result. As a further example, the generated single value can be an adjusted value based on the sum result.

[0139] In some cases, the maximum and / or minimum amount of available physical computing resources over a period of time can be calculated based at least in part on the specifications corresponding to the individual physical computing resources. Thereby, a user can estimate the maximum and / or minimum amount of available physical computing resources without access to the physical computing resources. In some cases, the calculated value can be compared with the measured value of method 600. For example, by comparing the calculated value with the measured value and adjusting the specifications and / or calculation method corresponding to the individual computing resources, the estimation of the maximum and / or minimum amount of available physical computing resources can be improved.

[0140] According to one aspect of the present disclosure, a computing network is provided. The computing network can include physical computing resources configured to execute at least one workload. The computing network can also include a meter configured to measure the amount of physical computing resources used over a period of time. The meter can generate a single usage value representing the amount of physical resources used over a period of time.

[0141] According to another aspect of the present disclosure, there is provided at least one computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform one or more operations. The one or more operations can include measuring an amount of one or more physical computing resources used over a period of time. The one or more operations can also include normalizing each measured amount of each physical computing resource used. The one or more operations can further include summing the normalized measured amounts of the physical computing resources used. The one or more operations can also include generating a single usage value representative of the physical computing resources used over that period, based on the sum of the normalized physical computing resources.

[0142] According to yet another aspect of the present disclosure, a method is provided. The method can include measuring an amount of one or more physical computing resources used over a period of time. The method can also include normalizing each measured amount of each physical computing resource used. The method can further include summing the normalized measured amounts of the physical computing resources used. The method can also include generating a single usage value representative of the physical computing resources used over that period, based on the sum of the normalized physical computing resources.

[0143] According to yet another aspect of the present disclosure, a computing network is provided. The computing network can include a computing system configured to communicatively couple to devices. The computing system can be configured to execute at least one workload. The computing system can include physical computing resources configured to execute at least one workload. The computing system can also include a meter configured to measure an amount of physical computing resources used over a period of time. The meter can generate a single usage value representative of the amount of physical resources used over that period.

[0144] According to yet another aspect of the present disclosure, a system for measuring heterogeneous computer resources is provided. The meter can include at least one collector. The collector can include at least one metric receiver configured to receive one or more metrics from at least one computing system over a period of time, where the metrics correspond to physical resource usage. The meter can also include at least one analysis platform configured to normalize each of the received metrics, sum the normalized metrics, and generate a single usage value representative of the physical resource usage over that period.

[0145] According to yet another aspect of the present disclosure, a computing network is provided. The computing network can include a computing system having a plurality of physical computing resources configured to execute one or more workloads, and a meter communicatively coupled to the computing system. The meter can be configured to measure the amount of physical computing resources used over a period of time. The meter can include at least one collector including at least one metric receiver configured to receive one or more metrics from the computing system, the metrics corresponding to the amount of physical resources used. The meter can also include at least one analysis platform configured to communicate with the collector to normalize each of the received metrics, sum the normalized metrics, and generate a single usage value representing the physical resource usage over that period.

[0146] According to yet another aspect of the present disclosure, a computing network is provided. The computing network can include a computing system configured to communicatively couple to a device. The computing system can be configured to execute at least one workload. The computing system can include physical computing resources configured to execute at least one workload. The computing system can be configured to communicate with a meter. The meter can be configured to measure the amount of physical computing resources used, and the meter is further configured to generate a single usage value representing the amount of physical resources used.

[0147] According to yet another aspect of the present disclosure, a computing network is provided. The computing network can include a computing system having a plurality of physical computing resources configured to execute one or more workloads. The computing network can also include at least one collector having at least one metric receiver configured to receive one or more metrics from the computing system, where the one or more metrics correspond to the usage of the physical computing resources. The computing network can further include at least one backup collector having a backup receiver configured to receive one or more metrics from the computing system. The computing network can also include at least one analysis platform that communicates with at least one of the metric collector or the backup collector, and the analysis platform can be configured to normalize each of the received metrics, sum the normalized metrics, and generate a single usage value.

[0148] According to yet another aspect of the present disclosure, a system for measuring heterogeneous computer resources is provided. The system can include at least one collector including at least one metric receiver and an account manager. The metric receiver can be configured to receive one or more metrics from at least one computing system, where the metrics correspond to physical resource usage. The account manager can be configured to manage a plurality of accounts. Each of the plurality of accounts can be associated with at least one resource-consuming user, and the physical resource usage of each resource-consuming user is associated with the respective account. The system can also include a usage analyzer configured to generate benchmarks for one or more accounts representing the cost per unit of physical resource usage.

[0149] According to yet another aspect of the present disclosure, a computing network is provided. The computing network can include a computing system configured to communicatively couple to devices. The computing system can also be configured to execute at least one workload. The computing system can include physical computing resources configured to execute at least one workload. The computing network can also include a meter configured to measure the amount of physical computing resources used over a period of time and generate a single usage value representative of the amount of physical resources used over that period. The meter can be further configured to selectively disable access by a device to at least a portion of the computing system.

[0150] According to yet another aspect of the present disclosure, a method is provided. The method can include measuring the amount of a plurality of physical computing resources used over a period of time. The method can also include normalizing each measured amount of each physical computing resource used. The method can also include summing the normalized measured amounts of the physical computing resources used to generate a single usage value representative of the physical computing resources used over that period.

[0151] In some cases, the plurality of physical computing resources can include a plurality of heterogeneous physical computing resources. In some cases, the plurality of physical computing resources can include one or more of a graphics processing system, a field programmable gate array, an application specific integrated circuit, a system on chip, a digital signal processor, a microcontroller, or an adaptive computing acceleration platform. In some cases, the plurality of physical computing resources includes a graphics processing system, and the graphics processing system can include a graphics processor and a graphics memory. In some cases, the method can further include measuring a maximum amount of the plurality of physical computing resources available over a period of time, normalizing each measured maximum amount of each available physical computing resource, and summing the normalized maximum measured amounts of the available physical computing resources to generate a single available resource value representing the maximum amount of the available physical computing resources over that period of time.

[0152] According to yet another aspect of the present disclosure, a computing network is provided. The computing network can include one or more computing systems having a plurality of physical computing resources configured to execute one or more workloads. The computing network can also include one or more metric collectors each having at least one metric receiver configured to receive a plurality of metrics from the one or more computing systems. The plurality of metrics can correspond to usage of the physical computing resources. The computing network can also include one or more analysis platforms configured to normalize each of the received metrics.

[0153] In some cases, one or more analysis platforms can be further configured to sum the normalized metrics to generate a single usage value. In some cases, at least a portion of the physical computing resource usage can be associated with one or more accounts. In some cases, one or more accounts can include multiple accounts, at least one account can be associated with a portion of the physical computing resource usage, and at least one other account can be associated with a different portion of the physical computing resource usage. In some cases, at least one of the metric collectors can further include a plugin database having one or more plugins, and the one or more plugins can be configured such that at least one of the metric collectors can communicate with one or more computing systems. In some cases, the analysis platform can be configured to generate a notification based at least in part on a comparison of the physical computing resource usage with a threshold. In some cases, one or more analysis platforms can include multiple analysis platforms, and at least one of the analysis platforms can be a backup analysis platform. In some cases, one or more analysis platforms can be configured to activate or deactivate at least one of the metric collectors at a predetermined time. In some cases, at least one of the metric collectors can include at least one of one or more analysis platforms. In some cases, one or more metric collectors can include multiple metric collectors, and at least one of the multiple metric collectors can be a backup metric collector. In some cases, the analysis platform can be further configured to determine one or more of the peak usage (e.g., maximum usage), bottom usage (e.g., minimum usage), or average usage corresponding to each physical computing resource over a period of time based at least in part on the metrics received.

[0154] According to yet another aspect of the present disclosure, there is provided at least one computer-readable storage medium storing one or more instructions that, when executed by one or more processors, cause the one or more processors to perform one or more operations. The operations can include measuring an amount of a plurality of physical computing resources used over a period of time. The operations can also include normalizing each measured amount of each physical computing resource used. The operations can also include summing the normalized measured amounts of the physical computing resources used to generate a single usage value representing the physical computing resources used over that period of time.

[0155] In some cases, the plurality of physical computing resources can include a plurality of heterogeneous physical computing resources. In some cases, the plurality of physical computing resources can include one or more of a graphics processing system, a field programmable gate array, an application specific integrated circuit, a system on chip, a digital signal processor, a microcontroller, or an adaptive computing acceleration platform. In some cases, the plurality of physical computing resources includes a graphics processing system, and the graphics processing system can include a graphics processor and a graphics memory. In some cases, the operation can further include measuring the maximum amount of the plurality of physical computing resources available over a period of time, normalizing each measured maximum amount of each available physical computing resource, and summing the normalized maximum measured amounts of each available physical computing resource to generate a single available resource value representing the maximum amount of physical computing resources available over that period. In some cases, the operation can also include accessing a plug-in database, and the plug-in database can include one or more plug-ins configured to enable measurement of the plurality of physical computing resources. In some cases, the operation can further include determining one or more of a peak usage amount (e.g., maximum usage amount), a bottom value usage amount (e.g., minimum usage amount), or an average usage amount corresponding to each physical computing resource over a period of time.

[0156] According to yet another aspect of the present disclosure, a method is provided. The method can include measuring the amount of graphics processing system resources used over a period of time, where the graphics processing system resources can include a graphics processor and graphics memory. The method can also include normalizing the measured amounts corresponding to the graphics processor and the graphics memory. The method can also include summing the normalized measured amounts to generate a single usage value representative of the graphics processing system resources used over that period.

[0157] Although several embodiments of the present disclosure have been described and illustrated herein, those skilled in the art will readily envision various other means and / or structures for performing the functions and / or obtaining the results and / or advantages described herein, and each of such variations and / or modifications is to be regarded as being within the scope of the present disclosure. More generally, those skilled in the art will readily understand that all parameters, dimensions, materials, and configurations described herein are meant to be illustrative and that the actual parameters, dimensions, materials, and / or configurations will depend upon the one or more specific applications for which the teachings of the present disclosure are used.

[0158] Those skilled in the art will recognize, or be able to ascertain using no more than routine experimentation, many equivalents to the specific embodiments described herein. Accordingly, the foregoing embodiments are presented by way of example only, and within the scope of the appended claims and their equivalents, the present disclosure may be practiced otherwise than as specifically described and claimed. The present disclosure is directed to each individual feature, system, article, material, kit, and / or method described herein. In addition, where such features, systems, articles, materials, kits, and / or methods do not mutually conflict, any combination of two or more of such features, systems, articles, materials, kits, and / or methods is included within the scope of the present disclosure.

[0159] As used in this specification and the claims, the indefinite articles "a" and "an" should be understood to mean "at least one" unless explicitly indicated otherwise.

[0160] As used herein, the terms "couple" and "coupled" include both direct and indirect couplings unless explicitly indicated otherwise.

[0161] As used in this specification and the claims, the phrase "and / or" should be understood to mean "either or both" of the elements so combined, i.e., the elements may be present consecutively in some cases and separately in other cases. Other elements, whether related or unrelated to those specifically identified, may optionally be present unless explicitly indicated otherwise for those specifically identified elements by the "and / or" clause.

Description of Reference Numerals

[0162] 100 Computing network 102 Device 104 Computing system 106 Network 108 Computing resource 110 Workload 112 Operating system 113 Idle process 114 Application 116 Display 118 Measuring instrument

Claims

**Claim 1** executing one or more workloads of a plurality of graphics processing systems (GPS); for each GPS of the plurality of GPSs, receiving a first graphics attribute and a second graphics attribute of graphics resources associated with each respective GPS, wherein the first graphics attribute and the second graphics attribute each include one or more of a default graphics processor clock speed, a variable graphics processor clock speed, a graphics processor load, a graphics memory default clock speed, a variable graphics memory clock speed, a graphics memory load, an available graphics memory, a used graphics memory, a shader, and a graphics memory bus; for each respective GPS, receiving a calculation portion associated with the graphics resources; for each respective GPS, receiving an allocation associated with the graphics resources; calculating a utilization by dividing a value associated with the first graphics attribute by 100 to calculate a quotient and multiplying the quotient by a value associated with the second graphics attribute; generating a conversion unit by dividing the allocation by the respective calculation portion; normalizing the utilization by dividing the utilization by the conversion unit to generate a normalized usage; thereby determining a normalized usage of each respective graphics resource; generating a single usage value of the plurality of GPSs by summing the normalized usages of each graphics resource; thereby determining a single usage value of the plurality of GPSs; A method comprising the above steps. **Claim 2** Each graphics resource includes one or more of a graphics memory resource and a graphics processor resource. The method according to claim 1. **Claim 3** The calculation portion includes one or more of a graphics processor calculation portion and a graphics memory calculation portion. The allocation includes one or more of a graphics processor allocation and a graphics memory allocation. The method according to claim 1. **Claim 4** executing one or more workloads of a plurality of graphics processing systems (GPSs); for each respective GPS of the plurality of GPSs to which graphics resources including graphics processor resources and graphics memory resources are respectively associated, (i) receiving a first graphics processor attribute and a second graphics processor attribute associated with the graphics processor resources, and (ii) receiving a graphics memory attribute associated with the graphics memory resources; for each GPS, receiving a calculation part including (i) a calculation part associated with the graphics processor resources and (ii) a calculation part associated with the graphics memory resources; for each GPS, receiving an allocation including (i) an allocation associated with the graphics processor resources and (ii) an allocation associated with the graphics memory resources; calculating a utilization by dividing a value associated with the first graphics processor attribute by 100 and multiplying the quotient by a value associated with the second graphics processor attribute; generating a conversion unit based on dividing the allocation by a calculation part of a specific graphics resource; normalizing the utilization by dividing the utilization by the conversion unit to generate a normalized usage amount; thereby determining a normalized usage amount for each respective GPS; generating a single usage amount value for the plurality of GPSs by summing the normalized usage amounts; thereby determining a single usage amount value for the plurality of GPSs; including, for each GPS, the maximum sum of (i) the calculation part associated with the graphics processor resources and (ii) the calculation part associated with the graphics memory resources being equal to 1; A method characterized by the above. **Claim 5** for each GPS, receiving a respective graphics memory speed attribute; for each respective GPS, receiving a calculation part associated with the graphics memory speed attribute; for each respective GPS, receiving an allocation associated with the graphics memory speed attribute; calculating a graphics memory speed utilization based on the graphics memory speed attribute; Based on the calculation part related to the graphics memory speed attribute and the assignment related to the graphics memory speed attribute of each respective GPS, generate a graphics memory speed conversion unit, Normalize the graphics memory speed utilization by dividing the graphics memory speed utilization by the graphics memory speed conversion unit to generate a normalized graphics memory speed usage, Thereby, for each respective GPS, determining a normalized usage amount of the graphics memory speed attribute of each respective GPS; The method according to claim 4, further comprising.

6. The graphics memory speed attribute includes one or more of a graphics memory default clock speed, a variable graphics memory clock speed, and a graphics memory load. The method according to claim 5.

7. The graphics processor attribute includes one or more of a default graphics processor clock speed, a variable graphics processor clock speed, and a graphics processor load, The graphics memory attribute includes one or more of available graphics memory and used graphics memory. The method according to claim 4.

8. Each assignment is an amount of graphics processor resources or graphics memory resources accessible to the user. The method according to claim 4.

9. Each respective calculation part is a dynamic weight adapted to consider changes in the workload. The method according to claim 4.

10. A system comprising at least one computer-readable storage medium storing one or more instructions that, when executed by one or more processors, cause the one or more processors to perform one or more operations, the operations comprising: Executing one or more workloads of a plurality of graphics processing systems (GPSs), Receiving, for each respective GPS of the plurality of GPSs, each associated with graphics resources, a graphics attribute including a first graphics processor attribute, a second graphics processor attribute, and a graphics memory attribute. For each respective GPS, (i) one or more computational components that adapt with dynamic weights so as to each consider changes in workload, and (ii) receive one or more allocations, Calculate a quotient by dividing the value associated with the first graphics processor attribute by 100, and calculate a utilization based on multiplying the quotient by the value associated with the second graphics processor attribute, Generate a conversion unit based on dividing the one or more allocations by the one or more computational components, Thereby determining a normalized usage amount of each respective graphics resource, Thereby determining a single usage value of the plurality of GPSs, Generate a normalized usage amount by dividing the utilization by the conversion unit, Generate the single usage value by summing the normalized usage amounts of each respective graphics resource, Is configured to, A system characterized by this.

11. The maximum total of the one or more computational components is equal to 1, The system according to claim 10.

12. The first and second graphics processor attributes include one or more of a default graphics processor clock speed, a variable graphics processor clock speed, and a graphics processor load, The graphics memory attributes include one or more of an available graphics memory and a used graphics memory, The system according to claim 10.

13. Each allocation of the one or more allocations is an amount of resources of each respective GPS accessible by a user, The system according to claim 10.

14. Executing the workload of a graphics processing system (GPS), Receiving from the GPS in which graphics processor resources and graphics memory resources are associated, (i) a first graphics processor attribute and a second graphics processor attribute associated with the graphics processor resources, and (ii) a graphics memory attribute associated with the graphics memory resources, Receiving from the GPS, (i) a computational component associated with the graphics processor resources of the GPS, and (ii) a computational component associated with the graphics memory resources of the GPS, Receiving, from the GPS, (i) an allocation related to the graphics processor resources of the GPS, and (ii) an allocation related to the graphics memory resources of the GPS; Calculating a quotient by dividing the value related to the first graphics processor attribute by 100, and calculating the utilization of the graphics processor resources based on multiplying the quotient by the value related to the second graphics processor attribute; Generating a conversion unit for the graphics processor resources based on the calculated portion and the allocation of the graphics processor resources; Normalizing the utilization of the graphics processor resources by dividing the utilization by the conversion unit to generate a normalized graphics processor resource usage; Thereby determining the normalized usage of the graphics processor resources of the GPS; Calculating a quotient by dividing the value related to the first graphics processor attribute by 100, and calculating the utilization of the graphics memory resources based on multiplying the quotient by the value related to the second graphics processor attribute; Generating a conversion unit for the graphics memory resources based on dividing the allocation by the calculated portion of the graphics memory resources; Normalizing the utilization of the graphics memory resources by dividing the utilization by the conversion unit to generate a normalized graphics memory resource usage; Thereby determining the normalized usage of the graphics memory resources of the GPS; Generating a single usage value for the GPS by summing the normalized graphics processor resource usage and the normalized graphics memory resource usage; Thereby determining a single usage value for the plurality of GPSs; Including, wherein the maximum sum of the calculated portion related to the graphics processor resources of the GPS and the calculated portion related to the graphics memory resources of the GPS is equal to 1; A method characterized by this.

15. The one or more graphics attributes include one or more of a default graphics processor clock speed, a variable graphics processor clock speed, a graphics processor load, available graphics memory, used graphics memory, a shader, or a graphics memory bus. The method according to claim 14.

16. Each allocation is an amount of the graphics processor resources or the graphics memory resources accessible to the user. The method according to claim 14.

17. Receiving a graphics memory speed attribute from the GPS; Receiving a calculation part related to the graphics memory speed attribute from the GPS; Receiving an allocation related to the graphics memory speed attribute from the GPS; Calculating a graphics memory speed utilization based on the graphics memory speed attribute; Generating a graphics memory speed conversion unit based on the calculation part and the allocation related to the graphics memory speed attribute; Normalizing the graphics memory speed utilization by dividing the graphics memory speed utilization by the graphics memory speed conversion unit to generate a normalized graphics memory speed usage; Thereby, determining a normalized usage amount of the graphics memory speed attribute; The method according to claim 14, further comprising.

18. A system comprising at least one computer-readable storage medium storing one or more instructions that, when executed by one or more processors, cause the one or more processors to perform one or more operations, the operations comprising: Executing a workload of a graphics processing system (GPS); Receiving, from the GPS to which graphics resources including (i) graphics processor resources and (ii) graphics memory resources are associated, (i) graphics processor attributes including first and second graphics processor attributes related to the graphics processor resources, and (ii) graphics memory attributes related to the graphics memory resources; Receive from the GPS: (i) a calculation part related to the graphics processor resources of the GPS, and (ii) a calculation part related to the graphics memory resources of the GPS. Receive from the GPS: (i) an allocation related to the graphics processor resources, and (ii) an allocation related to the graphics memory resources. Calculate a utilization by dividing the value related to the first graphics processor attribute by 100 and multiplying the quotient by the value related to the second graphics processor attribute. Generate a conversion unit based on dividing the allocation by each respective calculation part. Normalize the utilization of each respective graphics resource by dividing the utilization by the conversion unit to generate a normalized usage amount. Thereby determine the normalized usage amount of each graphics resource. Generate a single usage amount value of the GPS by summing the normalized usage amounts. Thereby determine the single usage amount value of the GPS. Be configured such that the maximum sum of the calculation part related to the graphics processor resources of the GPS and the calculation part related to the graphics memory resources of the GPS is equal to 1. A system characterized by this.

19. The step of executing one or more workloads of the plurality of GPSs provides at least one graphics attribute related to each respective GPS. The method according to claim 1.

Images