Cloud Benchmarking via Centralized Controller and Isolated Metrics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud computing customers face challenges in verifying and comparing the performance and infrastructure services offered by different cloud providers due to a lack of transparency and standardized measurement units, leading to difficulties in evaluating hardware configurations, software configurations, and infrastructure services.
Innovation Solution
A system and method for benchmarking cloud computing instances that selects and measures indicative metrics across various categories (compute, memory, disk, operating system, network, database, and cloud provider) to enable direct comparisons and statistical analysis, using a central controller to manage benchmarking applications and ensure accurate, trustworthy, and secure data collection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If cloud providers use proprietary unitary measures to describe computing capacity, then vendor-specific service delivery is simplified, but customer ability to verify and compare claims across different providers is degraded
Solution Approach 1:
The patent introduces a third-party benchmarking service as an intermediary between cloud providers and customers. This independent entity performs standardized benchmarking tests and provides objective verification of computing capacity claims, resolving the contradiction by maintaining vendor-specific service simplicity while enabling customer verification through an impartial mediator.
Solution Approach 2:
The patent establishes universal benchmarking standards and methodologies that can be applied across different cloud providers and service types. This universal approach allows the same verification process to work for various vendors and service configurations, enabling cross-provider comparison while maintaining ease of operation for each individual provider.
2Loss of information
If cloud providers disclose detailed hardware and software configurations, then transparency is improved, but complexity of evaluation for customers increases
Solution Approach 1:
The patent extracts the essential performance characteristics from complex hardware and software configurations by implementing standardized benchmarking tests. Instead of requiring customers to evaluate detailed configuration specifications, the system extracts measurable performance metrics through controlled benchmarking procedures, reducing evaluation complexity while maintaining transparency.
Solution Approach 2:
The patent transforms complex configuration parameters into standardized performance metrics through benchmarking. By changing the representation from detailed hardware/software specifications to normalized performance measurements, the system maintains full transparency of the underlying configurations while simplifying the evaluation process for customers.
3Adaptability or versatility
If cloud providers use different infrastructure services and configurations, then service differentiation and competitiveness are improved, but ability to compare providers objectively is degraded
Solution Approach 1:
The patent segments the evaluation process into distinct benchmarking categories and tests that can independently measure different aspects of cloud service performance. This segmentation allows providers to differentiate their services across multiple dimensions while enabling customers to compare specific performance attributes objectively without being overwhelmed by overall system complexity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Cloud computing benchmarking is performed wherein the resource usage of a measuring benchmarking application is compensated for as to not impact measurement. The measurements are of a cloud instance's benchmarking indicia which may include performance, functions and characteristics of the cloud instance. The benchmarking indicia use scalable measures as to allow the use of arithmetic operations such as those used in statistical functions. The benchmarking application is dispatched along with a configuration file and is controlled from a central controller to specified cloud instances. The dispatched benchmarking application takes measurements of the cloud instance based on the configuration file. The benchmarking application then stores the measurements in a results file for return back to the central controller. At the central controller, results files from one or more benchmarking applications are stored in a data store for comparative and statistical analysis.