Server Benchmarking Using Production Workload Traces
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current benchmarking techniques do not effectively use real-world data, leading to inaccurate performance assessments of computer systems, are inefficient, and require physical access to production servers, making it difficult to generate and interpret performance results, especially when comparing different systems.
Innovation Solution
A benchmarking process that collects real-world data from production servers handling live traffic and generates a benchmark program to simulate the workload on candidate servers, allowing for automated trace collection, workload data conversion, and centralized storage of performance results, eliminating the need for physical access and facilitating accurate comparisons.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If real-world data is used to benchmark systems, then measurement precision is improved, but device complexity increases due to requirements for physical access to production servers
Solution Approach 1:
The patent creates a copy of the production server's workload characteristics by capturing trace data and converting it into benchmark programs. Instead of requiring physical access to the production server for each benchmarking operation, the system creates reproducible benchmark copies that simulate the real workload, thereby improving measurement precision while reducing operational complexity.
Solution Approach 2:
The patent performs preliminary actions by capturing and storing trace data from production servers in advance. This trace data is then converted into benchmark programs that can be executed repeatedly without requiring further access to the production server. This preliminary data collection and conversion process eliminates the need for repeated physical access while maintaining accurate performance measurements.
2Measurement precision
If physical access to production servers is obtained to collect real-world data, then measurement precision is improved, but loss of time increases due to delays in obtaining production data
Solution Approach 1:
The system performs preliminary data collection by capturing trace data from production servers and storing it for future use. This pre-captured data can be converted into benchmark programs at any time without requiring further access to the production server, thereby eliminating delays while maintaining measurement accuracy.
Solution Approach 2:
The system creates copies of the production workload through trace data and benchmark programs. These copies can be generated and executed immediately without requiring time-consuming physical access to the production server, thus reducing time loss while preserving the accuracy benefits of real-world data.
3Adaptability or versatility
If benchmarking is performed repeatedly for various production data at different times, then adaptability is improved, but productivity decreases due to the impossibility of obtaining production data
Solution Approach 1:
The system creates reusable benchmark program copies from captured trace data. These benchmark programs can be executed repeatedly for different scenarios and times without requiring re-access to the production server, thereby maintaining adaptability while significantly improving productivity through automated, on-demand benchmarking.
Solution Approach 2:
The system enables self-service benchmarking by storing trace data and automatically converting it into executable benchmark programs. This automation eliminates the need for manual intervention to obtain production data, allowing benchmarks to be generated repeatedly and efficiently without human effort, thus improving productivity while maintaining versatility.
4Ease of manufacture
If performance results are generated in current formats, then ease of manufacture is maintained, but ease of operation deteriorates due to difficulty in interpreting results and making comparisons
Solution Approach 1:
The system creates standardized copies of performance results in a uniform format that facilitates easy comparison across different systems and benchmarks. This standardized result format maintains the simplicity of generating benchmarks while dramatically improving the ease of interpreting and comparing performance data across various scenarios.
Data Source
AI summary
The disclosure is directed to benchmarking a server computer (“server”), e.g., a storage system of the server. The process can benchmark a candidate server using data from a production server that is handling live traffic of an application, e.g., a social networking application. The process includes collecting, from the production server, production data that is indicative of a workload of the production server, and generating a benchmark program using the production data. Upon execution of the benchmark program, the candidate server is made to process a workload generated based on the production data. The benchmark program records the performance of the candidate server, which is indicative of a performance of the candidate server in processing a workload similar to the production data. The performance results can be used in determining whether to upgrade the configuration of the production server to that of the candidate server, e.g., to improve performance.