Performance Test Timing Compensation for Remote Cluster Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Performance regression testing in server clusters is hindered by network latency, leading to inaccurate determination of execution times and invalid test conclusions.
Innovation Solution
A method involving cluster algorithms to group network latency and performance test execution times, followed by a regression model to predict and compensate for network latency impacts, ensuring accurate execution time assessment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If performance tests are run in a remote cluster environment, then testing scalability and distributed system performance is improved, but network latency causes inaccurate execution time measurement
Solution Approach 1:
The patent introduces a mediator system that collects execution times from both remote and local cluster environments, processes this data through cluster algorithms and regression models, and generates compensated execution times that account for network latency. This intermediary processing layer resolves the measurement accuracy problem while preserving the ability to test in remote environments.
Solution Approach 2:
The patent transforms the raw execution time parameter by applying cluster algorithms and regression models to compensate for network latency effects. By changing the parameter from raw execution time to compensated execution time, the system maintains the ability to measure remote cluster performance while eliminating the measurement error introduced by network latency.
2Measurement precision
If network latency is compensated using simple methods, then measurement accuracy is improved, but the complexity of data processing increases
Solution Approach 1:
The patent segments the data processing into distinct stages: collecting execution times from multiple environments, grouping data into clustered groups using cluster algorithms, building regression models from these clusters, and applying compensation. This segmentation makes the complex processing manageable and systematic while achieving high measurement accuracy.
Solution Approach 2:
The patent performs preliminary actions by collecting execution time data from both remote and local cluster environments before actual performance testing, and by pre-building regression models from clustered data. This preliminary data collection and model building enables accurate compensation during actual testing without adding complexity to the test execution process.
3Reliability
If multiple cluster environments are used for testing, then test reliability is improved, but the time and resources required for testing increase
Solution Approach 1:
The patent uses a local cluster environment as a copy or reference model of the remote cluster environment. By running tests in both environments and using the local environment's execution times as a baseline, the system can compensate for network latency effects and derive accurate remote cluster performance metrics without requiring extensive direct remote testing.
Solution Approach 2:
The patent implements a feedback mechanism where execution times from the local cluster environment are used to calculate compensation values that are applied to remote cluster execution times. This feedback loop enables the system to continuously improve measurement accuracy by comparing expected local performance with actual remote performance and adjusting accordingly.
Data Source
AI summary
A computer-implemented method, system and computer program product for accurately identifying an execution time of a performance test. Network latency data is grouped into clustered groups of network latency data. Furthermore, the performance test execution times for the same group of performance tests run in the local and remote cluster environments are obtained. The test execution times impacted by network latency (compensation times) are then determined based on such obtained performance test execution times in the local and remote cluster environments. Such compensation times are then grouped into clustered groups of compensation times. A regression model is built to predict a performance test execution time impacted by network latency (compensation time) using the clustered groups of network latency data and compensation times. The execution time of a performance test run in the remote cluster environment is then generated that takes into consideration the compensation time predicted by the regression model.