Cloud Application Performance Validation via Neighbor Filtering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In cloud-based software applications, it is challenging to isolate and differentiate performance degradation caused by neighbor-based variability from other factors during performance evaluations, leading to inefficient resource allocation and false alarms in performance analysis.
Innovation Solution
A method that involves receiving historical run data and monitoring data from current test runs, determining a subset of similar historical runs, and validating performance degradation by comparing these with a baseline, thereby reducing the impact of neighbor-based variability on performance evaluation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If performance testing is conducted in cloud-based multitenant environments, then performance evaluation can be performed, but neighbor-based variability causes false alarms and reduces measurement precision
Solution Approach 1:
The patent segments the performance evaluation process into multiple independent runs and compares results across these segments. By dividing the evaluation into discrete test runs with multiple measurements, the system can identify and filter out neighbor-based variability through statistical comparison, thereby improving measurement precision in multitenant cloud environments.
Solution Approach 2:
The patent performs preliminary baseline measurements and establishes performance thresholds before conducting the actual performance evaluation. By pre-characterizing the system's performance under various conditions and setting acceptable variation ranges, the system can distinguish between normal neighbor-based variability and genuine performance degradation, reducing false alarms.
2Reliability
If detailed performance analysis is performed on all test runs, then performance issues can be detected, but resource allocation becomes inefficient due to false alarms
Solution Approach 1:
The patent applies partial action by performing detailed performance analysis only on test runs that exceed predetermined thresholds or show significant deviations from baseline performance. Instead of analyzing every test run in detail, the system filters results and applies comprehensive analysis only when necessary, thereby maintaining reliable detection of genuine performance degradation while improving resource allocation efficiency.
3Productivity
If performance testing is performed in shared cloud environments, then resource utilization is improved, but performance evaluation reliability decreases due to multitenancy effects
Solution Approach 1:
The patent segments performance measurements into multiple independent runs and uses statistical comparison to isolate multitenancy effects. By dividing the evaluation into discrete segments and analyzing variations between them, the system can distinguish between performance changes caused by multitenancy and genuine performance degradation, maintaining evaluation reliability while benefiting from shared resource utilization.
Solution Approach 2:
The patent implements feedback mechanisms by continuously monitoring performance metrics and comparing them against baseline values and predetermined thresholds. The system uses this feedback to dynamically adjust evaluation criteria and trigger detailed analysis only when genuine performance degradation is detected, thereby maintaining reliable performance evaluation in shared cloud environments while optimizing resource utilization.
Data Source
AI summary
Some embodiments of the present invention are directed towards techniques for validating performance degradation of cloud deployed application from neighbor based variability. Historical runs of an application deployed in a cloud environment are received. In these embodiments, a subset of these historical runs, using associated performance metrics recorded during the historical runs, are compared against performance metrics of a current version of the application which is deployed in a cloud environment to determine a subset of historical runs similar to the current version. The determined subset is then used to draw comparisons with performance metrics of a baseline run of the application to validate if a performance degradation has occurred by updating the application to the current version, reducing the impact of neighbor-based variability on evaluating performance degradation. Detailed performance analysis resources are less likely to be expended on performance degradation caused by neighbor activities in the cloud environment.