Automated Test Server Calibration for Software Performance Bottleneck Detection

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Performance testing of computer software applications is time-consuming and often misidentifies bottlenecks due to test servers running out of resources, leading to false positives or underutilization of equipment.

Innovation Solution

Automated methods and systems determine the optimal scale of a performance test server by calculating the maximum number of virtual users for linear scalability, adjusting test regimes, and calibrating test servers to maintain throughput, thereby identifying the correct load and resource requirements.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If performance testing is performed with high virtual user counts to identify bottlenecks quickly, then testing speed improves, but test servers run out of resources causing false positives

Engineering Contradiction:

Improvetesting speedVSAvoidbottleneck identification accuracy

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The system performs preliminary calibration testing with increasing virtual user counts to establish the maximum sustainable count before executing the actual performance test. This preliminary action prevents resource exhaustion during the main test by ensuring the virtual user count is set at an appropriate level that won't cause test server failures.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system monitors test server resource utilization during calibration testing and uses this feedback to determine the maximum sustainable virtual user count. This feedback mechanism ensures that subsequent performance tests are configured with appropriate virtual user counts that maintain reliability while achieving testing objectives.

Inventive Principle:

Principle #23Feedback

2Reliability

If performance testing is performed with low virtual user counts to maintain system stability, then resource exhaustion is avoided, but testing accuracy and bottleneck detection capability deteriorate

Engineering Contradiction:

Improvetest server stabilityVSAvoidbottleneck detection accuracy

Core Design Contradiction:

ReliabilityVSMeasurement precision

Solution Approach 1:

The system performs preliminary calibration testing with increasing virtual user counts to establish the maximum sustainable count before executing the actual performance test. This preliminary action ensures that the main test uses the highest possible virtual user count that maintains stability, thereby achieving both reliability and measurement precision.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system dynamically determines the optimal virtual user count parameter based on calibration test results. By changing this parameter to the maximum sustainable value identified during calibration, the system achieves both test server stability and sufficient load for accurate bottleneck detection.

Inventive Principle:

Principle #35Parameter changes

3Measurement precision

If manual calibration and test configuration is performed to optimize test server utilization, then testing accuracy improves, but time consumption increases

Engineering Contradiction:

Improvetest configuration accuracyVSAvoidcalibration time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The system automatically performs calibration testing and determines the maximum sustainable virtual user count without requiring manual intervention. This self-service approach eliminates time-consuming manual calibration while achieving accurate test configuration through automated resource monitoring and analysis.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system performs automated preliminary calibration testing to establish test parameters before the main performance test. This automated preliminary action replaces manual calibration processes, reducing time consumption while maintaining or improving configuration accuracy through systematic resource monitoring.

Inventive Principle:

Principle #10Preliminary action

4Measurement precision

If test servers are over-provisioned to handle maximum load, then bottleneck detection capability improves, but resource utilization efficiency deteriorates

Engineering Contradiction:

Improvebottleneck detection capabilityVSAvoidresource utilization efficiency

Core Design Contradiction:

Measurement precisionVSLoss of energy

Solution Approach 1:

The system dynamically determines the optimal virtual user count parameter based on actual test server capacity and calibration results. This parameter optimization ensures that tests use the minimum necessary virtual user count to achieve bottleneck detection, avoiding over-provisioning and improving resource utilization efficiency.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS10120782B2Methods, devices, systems, and non-transitory machine-readable medium for performing an automated calibration for testing of a computer software application

Publication Date: 2018.11.06 APPVANCE

AI summary

The present invention enables an automated testing of computer software applications for efficiently determining the quality and/or performance characteristics of the computer software applications and assists testing designers when determining software application scalability and performance under load. Embodiments of the present invention may be implemented to, for example, determine how many test servers are required to test computer software applications for correct function under the load of many concurrently active users, and periodically test and/or monitor computer software applications for quality control and/or other purposes. Additionally, embodiments of the present invention may be implemented to, for example calibrate a set of one or more test servers for testing a computer software application.