Performance Testing in Continuous Deployment Pipelines
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Continuous deployment pipelines typically lack performance and load tests due to time and cost constraints, resulting in incomplete quality guarantees for code changes as they do not measure performance under testing conditions.
Innovation Solution
A method is introduced to perform performance tests on applications within continuous deployment pipelines by identifying code changes between baseline and new builds, executing customized test scripts on both, and determining performance values by comparing test results, enabling easier identification of performance issues.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If performance and load tests are included in continuous deployment pipelines, then quality assurance is improved, but time and cost increase
Solution Approach 1:
The patent segments the testing process by dividing test scripts into different types (performance tests, load tests, functional tests) and selectively executing only relevant test subsets based on code change analysis. This segmentation allows performance testing to be included without executing the entire test suite, thereby improving quality assurance while controlling time consumption.
Solution Approach 2:
The patent implements partial action by executing only a subset of test scripts rather than the complete test suite. The system identifies code changes and selectively runs tests relevant to those changes, performing partial testing that is sufficient for quality assurance without the excessive time cost of comprehensive testing.
2Reliability
If performance tests are executed on every code change, then performance quality is improved, but productivity decreases
Solution Approach 1:
The patent applies preliminary action by analyzing code changes before executing performance tests. The system identifies what code has changed and pre-determines which test scripts are relevant, allowing performance testing to be targeted and efficient rather than exhaustive, thus maintaining performance quality without sacrificing deployment productivity.
Solution Approach 2:
The patent implements local quality by applying performance testing selectively based on local code changes rather than uniformly to all code. The system identifies specific modules or functions that have changed and executes performance tests targeted to those areas, improving performance quality where needed while maintaining overall productivity.
3Measurement precision
If comprehensive test suites are run, then measurement precision is improved, but loss of time increases
Solution Approach 1:
The patent uses partial action to execute only the necessary subset of test scripts required to achieve adequate measurement precision. By analyzing code changes and selecting relevant tests, the system obtains sufficient performance measurement accuracy without the time cost of running comprehensive test suites.
Solution Approach 2:
The patent segments the comprehensive test suite into individual test scripts that can be selectively executed. This segmentation allows the system to run only the specific tests needed for accurate performance measurement of changed code, maintaining measurement precision while reducing overall test execution time.
Data Source
AI summary
A method to perform performance tests on an application in a continuous deployment pipeline is provided herein. The method identifies code changes are two distinct builds in a performance test environment. The method obtains a baseline test result by executing a set of customized test scripts on a baseline build with a first code base. The method similarly tests the new build by executing the set of customized test scripts on the new build with a second code base to obtain a new test result. Performance values are determined by comparing the baseline test result and the new test result.