Prioritized Performance Test Workloads for Release Risk Profiles
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed software development, independent teams often introduce defects when updating their respective components, leading to inefficiencies in performance testing due to limited time before software releases, resulting in potential performance defects going undetected.
Innovation Solution
A method and system for recommending prioritized performance test workloads by creating a risk profile based on keyword searches from external resources, adjusting test workloads and variability dimensions to focus on high-risk areas, thereby increasing the likelihood of defect detection within the limited testing time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If performance tests are executed in multiple test scenarios to cover distributed development teams' updates, then the reliability of software release is improved, but the time required for testing increases
Solution Approach 1:
The patent segments the test workload into prioritized groups based on risk profiles associated with different development teams and their updates. By dividing the comprehensive test suite into high-risk and low-risk segments, the system enables focused testing on critical areas first, ensuring software release reliability while managing testing time through strategic prioritization rather than exhaustive testing of all scenarios equally
Solution Approach 2:
The patent performs preliminary analysis to create risk profiles before executing tests. By pre-identifying high-risk areas through static analysis, code change tracking, and team update monitoring, the system prepares prioritized test workloads in advance. This preliminary action allows testing resources to be immediately directed to critical areas when testing begins, resolving the contradiction between comprehensive reliability testing and time constraints
2Manufacturing precision
If comprehensive performance tests are conducted across all variability dimensions, then manufacturing precision of software quality is improved, but productivity of the testing process deteriorates
Solution Approach 1:
The patent applies local quality by assigning different testing depths and intensities to different variability dimensions based on their risk profiles. High-risk teams and code areas receive comprehensive, precision-focused testing, while low-risk areas receive streamlined testing. This localized approach to test quality ensures manufacturing precision is maintained for critical components while improving overall testing productivity through differentiated testing strategies
Solution Approach 2:
The patent dynamically changes testing parameters such as test depth, coverage scope, and resource allocation based on risk profile parameters. By adjusting these parameters according to the prioritized test workload generated from risk analysis, the system achieves high software quality precision in critical areas while maintaining high testing productivity through optimized parameter configuration across different test scenarios
Data Source
AI summary
Aspects of the present disclosure describe methods and systems for recommending prioritized performance test workloads. An example method generally includes searching one or more external resources using a keyword from a number of keywords associated with a baseline test workload for a software release. The method further includes creating a risk profile for the software release based, at least in part, on a number of matches found in search results resulting from the searching. In addition, the method includes generating a prioritized test workload for execution over one or more prioritized variability dimensions based on the risk profile and the baseline test workload. The method also includes executing a test of the software release based on the prioritized test workload.