Cloud Regression Test Suite Reduction via Parameter Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for regression test suite reduction in cloud systems are limited by assuming only two configured instances and equal environment agnostic configuration parameters, failing to account for the impact on dependent services and resource-sharing, and do not minimize testing impact on production traffic effectively.
Innovation Solution
A method that classifies configuration parameters into environment-dependent and environment-agnostic categories, using relationships between parameters, requirements, and service instances to select and summarize test cases into a configuration fault model, applying test suite reduction rules to validate cloud system configurations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all test cases from the acceptance test suite are retested after reconfiguration, then fault detection coverage is maximized, but testing time and resource consumption increase significantly
Solution Approach 1:
The patent extracts only the necessary test cases from the complete acceptance test suite by analyzing the relationship between reconfiguration changes and test case applicability. It identifies and removes test cases that are not affected by the reconfiguration, keeping only those that need to be retested to detect configuration faults.
Solution Approach 2:
The patent segments the acceptance test suite into multiple subsets based on the classification of configuration parameters (environment-dependent and environment-agnostic). This segmentation allows selective retesting of only those test cases relevant to the specific reconfiguration changes made.
2Reliability
If regression testing is performed in the production environment, then real-world fault detection is improved, but normal traffic is disrupted
Solution Approach 1:
Instead of running the complete test suite in production, the patent applies partial action by executing only the reduced subset of necessary test cases. This minimizes the impact on normal traffic while still achieving adequate fault detection coverage for the reconfigured services.
3Productivity
If the same software is used for different tenants with customizations, then time to market is reduced, but configuration faults increase
Solution Approach 1:
The patent addresses configuration faults by systematically analyzing and classifying configuration parameters. It uses parameter change analysis to identify which configuration changes may have introduced faults, enabling targeted testing and validation of customized configurations for different tenants.
4Loss of time
If existing regression test selection strategies are applied, then testing effort is reduced, but they fail to account for dependencies between services and resource sharing in cloud systems
Solution Approach 1:
The patent introduces an intermediary analysis layer that examines the relationships between configuration parameters, services, and resources. This intermediary step identifies indirect impacts of reconfigurations on dependent services and shared resources, ensuring comprehensive fault detection while maintaining testing efficiency.
Data Source
AI summary
The disclosure relates to a method, system and computer readable media for validating that a configuration of a cloud system meets requirements, using a reduced test suite selected from a test suite. The method includes classifying the requirements and service instances realizing the requirements composing the cloud system. The method includes applying test suite reduction rules to the test suite, using the classified requirements and service instances, to obtain the reduced test suite. The method includes applying the reduced test suite to the cloud system configured with the configuration, thereby validating that the configuration of the cloud system meets the requirements.