Regression Test Selection in Multi-Threaded Distributed Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current regression testing methods are inefficient for multi-threaded, distributed target programs as they require running entire test suites, leading to high costs and long execution times due to the need for single-threaded execution and frequent calibration, which is impractical and ineffective, especially in scenarios with user interfaces and distributed environments where thread relationships are complex and source code access is limited.
Innovation Solution
A method and system using a calibration engine with an execution-engine-tracer and throttling engine to isolate and map test cases to code modules in a multi-threaded distributed environment, establishing parent-child relationships between threads and regulating thread creation to reduce calibration time and enable parallel execution of impacted test cases.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If single-threaded execution mode is used for regression testing, then calibration time is reduced and thread mapping is simplified, but execution time increases exponentially and productivity decreases
Solution Approach 1:
The patent implements dynamic thread throttling that adjusts the degree of parallelization based on calibration needs. During calibration phases, threads are throttled to near-single-threaded operation to simplify mapping, while during execution phases, full parallelization is enabled to maximize productivity. This dynamic adjustment resolves the contradiction by allowing the system to operate in different modes depending on the operational context.
Solution Approach 2:
The patent performs preliminary calibration during idle periods or low-traffic times when the impact on productivity is minimal. By completing calibration tasks in advance and storing the resulting thread mappings for reuse, the system avoids the need for frequent full calibrations that would otherwise require single-threaded operation and significantly increase execution time.
2Productivity
If multi-threaded parallel execution is used for regression testing, then execution time is reduced and productivity increases, but calibration complexity increases and thread mapping becomes difficult
Solution Approach 1:
The patent introduces an intermediary thread mapping service that acts as a mediator between test cases and target program threads. This service maintains and manages the complex thread mappings, storing them in data structures that enable efficient lookup during parallel execution. By externalizing the mapping management to a dedicated intermediary component, the complexity of multi-threaded calibration is isolated and managed separately from the core testing logic.
Solution Approach 2:
The patent creates and maintains copies of thread mapping data in multiple locations and formats for different purposes. Thread mappings are copied from the calibration process into lookup tables that can be quickly accessed during parallel test execution. This copying approach allows the system to handle complex multi-threaded mappings during calibration while enabling simple, fast access patterns during execution, thereby reducing the perceived complexity.
3Measurement precision
If frequent calibration is performed for every code change, then test case mapping accuracy is improved, but execution time increases and productivity decreases
Solution Approach 1:
The patent implements partial calibration by identifying and calibrating only the specific portions of the system affected by code changes, rather than performing full calibration every time. The thread mapping service detects changed modules and triggers calibration only for test cases that exercise those modules, maintaining high mapping accuracy while minimizing the execution time overhead associated with frequent full calibrations.
Solution Approach 2:
The patent incorporates feedback mechanisms where the thread mapping service monitors code changes and automatically triggers calibration only when necessary. By using version control integration and change detection, the system receives feedback about what has changed and adjusts its calibration frequency accordingly, performing full calibration only when the change detection system determines it is necessary to maintain mapping accuracy.
4Adaptability or versatility
If distributed target program execution is tested, then real-world scenario coverage is improved, but tracking execution path at individual test case level becomes impossible
Solution Approach 1:
The patent segments the distributed target program execution into discrete, trackable units by associating each test case with specific target program threads through the thread mapping service. Even in distributed environments where multiple processes and threads are involved, the segmentation approach maintains clear boundaries between test case executions and the corresponding program state, enabling execution path tracking at the individual test case level while preserving real-world scenario coverage.
Data Source
AI summary
A method, a system and a computer program product for regression test selection in a multi-threaded distributed target program execution tested by multi-threaded test suites. The method includes monitoring changes to source code of the target program in real-time to determine changed code-modules and extracting the regression test cases which simulate the changed code-modules. A calibration engine according to an embodiment isolates test case execution flow at individual test case level and builds relationship between the test case of the multi-threaded test suite and the distributed target program code-modules simulated by the test case. The method enhances the run-time behavior of the target program execution engines and builds relationships between the threads executing in the engine through throttling mechanism without affecting the actual functional execution. It enables calibration to happen in a multi-threaded test environment.