Test Script Execution Order Based on Failure Probability

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In real-world software testing projects, determining an optimal test script execution order is challenging due to the lack of available data, especially in independent verification and validation where the testing entity lacks access to detailed information about the software element being tested, often relying solely on historical test results.

Innovation Solution

A testing platform generates a test script execution order based on historical test results, prioritizing scripts with a higher likelihood of being unsuccessful first, allowing for quicker identification of failures and reducing resource expenditure by aborting tests early when failures occur.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of time

If test scripts are executed in a traditional order without prioritization, then all test scripts can be executed completely, but the time to identify failures increases and resources are wasted on scripts unlikely to fail

Engineering Contradiction:

Improvetime to identify failuresVSAvoidtesting efficiency

Core Design Contradiction:

Loss of timeVSProductivity

Solution Approach 1:

The system performs preliminary analysis of historical test results before executing the current test cycle to determine the probability of failure for each test script. This preliminary action enables the system to prioritize and order test scripts in advance, executing those most likely to fail first, thereby reducing the time to identify failures without requiring complete execution of all test scripts.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The test script execution order is dynamically adjusted based on probabilistic analysis of historical test results. The system continuously updates the likelihood of failure for each test script and reorders the execution sequence accordingly, making the testing process adaptive rather than static. This dynamic approach optimizes resource utilization by focusing on high-risk areas while maintaining the ability to detect failures efficiently.

Inventive Principle:

Principle #15Dynamics

2Use of energy by moving object

If test scripts with low failure probability are executed first, then resources are conserved by aborting early when failures occur, but the time to identify failures increases

Engineering Contradiction:

Improveprocessing resourcesVSAvoidtime to identify failures

Core Design Contradiction:

Use of energy by moving objectVSLoss of time

Solution Approach 1:

The system changes the execution parameter of test scripts by ordering them according to their calculated probability of failure. By transforming the execution sequence from traditional or random ordering to probability-based ordering, the system maximizes the likelihood of detecting failures early in the test cycle, thereby conserving processing resources while minimizing the time to identify failures.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The system replaces traditional mechanical testing approaches with a probabilistic model that analyzes historical test results to predict failure likelihood. This substitution enables intelligent decision-making about test script execution order, allowing the system to optimize both resource usage and failure detection time by executing high-probability failure scripts first.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Reliability

If complete test cycles are executed without early abortion, then all potential failures are detected, but processing, memory, and networking resources are wasted on scripts unlikely to fail

Engineering Contradiction:

Improvefailure detection completenessVSAvoidprocessing resources

Core Design Contradiction:

ReliabilityVSLoss of energy

Solution Approach 1:

The system applies partial action by executing only a subset of test scripts in each test cycle - specifically those with the highest probability of failure. Rather than executing all test scripts completely, the system performs enough testing to detect failures efficiently, aborting early when failures are found. This approach maintains adequate failure detection capability while significantly reducing resource consumption.

Inventive Principle:

Principle #16Partial or excessive action

Solution Approach 2:

The testing system serves itself by automatically analyzing its own historical performance data to determine optimal execution orders. The system uses its accumulated knowledge of which test scripts are more likely to fail to make intelligent decisions about execution prioritization, enabling self-optimization without external intervention and balancing reliability with resource efficiency.

Inventive Principle:

Principle #25Self-service

Data Source

PatentEP3629182B1Generating a test script execution order

Publication Date: 2021.05.26 ACCENTURE GLOBAL SOLUTIONS LTD

Figure 1A
Figure 1B
Figure 1C

AI summary

A device may determine probabilities for test scripts associated with a test to be executed on a software element, where a respective probability is associated with a respective test script, indicates a likelihood that the respective test script will be unsuccessful in a test cycle, and is determined based on historical test results, associated with the software element, for the respective test script. The device may generate, based on the probabilities, a test script execution order, of the test scripts, for the test cycle, and may execute, based on the test script execution order, the test on the software element in the test cycle. The device may dynamically generate, based on results for the test in the test cycle, an updated test script execution order, and may execute, based on the updated test script execution order, the test on the software element in the test cycle.