Test Automate Impact Scoring for Continuous Execution Pruning

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The excessive consumption of infrastructure resources and increased costs due to the execution of non-valuable test automates in software testing, coupled with inefficient resource utilization and delays in automated test execution queues, are exacerbated by poorly designed or failing test automates that do not provide meaningful results.

Innovation Solution

An impact quotient determination system evaluates test automates based on static rules, software analysis, historical data, and expert opinion to assign an impact quotient, enabling the disabling of low-impact test automates and optimizing resource allocation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If test automates are executed to maintain software quality, then software quality is improved, but infrastructure resources are excessively consumed and costs increase

Engineering Contradiction:

Improvesoftware qualityVSAvoidinfrastructure resource consumption

Core Design Contradiction:

ReliabilityVSLoss of energy

Solution Approach 1:

The patent extracts and removes non-valuable test automates from the execution pool by evaluating each test automate's contribution using multiple criteria (static rules, software analysis, historical data, expert opinion). Tests that do not provide meaningful results are identified and excluded from further execution, thereby reducing infrastructure resource consumption while maintaining the quality assurance provided by valuable tests

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent introduces an impact quotient metric that quantifies the value of each test automate based on multiple parameters including failure rates, execution frequency, code coverage, and expert assessments. By changing the parameter set used to evaluate test automates from simple pass/fail to a comprehensive impact quotient, the system can differentiate between valuable and non-valuable tests, optimizing resource allocation

Inventive Principle:

Principle #35Parameter changes

2Reliability

If the number of test automates is increased to cover more scenarios, then testing coverage is improved, but resource utilization efficiency deteriorates

Engineering Contradiction:

Improvetesting coverageVSAvoidresource utilization efficiency

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent applies local quality by evaluating each test automate individually with its own impact quotient based on specific characteristics such as failure rate, execution frequency, and code coverage. Rather than treating all test automates uniformly, the system assigns different weights and criteria to different tests, allowing high-impact tests to be executed frequently while low-impact tests are pruned, thereby optimizing overall resource utilization while maintaining comprehensive coverage

Inventive Principle:

Principle #3Local quality

3Reliability

If test automates are executed continuously to ensure quality, then software quality monitoring is improved, but execution delays occur due to queue load

Engineering Contradiction:

Improvequality monitoringVSAvoidexecution delay

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent performs preliminary evaluation of test automates before execution by calculating impact quotients using static rules, software analysis, and historical data. This preliminary assessment identifies which tests are worth executing and which should be pruned in advance, preventing unnecessary tests from being added to the execution queue and thereby reducing execution delays while maintaining continuous quality monitoring of valuable tests

Inventive Principle:

Principle #10Preliminary action

4Reliability

If all test automates are maintained to provide comprehensive testing, then testing completeness is improved, but infrastructure costs increase

Engineering Contradiction:

Improvetesting completenessVSAvoidinfrastructure cost

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The patent systematically discards non-valuable test automates by evaluating them against multiple criteria and identifying those with low impact quotients. The system recovers infrastructure resources by stopping execution of pruned tests while maintaining execution of high-impact tests, thereby reducing overall infrastructure costs while preserving testing completeness through selective retention of valuable tests

Inventive Principle:

Principle #34Discarding and recovering

Data Source

PatentUS20250378014A1Impact quotient determination for pruning test automates

Publication Date: 2025.12.11 SAP SE

AI summary

As impact quotient for a test automate can be determined and assigned based on factors such as static rules, results of a software analysis of the test automate, results of an analysis of historical data regarding the test automate, and a functional expert opinion. The factors used in the impact quotient determination can be assigned different weights which are adjustable based on the functional expert opinion. A test automate whose assigned impact quotient is below a threshold can be disabled and thus excluded from continuous executions. A test automate whose assigned impact quotient is not below the threshold can maintain an enabled status; updated impact quotients can later be determined for the test automate in response to prompts, in response to which the test automate can either maintain its enabled status or be disabled. Accordingly, low impact test automates can be removed from the pool of continuously executed test automates.