Software Testing System Optimizing Test Case Scheduling

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Software testing systems require excessive time and computing resources due to large and expansive test suites, leading to delayed feedback and increased costs, as they often test entire codebases despite only minor changes, resulting in inefficient use of resources and prolonged testing runtimes.

Innovation Solution

Implement a system that collects test statistics to generate a reduced test suite, selecting only relevant test cases based on data analysis, thereby optimizing test scheduling and focusing on specific code blocks, reducing the number of test cases executed and streamlining error detection.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If a large and expansive test suite is used to test the entire codebase, then testing coverage is improved, but testing runtime and CPU consumption increase significantly

Engineering Contradiction:

Improvetesting coverageVSAvoidtesting runtime

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent segments the test suite into multiple subsets based on code change identification. Instead of executing the entire test suite, the system divides tests into relevant and irrelevant subsets, executing only those that pertain to the changed code portions. This segmentation maintains necessary testing coverage while reducing overall execution time and resource consumption.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent extracts and identifies only the relevant test cases from the full test suite that are applicable to the specific code changes. By taking out and isolating the necessary tests based on change analysis, the system eliminates unnecessary test executions while preserving coverage of affected functionality, thereby reducing runtime without sacrificing reliability.

Inventive Principle:

Principle #2Taking out (Extraction)

2Reliability

If a large and expansive test suite is used to test the entire codebase, then testing coverage is improved, but computing resources are consumed excessively

Engineering Contradiction:

Improvetesting coverageVSAvoidCPU consumption

Core Design Contradiction:

ReliabilityVSUse of energy by moving object

Solution Approach 1:

The test suite is segmented into relevant and irrelevant subsets based on the analysis of code changes. By dividing the test execution into targeted segments rather than running the complete suite, the system reduces CPU consumption while maintaining adequate coverage of the modified code areas.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent applies partial action by executing only the necessary portion of the test suite that corresponds to the changed code, rather than running the entire test suite. This partial execution approach provides sufficient testing coverage for the modifications while significantly reducing computing resource usage.

Inventive Principle:

Principle #16Partial or excessive action

3Measurement precision

If the entire test suite is executed for every code change, then error detection completeness is improved, but feedback time is delayed

Engineering Contradiction:

Improveerror detection completenessVSAvoidfeedback time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The system performs preliminary analysis of code changes before executing tests. By pre-identifying which test cases are relevant to the specific changes made, the system prepares a targeted test subset in advance, enabling faster execution while maintaining error detection completeness for the affected areas.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The test suite is segmented based on the preliminary analysis of code changes. This segmentation allows the system to execute only the relevant test cases that are likely to detect errors in the modified code, providing timely feedback without compromising detection completeness for the changed portions.

Inventive Principle:

Principle #1Segmentation

4Productivity

If test statistics are collected and analyzed to generate a reduced test suite, then testing efficiency is improved, but system complexity increases

Engineering Contradiction:

Improvetesting efficiencyVSAvoidsystem complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The system implements feedback mechanisms by collecting test statistics and using them to inform future test selections. The test results and code change data feed into an analysis process that generates optimized test subsets, creating a feedback loop that continuously improves testing efficiency while managing system complexity through automated decision-making.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The patent introduces an intermediary layer (the test analysis system) that sits between the code changes and the test execution. This intermediary collects statistics, analyzes changes, and determines the optimal test subset, thereby improving efficiency while encapsulating the complexity within a manageable intermediate component rather than requiring complex changes throughout the entire system.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS11003574B2Optimized testing system

Publication Date: 2021.05.11 SAP SE

AI summary

Described herein includes a software testing system that optimizes test case scheduling to efficiently and speedily analyze a block of code. The system enhances the performance of software testing by implementing a test controller using test statistics to optimize testing performance. The test controller may use the test statistics to determine relevant test cases to execute, and to provide better and/or faster feedback to users.