Intelligent Concurrent Software Testing for Cycle Time Reduction

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Continuous integration of software involves time-consuming testing protocols that require executing an entire test plan, wasting computing resources and developer time, as existing methods do not efficiently identify and execute only the necessary tests for code changes.

Innovation Solution

The system automatically detects changed code, selects a subset of tests, updates annotations, and distributes these tests into groups for concurrent execution, ensuring each group completes in approximately the same time, thereby reducing overall testing duration.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If the entire test plan is executed every test cycle, then testing completeness is ensured, but testing time and computing resource consumption increase significantly

Engineering Contradiction:

Improvetesting completenessVSAvoidtesting time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent segments the entire test plan into multiple test cases and further divides them into groups or buckets based on execution time characteristics. This segmentation allows the system to select and execute only relevant subsets of tests rather than the complete test plan, thereby reducing testing time while maintaining necessary testing coverage.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent implements partial action by executing only a subset of tests that are relevant to the current code changes being tested. The system identifies and runs only the necessary test cases rather than the full test plan, achieving sufficient testing coverage with reduced time and resource consumption.

Inventive Principle:

Principle #16Partial or excessive action

2Reliability

If the entire test plan is executed every test cycle, then all potential issues are detected, but computing resource waste increases

Engineering Contradiction:

Improveissue detection capabilityVSAvoidcomputing resource consumption

Core Design Contradiction:

ReliabilityVSLoss of energy

Solution Approach 1:

The patent extracts and identifies the specific subset of tests that are relevant to the current code changes from the entire test plan. By taking out only the necessary tests based on change analysis, the system detects relevant issues without consuming computing resources on unrelated test executions.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The system performs partial testing by executing only the subset of tests that correspond to changed code areas. This partial action approach maintains issue detection capability for the affected components while avoiding waste of computing resources on tests for unchanged components.

Inventive Principle:

Principle #16Partial or excessive action

3Device complexity

If tests are executed sequentially, then resource management is simplified, but total testing duration increases

Engineering Contradiction:

Improveresource management complexityVSAvoidtotal testing duration

Core Design Contradiction:

Device complexityVSLoss of time

Solution Approach 1:

The patent segments the test suite into multiple groups or buckets with balanced execution times. This segmentation enables concurrent execution of multiple test groups, reducing total testing duration while maintaining manageable resource allocation through structured grouping.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent transitions from sequential (one-dimensional) test execution to parallel/concurrent (multi-dimensional) execution by distributing test groups across multiple execution threads or processes. This dimensional change reduces total testing time while the grouping structure maintains resource management control.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

4Productivity

If a subset of tests is selected based on code changes, then testing efficiency improves, but test selection accuracy becomes more critical

Engineering Contradiction:

Improvetesting efficiencyVSAvoidtest selection accuracy

Core Design Contradiction:

ProductivityVSMeasurement precision

Solution Approach 1:

The patent implements feedback mechanisms where test execution results and code change analysis inform future test selections. The system learns from previous executions and refines its test selection accuracy over time, ensuring that the subset of tests selected based on code changes maintains high precision in identifying relevant test cases.

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS20240320135A1Intelligent concurrent testing for test cycle time reduction

Publication Date: 2024.09.26 HARNESS INC

AI summary

A system that automatically reduces the time to execute software testing through intelligent test selection and execution. The system automatically detects what tests to execute based on code that has been changed, which is a subset of the entire list of tests to run for the block of code. Once the subset of tests is identified, annotations for tests are processed to update the subset as desired by the code administrator. Once updated, the system then automatically obtains the tests for the updated subset of tests. The tests to be executed are then distributed into groups or buckets. The distribution is set so that each group of tests will have as close to the same execution time as possible. The tests in each group or bucket are then executed concurrently with the other grouped tests.