Code Flow Trees for Scalable Functional Coverage Test Selection

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing methods face challenges in efficiently representing and operating on large functional coverage test-spaces, which are crucial for ensuring software quality, as they often require explicit enumeration of all possible combinations, leading to scalability issues and increased testing effort.

Innovation Solution

The use of code flow trees, specifically Binary Decision Diagrams (BDDs), to symbolically represent the functional coverage test-space, allowing for efficient test planning and selection of optimal tests that cover all pairwise combinations of values, thereby reducing the complexity and effort of testing.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If explicit enumeration of all possible combinations is used to represent functional coverage test-space, then complete coverage can be achieved, but scalability deteriorates and testing effort increases

Engineering Contradiction:
Improvefunctional coverage completenessVSAvoidtest-space representation complexity
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The patent segments the functional coverage test-space into a hierarchical structure using a code flow tree, where the root node represents the system under test and leaf nodes represent individual code statements. This segmentation allows the test-space to be divided into manageable functional units (functions, procedures, blocks) that can be independently analyzed and covered, resolving the contradiction between achieving complete coverage and managing complexity.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent transitions from explicit enumeration in one dimension (listing all test cases) to a tree-based dimensional representation where test-space is organized hierarchically. This dimensional change allows compact representation of the test-space structure, enabling complete functional coverage to be achieved without explicitly enumerating all possible test combinations, thus reducing complexity while maintaining reliability.

Inventive Principle:
Principle #17Another dimension (Dimensionality change)

2Reliability

If explicit enumeration of all possible combinations is used, then all functional behaviors can be tested, but testing effort and time increase

Engineering Contradiction:
Improvefunctional coverage completenessVSAvoidtesting effort
Core Design Contradiction:
ReliabilityVSLoss of time

Solution Approach 1:

The patent performs preliminary action by automatically generating the code flow tree and identifying all executable statements and functions before test execution. This preliminary analysis of the test-space structure enables efficient test case selection and reduces the time required during actual testing, as the framework is already prepared with organized functional units and coverage requirements.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The code flow tree acts as an intermediary between the system under test and the test execution process. Instead of directly enumerating and executing all possible test combinations, the tree structure serves as a mediator that organizes functional units and guides test selection, thereby reducing testing effort while ensuring complete functional coverage is achieved.

Inventive Principle:
Principle #24Intermediary (Mediator)

3Productivity

If code flow trees are used to symbolically represent test-space, then scalability improves, but complexity analysis becomes necessary

Engineering Contradiction:
Improvetest-space representation efficiencyVSAvoidtree complexity
Core Design Contradiction:
ProductivityVSDevice complexity

Solution Approach 1:

The patent implements feedback by analyzing tree complexity metrics (such as tree depth, number of nodes, and branching factor) and using this information to guide test case selection and optimization. This feedback mechanism allows the system to automatically adjust test strategies based on the actual complexity of the code flow tree, ensuring scalable representation while managing complexity through data-driven decisions.

Inventive Principle:
Principle #23Feedback

Data Source

PatentUS11422924B2Customizable test set selection using code flow trees
Publication Date: 2022.08.23 INTERNATIONAL BUSINESS MACHINE CORPORATION
  • US11422924B2 patent drawing
  • US11422924B2 patent drawing
  • US11422924B2 patent drawing

AI summary

A method for identifying optimal tests includes defining functional coverage by a test suite based on a functional coverage model of a System Under Test (SUT). The test suite includes a plurality of tests. The functional coverage model includes a plurality of attributes, each attribute having a set of possible values. The functional coverage model defines possible combinations of values of the attributes as covered by the test suite. A subset of the possible combinations of values is determined. A subset of the plurality of tests is selected that is operative to cover the determined subset of the possible combinations of values. A plurality of trees is generated to represent the selected subset of tests. Complexity of the generated trees is analyzed based on user-specified criteria. An optimal tree is selected from the subset of the plurality of trees to achieve the objective of the test suite.