Automated System Verification Test Using Behavior Models

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional system verification test (SVT) methods, both manual and automated, are labor-intensive and time-consuming, especially in regression testing, and require significant human effort in designing, testing, and maintaining scripts, which limits efficiency and accuracy.

Innovation Solution

An automated SVT system that generates a behavior model representing the System Under Test (SUT) based on actions and expected reactions, eliminating the need for scripts by capturing primitives from test case runs and generating a generalized behavior model to identify anomalies.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Extent of automation

If conventional automated SVT uses scripts to execute test cases automatically, then test case execution is automated and repeated, but significant human labor is required to design, test, and maintain the scripts

Engineering Contradiction:

Improvetest case execution automationVSAvoidscript design and maintenance complexity

Core Design Contradiction:

Extent of automationVSDevice complexity

Solution Approach 1:

The system automatically generates test scripts by capturing primitives from test case executions and synthesizing them into executable behavior models. The SVT system serves itself by extracting actions and reactions directly from system responses, eliminating the need for manual script creation and maintenance while achieving full automation of test case execution

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent replaces the mechanical process of manually writing and maintaining test scripts with an automated computational system. The SVT system captures primitive actions and reactions from system responses, generates behavior models automatically, and executes tests without human intervention in script creation, thereby substituting manual mechanical work with automated intelligent processing

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Productivity

If conventional manual SVT is used for regression testing, then test cases can be executed, but the process is labor-intensive and time-consuming

Engineering Contradiction:

Improveregression testing throughputVSAvoidtime required for regression testing

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The system performs preliminary capture of primitive actions and reactions during initial test execution, storing them in a database. When regression testing is needed, the pre-captured primitives are reused to automatically generate and execute behavior models, eliminating the need to re-execute all test cases from scratch and significantly reducing regression testing time

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent creates copies of test case executions in the form of captured primitives and generated behavior models. Instead of manually re-executing test cases for each regression cycle, the system copies the essential action-reaction patterns from previous executions and reuses them, enabling rapid regression testing without repeating the full test suite manually

Inventive Principle:

Principle #26Copying

3Reliability

If conventional automated SVT requires script design and testing to ensure no false-negatives and false-positives, then test accuracy is improved, but significant human labor is required before automation benefits are realized

Engineering Contradiction:

Improvetest result accuracyVSAvoidscript testing and validation complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The system uses feedback from actual system responses to automatically refine and validate behavior models. By capturing real reactions from the system under test and comparing them against generated expectations, the SVT system self-validates test accuracy without requiring manual script testing, ensuring reliable results while eliminating the labor-intensive validation phase

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The SVT system performs self-validation by automatically comparing captured primitive reactions against generated behavior model expectations. The system tests itself to ensure no false-negatives or false-positives without requiring external manual verification, achieving high test reliability while eliminating the complex manual script testing process

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS7480602B2System verification test using a behavior model

Publication Date: 2009.01.20 SPIRENT COMM INC

AI summary

The present invention provides a system verification system that automatically generates a behavior model modeling the system under test in terms of actions of a test case and a range of expected reactions corresponding to those actions. In this regard, the system verification system obtains a set of actions and individual reactions corresponding to the actions from a plurality of runs of a test case for the system under test, and automatically generates the behavior model representing the system under test in terms of the set of actions and a range of expected reactions each range corresponding one of the actions. The range of expected reactions generalizes and includes all or most of the individual reactions corresponding to said one of the actions.