Consensus Testing Electronic Systems Using Recorded Traffic

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conformance testing faces challenges in creating accurate expected outputs, especially when specifications are incomplete, and model-based testing requires significant effort to create models, limiting the number and accuracy of test cases.

Innovation Solution

A tester that receives traffic from an electronic system, extracts and compares element values with another system, generating consensus information on interoperability without pre-defined expected outputs, allowing for a higher number of test cases and reducing the need for expert mathematical or modeling skills.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If model-based testing is used to create test cases, then the number of test cases can be increased, but the effort required to create the model is comparable to creating actual conformance test cases

Engineering Contradiction:

Improvenumber of test casesVSAvoideffort to create model

Core Design Contradiction:

Quantity of substanceVSDevice complexity

Solution Approach 1:

The patent uses recorded traffic from actual system executions as copies of real behavior patterns. Instead of creating models abstractly, the system captures concrete examples of protocol traffic and uses these recordings as the basis for generating test cases, eliminating the need for complex model creation while maintaining test coverage

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The system performs preliminary recording of traffic from actual system executions before generating test cases. This preliminary action captures real-world behavior patterns that can then be reused to generate multiple test cases without requiring repeated manual model creation or expert intervention

Inventive Principle:

Principle #10Preliminary action

2Measurement precision

If traditional conformance testing is used, then expected outputs can be defined, but the difficulty of coming up with correct expected outputs increases when specifications are incomplete

Engineering Contradiction:

Improveaccuracy of expected outputsVSAvoiddifficulty of designing test cases

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The system uses recorded traffic from actual system executions to automatically generate expected outputs. The recorded traffic itself serves as the reference for what correct behavior looks like, eliminating the need for experts to manually define expected outputs based on potentially incomplete specifications

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system continuously refines test case generation by analyzing recorded traffic patterns and using these patterns to generate and validate new test cases. This feedback loop ensures that expected outputs are derived from actual observed behavior rather than potentially flawed specifications

Inventive Principle:

Principle #23Feedback

3Quantity of substance

If model-based testing is used, then more test cases can be generated, but the accuracy and relevance of test cases depend solely on the model which adds a level of indirection

Engineering Contradiction:

Improvenumber of test casesVSAvoidaccuracy of test cases

Core Design Contradiction:

Quantity of substanceVSMeasurement precision

Solution Approach 1:

The patent directly copies recorded traffic patterns from actual system executions to generate test cases. This direct copying approach eliminates the indirection introduced by abstract models, ensuring that test cases accurately reflect real-world behavior while still enabling generation of multiple test cases from the recorded patterns

Inventive Principle:

Principle #26Copying

4Reliability

If regression testing or back-to-back testing is used, then results of two implementations can be compared, but both methods are limited to situations where results of two implementations are compared to pinpoint differences

Engineering Contradiction:

Improveinteroperability testing capabilityVSAvoidtesting scenario flexibility

Core Design Contradiction:

ReliabilityVSAdaptability or versatility

Solution Approach 1:

The system uses recorded traffic as universal test material that can be applied across multiple testing scenarios. The same recorded traffic patterns can be used to test different implementations, generate consensus test cases, and verify interoperability, making the testing approach versatile rather than limited to single-purpose comparison

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS7797590B2Consensus testing of electronic system

Publication Date: 2010.09.14 BLACK DUCK SOFTWARE INC

AI summary

Consensus testing of electronic system. A tester (112) for testing an electronic system (100) includes: a traffic interface (114) to receive traffic (102) from a test of an electronic system (100); an element comparator (118) to extract a value from an element of the traffic (102) and to compare the extracted element value with an element value (110) obtained from another test of another electronic system (104, 106, 108); and a test result generator (122) to generate consensus information (124) on the interoperability of the electronic system (100), based on comparing (120) the extracted element values of the electronic system (100) with the element values obtained from the other test of the other electronic system (104, 106, 108).