Test Operation Software for Automated System-Under-Test Simulation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The testing of complex applications and systems, such as ASICs, is inefficient and costly due to the need for manual test scenario creation, limited test space coverage, and the inability of existing methods to automatically generate high-quality test data, leading to insufficient testing and manual analysis of simulation results.

Innovation Solution

A method using a supervisory system to automatically generate test operation software, measure output values, and iteratively alter the software to achieve expected results, thereby expanding the test space without human intervention.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If manual test scenario creation is used, then testing can be performed, but productivity is low and cost is high

Engineering Contradiction:

Improvetesting productivityVSAvoidtest scenario complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The system performs self-testing by automatically generating test scenarios, executing simulations, and evaluating results without manual intervention. The test system serves itself by using the SUT model to generate and validate its own test cases, eliminating the need for manual test creation while maintaining comprehensive test coverage

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system automatically varies test parameters and input values to generate diverse test scenarios. By systematically changing parameters within defined ranges and combinations, the system creates comprehensive test coverage without manual effort, directly addressing the productivity vs. complexity contradiction

Inventive Principle:

Principle #35Parameter changes

2Reliability

If the test space is expanded to cover more scenarios, then testing completeness improves, but the number of tests grows towards infinity making manual control impossible

Engineering Contradiction:

Improvetesting completenessVSAvoidtest space complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The system segments the vast test space into manageable subsets by defining specific test objectives, constraints, and parameter ranges. Each test scenario focuses on specific aspects of the SUT, allowing comprehensive coverage through organized segmentation rather than exhaustive brute-force testing

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system generates more test scenarios than strictly necessary by exploring the full parameter space, then filters and prioritizes results to identify the most valuable tests. This excessive generation followed by selection ensures completeness while managing complexity through post-processing

Inventive Principle:

Principle #16Partial or excessive action

3Productivity

If random test data generators are used, then test data generation is automated, but quality of test data is poor and coverage level is low

Engineering Contradiction:

Improvetest data generation speedVSAvoidtest data quality

Core Design Contradiction:

ProductivityVSManufacturing precision

Solution Approach 1:

The system uses feedback from simulation results to iteratively improve test data quality. By measuring actual SUT behavior against expected outcomes and analyzing coverage metrics, the system refines its test data generation strategy to produce higher quality tests while maintaining automation

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The system performs preliminary analysis of the SUT model and requirements before generating test data. By pre-defining valid parameter ranges, constraints, and expected behavior patterns based on domain knowledge, the system ensures high-quality test data generation from the outset rather than relying on pure randomness

Inventive Principle:

Principle #10Preliminary action

4Extent of automation

If AI-powered test automation with learning algorithms is used, then test generation is automated, but computational power requirements are large

Engineering Contradiction:

Improvetest automation levelVSAvoidcomputational power consumption

Core Design Contradiction:

Extent of automationVSUse of energy by moving object

Solution Approach 1:

Instead of using computationally expensive AI learning algorithms that require sustained high computational power, the system employs simpler, deterministic test generation methods that consume less energy. The system accepts that individual test scenarios may be less sophisticated but achieves overall effectiveness through volume and systematic coverage

Inventive Principle:

Principle #27Cheap short-living objects (Disposable)

Data Source

PatentEP4645096A1Method for operating a system under test

Publication Date: 2025.11.05 SIEMENS SRL

Figure 1
Figure 2

AI summary

It is provided a method for operating a system under test, SUT (35). The method being performed using a supervisory system, SS (20), including a computer-implemented generation unit (21) adapted to generate test operation software, TOS, for the operation of the SUT (35). Said SS (20) further includes a measuring unit, MU (22), adapted to measure output values of at least one user-defined observation variable within a behavior and/or a configuration resulting from an operation of the SUT (35). The method further uses a testing system, TS (30), which is controllable by the SS (20). The TS (30) includes the SUT (35) and a simulator. Via an interaction of the SS (20) and the TS (30), the method generates new test operation software for the operation of the SUT (35). Furthermore, a corresponding test and operation system, a corresponding computer program product and a corresponding computer-readable medium are provided.