Command Coverage Assessment Using Actual Test Execution Logs

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional code coverage testing methods fail to prioritize code paths that are actually used by customers, as they rely on assumed inputs rather than actual customer usage, leading to gaps between functional specifications and actual code usage.

Innovation Solution

A command coverage assessment system that generates a report using a command tree map based on actual test case logs, prioritizing code paths touched by users and bridging the gap between functional specifications and actual customer usage.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If conventional code coverage testing methods are used that rely on assumed inputs, then testing can be performed without actual customer usage data, but the testing fails to prioritize code paths that are actually used by customers

Engineering Contradiction:

Improvetest coverage accuracyVSAvoidcustomer usage information

Core Design Contradiction:

ReliabilityVSLoss of information

Solution Approach 1:

The system performs preliminary actions by collecting and analyzing customer usage data before executing regression tests. Test case logs are gathered in advance to identify actual code paths used by customers, which then informs the prioritization of test execution. This preliminary analysis ensures that testing focuses on the most relevant code paths based on real usage patterns rather than assumptions.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements feedback by continuously analyzing test case logs to determine which code paths are actually executed during customer usage. This feedback loop identifies gaps between assumed and actual code usage, allowing the system to adjust test priorities and improve coverage accuracy over time. The feedback mechanism ensures that testing evolves based on real-world usage data.

Inventive Principle:

Principle #23Feedback

2Productivity

If functional testing is performed to capture functional regression issues early, then release delays can be reduced, but issues are often still caught during system integration testing phase

Engineering Contradiction:

Improverelease speedVSAvoiddefect detection effectiveness

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The system applies dynamics by adaptively adjusting test execution priorities based on actual code path analysis from test logs. Rather than using static test schedules, the system dynamically reweights and reprioritizes test cases based on which code paths are most frequently executed by customers. This dynamic approach allows functional testing to focus on critical paths while maintaining flexibility to adapt as usage patterns are better understood.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system performs partial action by selectively executing only the most relevant test cases based on code path analysis, rather than running all possible tests. By identifying and focusing on the specific code paths that are actually used by customers, the system achieves effective defect detection with a subset of tests, improving release speed without sacrificing reliability.

Inventive Principle:

Principle #16Partial or excessive action

3Ease of manufacture

If regression tests are developed based on envisioned customer usage rather than actual usage, then testing can proceed without waiting for customer data, but gaps between functional specifications and actual code usage remain

Engineering Contradiction:

Improvetest development easeVSAvoidcoverage accuracy

Core Design Contradiction:

Ease of manufactureVSManufacturing precision

Solution Approach 1:

The system performs preliminary actions by collecting and analyzing actual customer usage data from test case logs before finalizing test development. Rather than relying solely on envisioned usage, the system proactively gathers real usage information to inform test case creation and prioritization. This preliminary data collection bridges the gap between assumed and actual usage patterns.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system substitutes manual estimation of customer usage with automated analysis of actual test case logs. Instead of relying on developers' visions of how customers will use the software, the system mechanically processes real usage data to identify actual code paths. This substitution replaces subjective judgment with objective data-driven insights.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS20250307126A1Practical coverage checking mechanism based on actual test execution logs

Publication Date: 2025.10.02 DELL PROD LP

AI summary

Methods, system, and non-transitory processor-readable storage medium for a command coverage assessment system are provided herein. An example method includes executing at least one test case on a system. A command coverage assessment system obtains at least one test case log resulting from the execution of the one or more test case. The command coverage assessment system generates a command coverage report using a command tree map, based on the at least one test case log.