Test Coverage Optimizing Mechanism Based on Metric Evaluation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional technologies for optimizing test coverage in software development are time-consuming, subjective, and inefficient, making it difficult to evaluate and improve test coverage across multiple test cases effectively.
Innovation Solution
A test coverage optimizing system that executes test cases on a system, using a weighted average baseline metrics calculation module to assess test-related failures and a coverage imbalance module to identify areas of imbalance, providing an automated and comprehensive assessment of test coverage and recommending improvements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional static review techniques are used to evaluate test coverage, then subjective assessment can be performed, but the process becomes time-consuming and inefficient
Solution Approach 1:
The patent replaces manual static review techniques with an automated computational system that executes test cases and calculates coverage metrics programmatically. The system automatically runs test cases, collects execution data, and computes coverage statistics without human intervention, thereby eliminating the time-consuming nature of manual reviews while maintaining or improving measurement precision through systematic data collection and calculation.
Solution Approach 2:
The test coverage optimization system performs self-evaluation by automatically executing its own test cases and assessing its coverage metrics. The system independently calculates coverage statistics, identifies imbalances, and generates optimization recommendations without requiring external manual assessment, thereby resolving the contradiction between evaluation accuracy and time consumption.
2Measurement precision
If comprehensive test coverage evaluation is performed across multiple test cases, then accurate assessment can be achieved, but the complexity of the evaluation process increases
Solution Approach 1:
The patent divides the comprehensive test coverage evaluation into distinct modular components: test case execution modules, data collection modules, metric calculation modules, and analysis modules. Each module handles a specific aspect of the evaluation process, making the overall complex system manageable through functional segmentation. This modular architecture allows comprehensive evaluation while controlling system complexity through clear separation of concerns.
Solution Approach 2:
The evaluation system is designed with universal components that can handle multiple test cases and various coverage metrics through a unified framework. The same core infrastructure executes different test cases, collects diverse data types, and calculates multiple coverage statistics, thereby achieving comprehensive assessment without proportionally increasing system complexity through repeated specialized components.
3Productivity
If automated test execution is implemented, then evaluation efficiency improves, but the need for systematic metric calculation and analysis increases complexity
Solution Approach 1:
The system performs preliminary setup by pre-defining coverage metrics, test case configurations, and evaluation criteria before execution. Data collection structures and calculation frameworks are prepared in advance, so that during automated test execution, the system can efficiently process results without ad-hoc complexity. This preliminary preparation enables high-speed automated evaluation while managing calculation complexity through pre-established methodologies.
Data Source
AI summary
Methods, system, and non-transitory processor-readable storage medium for test coverage optimizing system are provided herein. An example method includes executing at least one test case on a system. A weighted average baseline metrics calculation module calculates a weighted average baseline metric for at least one test case using test related failures that occurred during the execution. A coverage imbalance module calculates a coverage imbalance associated with a test domain using the weighted average baseline metric, where the test domain comprises the at least one test case.