Aggregated Code Coverage Metrics for Software Promotion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Software development organizations face challenges in ensuring adequate testing of software systems before release, leading to potential software bugs and increased costs due to unclear code coverage metrics across various testing stages.
Innovation Solution
The implementation of instrumenting software modules during testing to generate and aggregate code coverage metrics, which are then used to determine promotion criteria within a multi-level software deployment hierarchy, ensuring that software modules meet specific quality thresholds before being promoted.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If extensive testing is performed on distributed computing devices at various stages in development, then software quality is improved, but the software development schedule is consumed
Solution Approach 1:
The patent applies preliminary action by establishing code coverage thresholds and promotion criteria before the software development cycle begins. The system pre-defines the multi-level deployment hierarchy and the metrics required for promotion, enabling automated quality gates that must be satisfied before code can be promoted between levels. This prevents scope creep in testing while maintaining quality standards.
Solution Approach 2:
The patent implements feedback mechanisms where code coverage metrics are automatically collected, aggregated, and used to determine whether software modules meet promotion criteria. The system provides real-time feedback to development teams about which modules meet quality thresholds and which do not, enabling data-driven decisions about promotion without manual testing review.
2Measurement precision
If code coverage metrics are aggregated across multiple testing stages, then measurement precision of testing completeness is improved, but device complexity increases
Solution Approach 1:
The patent segments the software deployment hierarchy into multiple distinct levels (e.g., development, testing, staging, production), each with its own promotion criteria. Code coverage metrics are aggregated separately for each level, and the system maintains distinct tracking for different types of testing (unit tests, integration tests, system tests). This segmentation makes the complex aggregation process manageable and scalable.
Solution Approach 2:
The patent introduces an intermediary system that automatically collects, aggregates, and processes code coverage metrics from multiple testing stages. This intermediary layer abstracts the complexity of metric aggregation from individual development teams and provides a unified view of testing completeness across the entire software lifecycle.
3Productivity
If software modules are promoted without clear code coverage metrics, then productivity is maintained, but reliability deteriorates due to potential software bugs
Solution Approach 1:
The patent enables the software development system to self-assess its own quality through automated code coverage measurement. The system automatically determines whether modules meet promotion criteria based on objective metrics, eliminating the need for manual quality assessment and reducing dependency on human judgment while maintaining high release speeds.
Data Source
AI summary
Techniques are described for aggregating code coverage data generated from various types of testing of software modules, and automatically determining whether to promote software upwards in a multi-level software deployment hierarchy based on the aggregated code coverage data. In embodiments, a code coverage metric is determined for a software module, and the metric is compared to a set of promotion criteria, including whether the metric meets a predetermined threshold for quality. In some cases, the threshold may be a general threshold, a threshold based on the level of possible promotion, and/or a threshold that is based on an identified category for the software module such as whether the module is a front-end module, a shared module, a legacy module, or a critical module.