Test-to-file map for parallel code coverage accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In the context of continuous integration processes for software applications, existing code coverage testing methods produce inaccurate reports due to parallel testing, leading to inefficiencies in identifying covered code portions and attributing test failures to responsible developers.
Innovation Solution
An incremental learning-based approach is employed to generate and update a test-to-file map, correlating specific tests with application code files, and identifying responsible developers for test failures through correlation analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If parallel code coverage tests are run to improve productivity, then testing efficiency is improved, but measurement precision of code coverage reports deteriorates
Solution Approach 1:
The patent segments the code coverage measurement process by creating a test-to-file map that associates specific test cases with specific source code files. This segmentation allows parallel tests to be tracked independently, preventing the mixing of coverage data that occurs in traditional parallel testing, thereby maintaining measurement precision while preserving testing efficiency
Solution Approach 2:
The patent introduces an intermediary mechanism (the test-to-file map data structure) that mediates between parallel test executions and the final code coverage report. This intermediary tracks which files were modified and which tests covered those files, allowing accurate attribution of coverage results to specific tests even when tests run in parallel, thus resolving the precision problem without sacrificing productivity
2Ease of manufacture
If traditional code coverage reporting is used to identify covered code portions, then implementation is simple, but loss of information occurs regarding which specific tests covered which files
Solution Approach 1:
The patent performs preliminary action by building the test-to-file map before generating the final code coverage report. During the testing phase, the system pre-establishes which tests cover which files by tracking file modifications and test executions, storing this information in the map. This preliminary organization preserves complete test-to-file attribution information that can then be used to generate accurate reports without losing any coverage information
Solution Approach 2:
The patent implements feedback by using the test-to-file map to provide detailed information back to developers about which specific tests covered which modified files. When code is modified, the system queries the map to identify relevant tests, providing feedback that helps developers understand test coverage status and improve their test suites, thereby preventing information loss about test-to-file relationships
3Reliability
If all tests are run to ensure complete coverage after code modification, then reliability is improved, but loss of time occurs due to unnecessary test executions
Solution Approach 1:
The patent applies partial action by running only the subset of tests that are relevant to modified files rather than executing all tests. By using the test-to-file map to identify which tests cover which modified files, the system performs just enough testing to ensure quality (partial action) without the waste of running unnecessary tests, thus reducing time loss while maintaining reliability
Solution Approach 2:
The patent introduces dynamics by making the test selection process adaptive based on which files have been modified. Rather than a static approach of running all tests, the system dynamically determines the appropriate test subset by querying the test-to-file map with the list of modified files, adjusting the testing scope to match the actual changes made, thereby optimizing both reliability and time efficiency
4Productivity
If parallel testing is implemented to increase productivity, then testing speed is improved, but difficulty of detecting and measuring test failure sources increases
Solution Approach 1:
The patent uses the test-to-file map as an intermediary to resolve test failures in parallel testing environments. When a test fails, the map provides information about which files were modified and which tests should have covered those files, enabling systematic identification of the failure source even when multiple tests run in parallel, thus reducing the difficulty of detecting and measuring test failure sources while maintaining high testing speed
Data Source
AI summary
In some examples, a method may include receiving, by a system comprising a processor, for a first invocation of a first test of a plurality of code coverage tests run at overlapping times, a first report indicating a first subset of files of application code covered by the first test. The method may include generating, by the system, a test-to-file map associating the first subset of files to the first test. The method may include receiving, by the system, a second report indicating a second subset of files of the application code covered by a second invocation of the first test. The method may include modifying, by the system, the test-to-file map to associate the second subset of files to the first test.