Code Filter for Selective Test Coverage Evaluation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Workflow management of software development and testing is complex, especially in managing millions of lines of code developed by different individuals, where keeping track of what code requires testing and ensuring it is adequately tested is a challenging task due to the interleaved nature of dynamic and static testing processes.
Innovation Solution
A method is provided to configure a computer to automatically determine if specific portions of code have been adequately tested by using a code filter to include or omit portions from test results, leveraging static and dynamic analysis techniques, and source control management to evaluate test coverage based on user-defined policies and metadata.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If comprehensive testing of all code portions is performed, then test coverage is improved, but testing time and resources are excessively consumed
Solution Approach 1:
The patent segments the codebase into different portions based on change history and metadata, allowing selective testing of only those segments that require testing rather than testing the entire codebase. This segmentation enables the system to identify and test only the affected portions of code, thereby reducing overall testing time while maintaining adequate coverage of critical areas.
Solution Approach 2:
The patent applies local quality by assigning different testing priorities and requirements to different code portions based on their metadata and change history. Critical code sections that have been modified receive higher testing priority, while stable, unchanged code sections receive lower priority or are excluded from testing. This differential approach optimizes testing resources focused on areas needing verification.
2Productivity
If selective testing of code portions is performed, then testing efficiency is improved, but risk of missing critical bugs increases
Solution Approach 1:
The patent implements feedback mechanisms where test results and code change history are continuously analyzed to refine future testing decisions. The system learns from previous test outcomes and change patterns, adjusting which code portions require testing in subsequent cycles. This feedback loop ensures that critical bug-prone areas are consistently identified and tested, maintaining high bug detection completeness despite selective testing approaches.
Solution Approach 2:
The patent performs preliminary analysis of code change history, metadata, and impact assessments before executing tests. By pre-identifying which code portions are likely to contain bugs based on historical data and change patterns, the system prepares targeted test suites in advance. This preliminary action ensures that critical testing is performed on the right code sections without requiring exhaustive testing of all code.
3Adaptability or versatility
If manual tracking of code testing requirements is performed, then flexibility in test policy is maintained, but workflow complexity increases
Solution Approach 1:
The patent implements self-service automation where the system automatically determines which code portions require testing by analyzing code change history, metadata, and interdependencies. The automated system generates test plans and executes tests without requiring manual intervention to track testing requirements. This self-service approach maintains test policy flexibility through configurable rules while eliminating the manual tracking complexity that would otherwise be required.
Solution Approach 2:
The patent introduces an intermediary automated analysis layer between manual test policy definitions and actual test execution. This intermediary system interprets high-level test policies, analyzes code changes, and automatically determines which specific code portions need testing. By acting as an intermediary, the system translates flexible policy definitions into concrete testing actions without requiring manual workflow complexity, bridging the gap between policy flexibility and execution efficiency.
Data Source
AI summary
A method is provided to evaluate tests of computer program code comprising: configuring a computer to produce, in a computer readable storage device, a code filter to indicate one or more respective portions of the computer program code to respectively either omit from or to include in a determination of adequacy of results; and comparing test results with the computer program code with the one or more respective portions filtered using the code filter to respectively either omit the respective portions from or include the respective portions in the determination as indicated by the code filter.